Gathering Indian Railways Data

Over the last few months, Sanjay and I have been gathering data about the Indian Railways. We started with an interest in schedules and worked off of some old data that Sanjay had. We spent several weekends experimenting, gathering and verifying a lot of the data out on the Internet, and we are super happy to put all of this out for everyone to use.

Trains, routes, stations. Click to view interactive map.

The dataset has three major subsets – trains, stations, schedules – all related. The train number and the station code will help you connect the three together if you need to. There are a lot of problems with this dataset. Some trains are old, have different schedules or don’t run anymore. There are new trains that are not part of this set. It’s missing several stations and doesn’t capture the right location for many. We are publishing this in the CC0 license – for everyone to use with no restrictions. Here’s something that I made with this data:

Reachability map – all destinations you can get to from a station, without switching. Click to view interactive map.

We found this an interesting experiment and a great opportunity to learn about one of the largest railway systems in the world. You can read more and download the data here, and drop a line to me (hello@sajjad.in) or Sanjay (sanjaybhangar@gmail.com) if you have questions. If you find something obviously wrong, open a ticket, or make a pull request.

Save the map that saves lives

On April 25th, 2015, Nepal was hit by a 7.5 magnitude earthquake, killing over 8,000 people and injuring more than 21,000. The earthquake shook infrastructures, leaving many homeless, and damaging few of the most popular world heritage sites in Nepal. In the following days, aftershocks, rain and bad weather made rescue operations and relief work difficult.

I’m part of a community of mappers and cartographic emergency responders called the Humanitarian OpenStreetMap Team (HOT). HOT along with OpenStreetMap, a global community of over 2.1 million members, activates to help humanitarian agencies in times of crisis to provide up-to date geospatial information about settlements and infrastructures. HOT invites volunteer mappers to map buildings, settlements, roads, schools, hospitals, water sources, helipads, open areas and anything that can have a practical application in aiding disaster response.

Mapping Internally Displaced People Camps

The Nepal earthquake had a few of us not sleep for weeks. We mapped and worked with the single local agency in Kathmandu – the Kathmandu Living Labs – to understand the needs of aid responders on the ground, identify priority mapping areas, procure digital aerial and satellite imagery, design print maps, and communicate the needs to the global community over Skype and Internet Relay Chat. During the earthquake, there were over 30 mapping projects involved in mapping different parts of Nepal.

High contrast print map of Dudhpati village. Highlights humanitarian features – OpenStreetMap

USGS earthquake shake map on top of physical features to identify intensity – Mapbox

Within the 48 hours following the quake, over 2,000 mappers quadrupled road mileage and added 30% more buildings and infrastructure data. This is in addition to the data created by the local community well before the quake. These maps and data were used by agencies like the American Red Cross, Canadian Armed Force and the Nepalese National Army to support the effort of saving lives. Maps save lives.

Captain James Borer, Canadian Armed Forces Disaster Assistance Response Team and Captain Animesh Adhikari, liaison officer for Nepalese National Army, discuss with local authorities about roads status, in Sindhupalchok District, Nepal. Photo by Corporal Kevin McMillan.

These maps and numbers are powerful – more powerful than the Geospatial Information Regulation Bill from the Government of India. The bill proposes to control acquisition, dissemination, publication and distribution of geospatial data both within India and by Indian citizens outside the country. The Bill will make years of work of millions of geographers, cartographers, and technologists around the world, including mine, futile and potentially illegal or criminal.

Flooded Streets helped anyone find streets that are safe or otherwise in Chennai

In January 2016, when Chennai was hit by a massive flood that displaced over 1.8 million people, citizen collectives and organisations came together to the rescue. With fellow cartographers and volunteers, we created a map to identify flooded streets to facilitate transporting food, water and consumables into the city from elsewhere. Earlier, in October 2015, when parts of Afghanistan and Pakistan were hit by a 7.5 magnitude earthquake, volunteers got together to create maps, but unclear mapping laws made it complicated.

Soon after, I wrote about what change I would like to see in the HOT community – being able to work closely with traditional agencies.

The proposed Geospatial Information Regulation Bill has the effect to shutdown Humanitarian OpenStreetMap Team in India and silence volunteer mappers, cartographers, and data analysts who support humanitarian response, basically stopping them from saving lives. The Bill prevents any individual or organisation from using satellite or aerial imagery; it also blocks the use of assisted equipments to collect geospatial information and the possession of such data. This means that, from now on, no one will  ever be able to make a map of India, or parts of this country, for anything – humanitarian or business – without the approval of the authorities.

The potential consequences of the Bill are dramatic. Preventing people from making maps affects how they live in our country, it breaks down disaster response, and troubles our economy.

We need your help to save the map.

2.5 years.

I joined the Karnataka Learning Partnership in May 2012. We just launched version 11.0 of our web platform.

I’ve been meaning to write this post for a while now. Recently, it marked 2.5 years of my association with the Karnataka Learning Partnership. This is not my going away post and I’m not going away. I want to talk about how amazing the environment is for someone entirely new to the public education space. I want to talk about how extraordinary the team is. I want to talk about how different our processes are. And I want to talk about how patience, understanding and teamwork will go a long way.

If you haven’t heard about the Akshara Foundation or the Karnataka Learning Partnership, KLP is incubated at Akshara Foundation and they both work closely to improve schooling in Karnataka. KLP does great amounts of work on streamlining data collection processes, building dashboards and acting as an interface between several organisations in the education space in Karnataka and India.

When I joined KLP, it was a team of three – me, Megha Vishwanath, and Shivangi Desai, headed by the inimitable Gautam John. My role was largely around organising the wealth of spatial data that we just collected and collated from various sources. Wrangling dirty data and organising things makes me happy. KLP has a pretty strong software infrastructure history. Even though Python and PostgreSQL made it to the mix, there were other unmaintained open source software and a lot of unseen complexity in the code base. To be true to my open map heart, I rewrote the maps infrastructure. It was mostly a matter of throwing in couple of new end-points to the API and writing the front-end in JavaScript. Pretty basic. I wasn’t quite ready to dive into the madness of our existing API, so I made sure I touched little of the legacy code base. Couple of months down the line, I built a dashboard for showing SSLC performance. We worked on an array of projects right after – DISE dashboard, reports, status and a lot more. We were quite sure about an overhaul of our infrastructure and soon started talking about it. I was fairly open to the idea, but concerned, considering that there are only 3 of us and a rewrite is a lot of work.

Today, we are seven people, people I must mention because I owe them so much for understanding how diverse a team could be and accepting the fact that everybody is trying – Gautam John, Bibhas Debnath, Sanjay Bhangar, Vinayak Hegde, Brijesh Poojary, Deviprasad Adiga. Megha and Shivangi did much of the tough work of getting all the things in place for us to start. Everything we have built is based on what we have had. If there is a new feature, we have made sure that all of us thoroughly understand that it is required and how it should work. Our team is geographically disconnected – between Bombay, Calcutta, Pune and Bangalore. This is not the only thing we are doing. Each of us are involved in several other exciting projects and have weird schedules that barely overlap. To me, it is amazing to see how we have pulled this new platform in under five months. We have met twice in the process and rest of it happened over video conferences and telephone calls. I personally have had a very rough time over the last two months, which was when we were gearing up towards the crunch period. I was moved by the way the team managed to make sure that I was involved in a very limited capacity, but made sure that what needs to be done was taken care of.

To be honest, rewrites are hard. I try to avoid them as much as possible but when software becomes a liability, you have to make sure it works and continues to do so. Rewrites are usually a fairly intense process, because you are never starting from a clean slate. For the most part, you have a team which doesn’t quite understand most of how the legacy system works. Disconnect within the team is not uncommon, but it’s difficult. I have seen how the seven of us accommodated each of the ideas on the table. It’s about listening and understanding each other. It’s about respecting and realising that the other person is also committed to building the best product. It’s about being polite and being patient. And finally it’s about embracing the fact that there are only three certainties – life, death and fucked up data.

Update: We also work with some of the best technologists on and off, most recently Vamsee Kanakala and Rahul Gonsalves along with his team at Uncommon.

Bringing together map enthusiasts in Bangalore.

A few months back, Kaustubh and I started GeoBLR – Bangalore’s monthly spatial gathering. Few weeks back we did the fifth meetup. I have been wanting to write why I’m organising GeoBLR and more importantly, why I am taking it slow.

We started off GeoBLR to keep it quite different from other events in town. To begin with, we don’t necessarily have someone speaking at all the events. The GeoBLR events are more open in terms of what we want to do. GeoBLR runs closely with the DataMeet on issues centric to spatial data and maps. We had several discussions around the spatial data situation in India and how building proper relationships at the government and non-government level is important for the community to gain access to the datasets.

There has never been a space in Bangalore to talk about issues like this. When proprietary companies around the world are trying to be the single source of ground truth, opening map data is a critical task. Improving OpenStreetMap is at the core of our agenda. GeoBLR will be a space for people to come and talk about maps, discover new tools, identify missing datasets, find them, and solve spatial data related issues.

I have been getting fair amount of requests to turn GeoBLR into a series of workshop and I’m not keen on that idea. GeoBLR, in a way, is an experiment for me. From simple things like ‘which day of the week/month is best for people to attend the event’ to ‘what model the event should focus’, I’m learning a lot and subsequent events get more clarity.

If you want to learn more, join us!