Gathering Indian Railways Data

Over the last few months, Sanjay and I have been gathering data about the Indian Railways. We started with an interest in schedules and worked off of some old data that Sanjay had. We spent several weekends experimenting, gathering and verifying a lot of the data out on the Internet, and we are super happy to put all of this out for everyone to use.

Trains, routes, stations. Click to view interactive map.

The dataset has three major subsets – trains, stations, schedules – all related. The train number and the station code will help you connect the three together if you need to. There are a lot of problems with this dataset. Some trains are old, have different schedules or don’t run anymore. There are new trains that are not part of this set. It’s missing several stations and doesn’t capture the right location for many. We are publishing this in the CC0 license – for everyone to use with no restrictions. Here’s something that I made with this data:

Reachability map – all destinations you can get to from a station, without switching. Click to view interactive map.

We found this an interesting experiment and a great opportunity to learn about one of the largest railway systems in the world. You can read more and download the data here, and drop a line to me ( or Sanjay ( if you have questions. If you find something obviously wrong, open a ticket, or make a pull request.

Save the map that saves lives

On April 25th, 2015, Nepal was hit by a 7.5 magnitude earthquake, killing over 8,000 people and injuring more than 21,000. The earthquake shook infrastructures, leaving many homeless, and damaging few of the most popular world heritage sites in Nepal. In the following days, aftershocks, rain and bad weather made rescue operations and relief work difficult.

I’m part of a community of mappers and cartographic emergency responders called the Humanitarian OpenStreetMap Team (HOT). HOT along with OpenStreetMap, a global community of over 2.1 million members, activates to help humanitarian agencies in times of crisis to provide up-to date geospatial information about settlements and infrastructures. HOT invites volunteer mappers to map buildings, settlements, roads, schools, hospitals, water sources, helipads, open areas and anything that can have a practical application in aiding disaster response.

Mapping Internally Displaced People Camps

The Nepal earthquake had a few of us not sleep for weeks. We mapped and worked with the single local agency in Kathmandu – the Kathmandu Living Labs – to understand the needs of aid responders on the ground, identify priority mapping areas, procure digital aerial and satellite imagery, design print maps, and communicate the needs to the global community over Skype and Internet Relay Chat. During the earthquake, there were over 30 mapping projects involved in mapping different parts of Nepal.

High contrast print map of Dudhpati village. Highlights humanitarian features – OpenStreetMap

USGS earthquake shake map on top of physical features to identify intensity – Mapbox

Within the 48 hours following the quake, over 2,000 mappers quadrupled road mileage and added 30% more buildings and infrastructure data. This is in addition to the data created by the local community well before the quake. These maps and data were used by agencies like the American Red Cross, Canadian Armed Force and the Nepalese National Army to support the effort of saving lives. Maps save lives.

Captain James Borer, Canadian Armed Forces Disaster Assistance Response Team and Captain Animesh Adhikari, liaison officer for Nepalese National Army, discuss with local authorities about roads status, in Sindhupalchok District, Nepal. Photo by Corporal Kevin McMillan.

These maps and numbers are powerful – more powerful than the Geospatial Information Regulation Bill from the Government of India. The bill proposes to control acquisition, dissemination, publication and distribution of geospatial data both within India and by Indian citizens outside the country. The Bill will make years of work of millions of geographers, cartographers, and technologists around the world, including mine, futile and potentially illegal or criminal.

Flooded Streets helped anyone find streets that are safe or otherwise in Chennai

In January 2016, when Chennai was hit by a massive flood that displaced over 1.8 million people, citizen collectives and organisations came together to the rescue. With fellow cartographers and volunteers, we created a map to identify flooded streets to facilitate transporting food, water and consumables into the city from elsewhere. Earlier, in October 2015, when parts of Afghanistan and Pakistan were hit by a 7.5 magnitude earthquake, volunteers got together to create maps, but unclear mapping laws made it complicated.

Soon after, I wrote about what change I would like to see in the HOT community – being able to work closely with traditional agencies.

The proposed Geospatial Information Regulation Bill has the effect to shutdown Humanitarian OpenStreetMap Team in India and silence volunteer mappers, cartographers, and data analysts who support humanitarian response, basically stopping them from saving lives. The Bill prevents any individual or organisation from using satellite or aerial imagery; it also blocks the use of assisted equipments to collect geospatial information and the possession of such data. This means that, from now on, no one will  ever be able to make a map of India, or parts of this country, for anything – humanitarian or business – without the approval of the authorities.

The potential consequences of the Bill are dramatic. Preventing people from making maps affects how they live in our country, it breaks down disaster response, and troubles our economy.

We need your help to save the map.

Mapping public transit in Bangalore

In February, I spent some time looking at BMTC data from to understand the network better. This post first appeared on Mapbox. 

Buses are Bangalore’s most popular mode of transport. The Bangalore Metropolitan Transport Corporation (BMTC), one of the oldest transport organizations in India, operates over 2,000 routes with a fleet strength of about 6,500 buses. BMTC recorded a ridership of 5.02 million every day in September 2015, that’s on the order of daily subway ridership in New York City.

To understand this massive network better, we need open data. Public transit data in India are not available by default, but activist groups like Open Bangalore go out and create them. I spent some time last week analyzing the network, location of bus stops, timing and distribution.

Longest route

BMTC is known for its many long routes. Route 600 is the longest, making a roundtrip around the city, covering 117 km in about 5 hours. There are 5 trips a day, and these buses are packed throughout.


Next, I wanted to look at the frequency of different routes. In the image below, stroke thickness indicates how many trips each route makes in a day. You can see north Bangalore has fewer, but more frequent routes, whereas the south has more routes with less frequency. You can also see the Outer Ring Road, which circles the entire city.


I defined reachability as the destinations a passenger can get to from a given stop without changing busses. The BMTC network operates long but direct routes covering the entire city. The map shows straight lines between bus stops that are connected by a single route. The furthest you can get is from Krishnarajendra Market to the eastward town of Biskuru: roughly 49 km as the crow flies.



Which directions does BMTC run? It is interesting that BMTC covers the city North – South (blue) and East – West (brown) with almost equal distribution.


BMTC routes are categorized into different series. Starting from 1 – 9 and A – W. I analyzed coverage based on series 2 (blue) and 3 (green) and they make up almost 76% of the entire network.

For this analysis, I used QGIS and Turf.js to inspect the route data. You can see some of the scripts on Github and the maps are all made using Mapbox Studio.

Maps, Legos, Openness

I spoke about maps, legos and openness to the wonderful folks at Khosla Labs yesterday. Highlighting the story behind OpenStreetMap, and comparing other commercial entities collecting and locking location data and why this openness is a movement than a project.

I went to the Survey of India.

I went up to Dehradun last week to do a workshop on OpenStreetMap and Mapbox at the Indian Institute of Remote Sensing.

IIRS is run by the Indian Space Research Organisation (ISRO). They are also neighbors to the Survey of India – the national mapping agency. SoI governs all mapping activities in the country, maintains the National Map Policy and enforces publishing the correct international borders.

Recently, Google Maps got into trouble for large scale mapping activities without the permission from SoI.

SoI campus in Dehradun is massive and houses the only printing shop in India at the moment. They also sell some of the treasures and I was aiming for this.

The first day, they closed the store as I walked in at 4.45PM  and refused to let me do anything except for looking at the catalog; worth the walk. They sold maps worth 850 INR that day.

The most recent map catalog. 1993

I went back the next day, right after the workshop to see more of the map room and talk to the folks there.

Gentlemen there told me that they don’t print much maps anymore. Congress government cut down funds. But SoI is not concerned, he said – ‘we are over 200 years old, most of the maps are complete now, you know.’

He pointed me to this map and said the state geospatial centers have lost their power and don’t do any printing at all. ‘People aren’t buying these maps.’ – well, duh, of course.

Everything except the toposheets are stored here

I talked about how they update the maps and they didn’t have much idea, not surprising. But I found out that SoI stores all vector data in DWG. You can buy the road network in DWG for ~ INR 6000.

I bought as many as I could.

1979 Bangalore

SoI is still the primary organisation that the planning commission resorts to gather data for urban planning and infrastructure – much of this is outsourced to small agencies across the country.

2.5 years.

I joined the Karnataka Learning Partnership in May 2012. We just launched version 11.0 of our web platform.

I’ve been meaning to write this post for a while now. Recently, it marked 2.5 years of my association with the Karnataka Learning Partnership. This is not my going away post and I’m not going away. I want to talk about how amazing the environment is for someone entirely new to the public education space. I want to talk about how extraordinary the team is. I want to talk about how different our processes are. And I want to talk about how patience, understanding and teamwork will go a long way.

If you haven’t heard about the Akshara Foundation or the Karnataka Learning Partnership, KLP is incubated at Akshara Foundation and they both work closely to improve schooling in Karnataka. KLP does great amounts of work on streamlining data collection processes, building dashboards and acting as an interface between several organisations in the education space in Karnataka and India.

When I joined KLP, it was a team of three – me, Megha Vishwanath, and Shivangi Desai, headed by the inimitable Gautam John. My role was largely around organising the wealth of spatial data that we just collected and collated from various sources. Wrangling dirty data and organising things makes me happy. KLP has a pretty strong software infrastructure history. Even though Python and PostgreSQL made it to the mix, there were other unmaintained open source software and a lot of unseen complexity in the code base. To be true to my open map heart, I rewrote the maps infrastructure. It was mostly a matter of throwing in couple of new end-points to the API and writing the front-end in JavaScript. Pretty basic. I wasn’t quite ready to dive into the madness of our existing API, so I made sure I touched little of the legacy code base. Couple of months down the line, I built a dashboard for showing SSLC performance. We worked on an array of projects right after – DISE dashboard, reports, status and a lot more. We were quite sure about an overhaul of our infrastructure and soon started talking about it. I was fairly open to the idea, but concerned, considering that there are only 3 of us and a rewrite is a lot of work.

Today, we are seven people, people I must mention because I owe them so much for understanding how diverse a team could be and accepting the fact that everybody is trying – Gautam John, Bibhas Debnath, Sanjay Bhangar, Vinayak Hegde, Brijesh Poojary, Deviprasad Adiga. Megha and Shivangi did much of the tough work of getting all the things in place for us to start. Everything we have built is based on what we have had. If there is a new feature, we have made sure that all of us thoroughly understand that it is required and how it should work. Our team is geographically disconnected – between Bombay, Calcutta, Pune and Bangalore. This is not the only thing we are doing. Each of us are involved in several other exciting projects and have weird schedules that barely overlap. To me, it is amazing to see how we have pulled this new platform in under five months. We have met twice in the process and rest of it happened over video conferences and telephone calls. I personally have had a very rough time over the last two months, which was when we were gearing up towards the crunch period. I was moved by the way the team managed to make sure that I was involved in a very limited capacity, but made sure that what needs to be done was taken care of.

To be honest, rewrites are hard. I try to avoid them as much as possible but when software becomes a liability, you have to make sure it works and continues to do so. Rewrites are usually a fairly intense process, because you are never starting from a clean slate. For the most part, you have a team which doesn’t quite understand most of how the legacy system works. Disconnect within the team is not uncommon, but it’s difficult. I have seen how the seven of us accommodated each of the ideas on the table. It’s about listening and understanding each other. It’s about respecting and realising that the other person is also committed to building the best product. It’s about being polite and being patient. And finally it’s about embracing the fact that there are only three certainties – life, death and fucked up data.

Update: We also work with some of the best technologists on and off, most recently Vamsee Kanakala and Rahul Gonsalves along with his team at Uncommon.

OpenStreetMap as Infrastructure – The Fifth Elephant, Bangalore, 2014

I spoke at the premier data conference in the country yesterday, the Fifth Elephant run by HasGeek, about using OpenStreetMap as infrastructure for geographic datasets and applications featuring Moabi – a collaborative mapping platform for monitoring natural resources extraction in the Democratic Republic of Congo. Find the summary of the narration here and a more detailed outline here. The slide deck is below.

Download (PDF, 6.12MB)