The visualisation below shows the regions of the world where the HOT community has contributed edits to OSM, which is one way in which we can show the impact of our community. The chart visualises contributions before 23rd Sept 2016. By this date, 32,000 people had contributed at least one edit, accounting for a total of 182,000,000 edits. This took an estimated 240,000 labour hours.

As mentioned before, I’ve been showing the visualisation in talks for a while now, and I regularly receive messages by people who would like to use it for their own slides, for mapathons and training sessions, and other uses.

There is also a PDF version (11MB), a high-resolution PNG (1.3MB), and a folder with older versions if you want to do a visual comparison of map growth. Send me an email if you would prefer a version without annotations – I simply ask that you provide credit when you’re using it.

… See full entry

A few months ago I posted a draft specification for an OSM quality assurance tool. The first beta for the project was launched last week, it is now called OSM Analytics. Cristiano Giovando posted an announcement on the HOT blog.

The code for frontend and backend is on Github; it’s a very nice Javascript codebase, making use of many existing OSM frameworks and infrastructure pieces. We welcome your bug reports and pull requests!

I also gave a brief introduction to the tool and its uses at the most recent Missing Maps mapathon in London, there’s a recording by the BRC maps team on YouTube. Unfortunately we had wifi problems at the venue, so it’s not a very fluid presentation, but Chris Glithero took care to edit out the gaps so it’s still a decent flow.

Today I took some time to update my list of HOT mapping initiatives – a bit of a messy process because there’s no official listing. These days I simply review new projects in the OSM edit history that have a minimum number of contributors, and label them with a simple term. The intention is to identify groups of projects that have a common theme. Typically these are disaster events, larger mapping campaigns like Missing Maps, or organisations that organise projects for their members. Of course the boundaries between them are blurry, e.g. Missing Maps is really a meta-initiative across many discrete projects.

Here’s a timeline of the initiatives I’ve identified so far – let me know if I missed any! There’s also a PDF version, in case you want to include this in presentation slides.

Andrew Braye, Jo Wilkin and I spoke at the Oxford Internet Institute earlier this month as part of their ICT4D seminar series. Andrew gave a high-level overview of HOT and Missing Maps, Jo spoke about data collection in the field, and I spoke about my HOT community research. We had a great time! The video is now on YouTube and is about 1h long.

Our slides are all online: Andrew’s intro to Missing Maps (Google docs), Jo’s field mapping discussion (Google docs), and my community engagement analyses (PDF).

… See full entry

I’ve tweeted versions of this in the past (Feb 2015, May 2015), and used it in talks. Here’s an updated version with data up to 13th January 2016.

In total this covers around 120 million changes to the map, by almost 20,000 contributors across 1,000 projects. This required an estimated 165,000 hours of volunteer work! There’s a monthly breakdown of this activity in this Google spreadsheet: “2016-01 HOT activity timeline”.

I’m keen to do an animated version at some point! Also, could a cartography geek please recommend a suitable projection for this map? Atm it’s just the default WGS84, with apologies :)

… See full entry

Blake sent an email to the HOT Community WG asking for ideas on how to increase the number of regular HOT mappers. This is squarely in my research domain, so it was a fun question to respond to… I suggested things that now to me are pretty obvious, but weren’t just a year ago.

My suggestions follow, in no particular order.

---

Identify existing communities with a propensity for this kind of work: GIS experts, aid org volunteers, and others who are similarly embedded in existing contributor communities.

Partner with more large corporates, but choose the right ones: where there are already some HOT mappers on staff, and people who can coordinate company mapathons. Don’t go through exec, instead identify existing mappers who care. (Cf Arup, others)

Set up regular online events where people can come together in a more social fashion. Online chats, twitch streams, etc; play with the format.

More regular mapathons around the world, organised by new groups; learn from Missing Maps in London, they’re now world experts in how to do it well.

Better communication of ongoing needs: e.g. a weekly (or monthly) email which provides background info on current projects, incl mapping tips about specific pitfalls.

A well-managed validator process, similar to Missing Maps in London: try to ensure that new contributors receive good and constructive feedback early.

Better guidance on the TM homepage: instead of “pick from infinite list of words”, try to emphasise different aspects that may resonate with particular types of mappers. The easy ones: degree of urgency, type of purpose, participating organisations, “almost done” projects, projects in specific countries, … I’m sure there are loads more aspects. (Then measure which of these things people actually respond to.)

Find means of identifying people who are actually interested (or likely to be interested), and then give them more specific support. For example, make sure they’re connected to a mentor or a peer group.

… See full entry

Joost asks in a direct message:

I’m organizing a Missing Maps event in Antwerp. One of the co-organizers wants to try giving a tweaked JOSM version on a USB stick to all the participants (preloaded settings etc) and use JOSM as a default editor. […] Did anyone try this at an event? Did you have a look at first timers using JOSM having a higher or lower OSM/MM retention? (It might be too much self-selection to really prove anything…)

I thought this was an interesting angle, and it connects with some of the work I’m currently doing, so I had a look at the data and am posting the results here. The short answer, based on a small sample: we’ve actually seen a difference in retention! However not in the way you might expect. I was surprised.

Before I begin I should say that I’m very interested in other perspectives on this question, particularly actual teaching experiences. This is a good scenario where statistics might be misleading, and where it helps to have actually talked to the mappers and observed what happened. Looking forward to people’s comments!

Preliminary caveats

It’s actually really hard to measure this well and generalise from past experiences, because every mapathon has its own story; different people attending, different things going right or wrong, etc. Different editors are also often used for different kinds of work: JOSM often gets used for field paper tracing and validation as well as satellite tracing. Unfortunately I haven’t been to most of the JOSM training sessions I’ll quantify below, so I don’t know what people actually did!

Furthermore, editor choice affects all kinds of follow-up considerations that may affect the outcomes of such a study; e.g. I’ve seen people forget how to launch JOSM a month after they first installed it, or OS updates causing java versioning issues, all of which is not something that can happen with iD.

And so on. You get the idea: many factors to keep in mind when we look at these numbers.

… See full entry

I had a recent shift in perspective in my research of HOT contributor engagement. I will try to articulate a growing intuition: a sense that current-generation HOT tools and processes would do well to also recognise the secondary benefits HOT volunteers get from their participation, for example their social experiences. I think we currently don’t necessarily create social online spaces for new contributors, and that is an omission of some consequence. In contrast to Wikipedia and comparable platforms, HOT contributors are not also typically the primary beneficiaries of the collective output. Secondary benefits can make up for this lack in direct utility: they have important motivational power.

As usual, please let me know your thoughts on this. It’s informed by my own experiences of the HOT and Missing Maps community, and I am very curious to learn what I might have overlooked, how else to express it, or find other ways to look at things.

… See full entry

Inspired by recent Transifex discussions I though it’d be interesting to see what languages our contributors actually speak — to the extent that we can easily find out. It turns out that as of May 2015, iD now submits a “locale” changeset tag — JOSM has been sending that information for a while already.

The top entries across both editors are shown below, for May-October 2015 (inclusive). Note that a locale with a small number of contributors is not a locale that matters less – as we’ve established before, a small number of contributors can make a significant impact on the maps of a region.

There’s also a Google spreadsheet with separate tabs for iD and JOSM contributors, if you want to dive further into the data: “HOT contributor locales, May-Oct 2015”. Or as CSV files: combined, iD, JOSM. It’s interesting to compare their distributions. E.g. iD has a much longer tail, which I guess is not a surprise – browser locale vs limited JOSM translations?

UPDATE: Ilya Zverev has kindly amended the spreadsheet to also show translation progress for each locale!

… See full entry

Harold D. Craft’s classic visualisation technique applied to a timeline of HOT project activity. As previewed before and used in the Missing Maps review, but updated for early November 2015. Click through for the full version.

A line per tasking manager project, its height along the implied z-axis is proportional to the number of project contributors on the respective date. Projects tend to be most active in the beginning, and then activity tails off. However some large projects are eternally active… MapLesotho (#597/599) is among these, partially covering the equally long-running South Sudan (#591). Remarkable how massive the Nepal contributor community actually was, in the scheme of things – the big spike in the centre would be even taller if the work hadn’t been spread across multiple projects (between #994 and #1090).

There’s also a PDF version if you want to print it out.

… See full entry

Pete asks:

On a skype today, Kate Chapman said that analysis after the earthquake in Haiti, she found that ‘40 people did 90% of the work’ within the community.

Is the workload more evenly spread throughout the community when it comes to Missing Maps tasks as opposed to HOT tasks? Is it more evenly spread during non-emergencies?

I thought I can look at this quickly because I’d done similar work around participation inequality in the context of OSM; in the end took much longer than expected and I can’t see that I found a simple answer. If anything it serves as a good reminder why it’s challenging to produce meaningful statistics for social spaces: the devil is in the many nuances. This writeup here can probably give you some impression of that.

Unfortunately I don’t have contributor statistics for Haiti since it predates the tasking manager, instead I will compare Missing Maps with other large HOT initiatives, most importantly Typhon Haiyan in the Philippines in 2013, but also the Ebola activation in 2014, and Nepal in 2015.

The impatient can skip the more in-depth discussion and jump to the conclusion section at the bottom. Note that this is just a quick exploration, not a thorough statistical analysis. I’m sure I’ve overlooked things, so please give feedback.

As usual I’m looking at labour hours as a measure of work. The results are probably not that different than if I’d used map edits, however I find they’re a better reflection of the effort spent on contributing. Time moves at the same pace for everyone, while the same number of clicks could yield a different number of edits depending on what you’re doing. Edit counts are also a potentially confusing measure because there’s no standard way of counting them: as the number of version increases of geometries, or the number of changesets? Etc. So here’s a key limitation of these stats: I’m not actually looking at map impact, instead I’m looking at a measure of individual effort.

… See full entry

Stats & charts I put together for a session at the Missing Maps powwow in Toronto (PDF)

I had no time to prepare for this, so on the plane from London I simply went through my work of the last few months and collected things that seemed appropriate for the occasion. I intended this as a quick 30-minute review, but it ended up stimulating lots of debate throughout… so the session took 2h instead. For a conference setting this would have been disastrous, but since this was a team gathering it was actually quite useful. Data visualisation as conversational catalyst!

See also Dale’s recap of the powwow.

… See full entry

A few weeks ago pedrito1414 asked me to determine the share of HOT contributions that are attributable to Missing Maps. It took me a while to get around to it… but I finally did. If you follow me on Twitter you may already have seen a couple of these, but here’s the full set.

… See full entry

I only recently realised that HOT contributors need to mark at least one task as “done” to be listed as project contributor in the tasking manager. This made me wonder: how many people start contributing to a HOT project but never finish their first task? What proportion of all HOT edits are contributed in this manner?

Summary: about half of all HOT contributors never complete their first task on a project, although they do contribute to the map. These “partial” contributions account for 10-20% of all HOT edits.

Here’s a timeline of the number of monthly HOT contributors, compared with the number of those who completed at least one task:

And here the corresponding timeline of the number of edits contributed by both groups of people:

… See full entry

If you’re subscribed to the HOT mailing list you’ve seen a recent invitation to help develop a funding application for the Knight Prototype Fund, coordinated by Russ and Blake. The intention was to discuss project proposals that may be suitable for this grant. The initial IRC meeting then developed into a larger conversation around current HOT needs for better tools: the resulting Google Doc with meeting notes lists six project ideas.

The strongest candidate was a proposal to develop a HOT/OSM tool to support Quality Assurance (QA). You can read some details in the grant proposal writeup, however it’s a fairly high-level text. Informed by our discussion I also developed a draft specification, with a more detailed list of considerations and potential features.

I’m posting this draft specification here to get your feedback, and to hopefully stimulate some debate about what a good QA support tool might look like. The proposal is a result of conversations with HOT practitioners, and based on my own use of HOT and OSM data. However there are likely many community members with further ideas, and some may even have worked on HOT QA initiatives. We would love to hear from you! In particular we would love to hear from validators, and from existing users of HOT data. What specific data quality concerns arise in practice?

(I should also state that I don’t have a deep understanding of the Humanitarian Data Model – there are likely some useful concepts in there that could be more emphasised in the spec.)

Considerations

Our general ambition is to make HOT progress more visible. More specifically, the proposal aims to support our existing QA processes around HOT validation. Crucially it further aspires to provide a means of demonstrating HOT data quality to prospective users of the maps.

Aims of the proposed QA support tool:

… See full entry

In a response to an acute shortage of validators, Missing Maps in London are now training people up at their monthly events: first to learn JOSM, then validation. I think that’s great! It’s particularly fitting that validators are trained from the same volunteer pool as new HOT contributors. That way, at least in principle, their numbers can grow together. While currently validation is often on the shoulders of a few expert insiders, in this new model it instead can become an important training aspect for larger numbers of highly engaged HOT contributors. Becoming a validator could be an important rite of passage for certain new contributors.

… See full entry

(Sneak preview of new data visualisation work. While playing with HOT-related time series data I realised that Harold D. Craft’s classic visualisation technique was a very suitable means of illustrating certain temporal patterns…)

(Hallo! I’m Martin Dittus, a PhD student at UCL. You can read more about my research in an earlier post.)

The volunteers of the Humanitarian OpenStreetMap Team (HOT) and its affiliated projects have spent many thousands of labour hours on the creation of new maps for humanitarian purposes. Yet mapping all the undocumented and crisis-stricken regions of the world is a formidable task. The 2014 response to the Ebola epidemic illustrated this well: even after months of work by thousands of volunteers, the new maps of Central and West Africa are still nowhere near complete.

Many people within HOT now believe that this can best be addressed by growing the community by a few orders of magnitude. An MSF article about Missing Maps articulates this ambition:

To reach our goal, we need the Missing Maps Project to be the biggest instance of digital volunteerism the world has ever seen.

So let’s say we’d want to grow HOT to a million volunteer contributors. How can we train new contributors at that scale? What are our barriers to entry? How can we retain contributors once they’ve had first experiences? Etc… many open questions.

As a first step let’s learn from existing experience. How does engagement compare across the different mapping initiatives right now? Let’s start with a simple comparative study.

Comparing three large HOT initiatives

I’m particularly interested in the engagement profile of first-time contributors: people who may have OSM experience, but who have never before contributed to HOT. How much work do they provide in the first couple of days? How long do they stick around?

In this post I’ll compare the first-time contributor engagement profiles of three initiatives. Each has a different purpose, and a different mode of organisation:

… See full entry

Hallo! My name is Martin Dittus, and I’m a PhD student at the ICRI Cities at University College London. I research community engagement in the Humanitarian OpenStreetMap Team (HOT), a volunteer initiative with thousands of contributors. At its core this is quantitative work, and my main outputs are statistics and data visualisations. I also spend a lot of time with the HOT community, am a contributor myself, and have spent much of the last decade with a range of similar community organisations.

I like that my job allows me to combine my experience in large-scale data analysis with my personal interest in community organisations. I spend a lot of time exploring data sets, producing things like this:

… See full entry