OpenStreetMap logo OpenStreetMap

HOT Validation

Posted by Adrian Shobrooke on 22 November 2025 in English.

My diary entries all all my own thoughts and do not represent OpenStreetMap, The Humanitarian OpenStreetMap Team (HOT) or any organisation using the HOT systems. Any errors are all my own work.

This first diary entry is based on my response to a mapper on the HOT Slack Channel asking about validation methods.

When it comes to HOT mapping and validation, I’d say there are not really any standards, or training to judge mapping against those standards. It’s a constant frustration of mine. However, the use of the data should be considered when validating tasks. If a road in Africa is a few meters out from a vehicle GPS while driving, does it really matter? The driver is going to follow the track. Relative positions are more important - do things match what might be seen in the field?

Skipping a task that you don’t feel experienced enough to validate seems a mature approach to me. I also try to map a few tasks in a project before validating tasks to get some understanding of the problems mappers may face.

I’ve only been seriously HOT mapping with some validation since early 2024, although I’ve mapped in a small way since April 2019, including my own locality. I’ve developed the thought that there are three basic types of validators;

  1. Validators who simply approve or reject tasks. These validators are usually new validators and sometimes they are still new mappers! These validators may give basic feedback such as ‘more mapping needed’. This is not very useful. It is not targeted to the task problem or the mapper.
    I’ve seen projects requiring roads, buildings and waterways with validated tasks without rivers mapped where rivers are visible. This is task fixation on buildings and roads because this is the requirement of many projects. It shows that mappers and validators are not reading the instructions. Sadly, some instructions are very poor, but that’s a separate discussion!

  2. Validators who reject tasks with feedback that is relevant to specific problems and named mappers, often with image based examples. I try to be one of these. I’m not just looking to get the project finished, but to develop the mapper skills. This is more beneficial to the long term mapping quality. I call this Validation for the Mapper. I’m trying to develop mapper skills to make future validation easier and hopefully complete projects more quickly.
    It is probably the most time consuming and frustrating type of validation as many comments are ignored as most mappers never review the history comments. Several cycles of new mapper submissions and rejections with no mapping changes may occur before a mapper does read the history and correctly maps the task.

  3. Validators who used to be type 2 validators. but may have become frustrated with the cycles described in 2 and just fix all the problems themselves. The project makes more rapid progress, but the mappers have no idea if they mapped well or not. I call this Validation for the Project. It gets the project completed with good quality, but may take up unnecessary validator time and it does not improve mapping skills in others.
    This can result in more validation work over the long term for all types of validator because the mappers may think that what they have mapped was good enough when it was not.

I usually fix small problems such as two or three missed or misshapen buildings, but I will state what I did in feedback, sometimes with images. This is very time consuming. I won’t do lots of mapping as a validator as I value my time. I’d rather map well first time than have my time wasted fixing up large amounts of poor mapping. I reject such tasks with appropriate feedback.

The biggest issue with rejecting tasks is that most mappers only map once. The statistics are obvious. https://tasks.hotosm.org/ shows over half a million mappers, but the number of on-line mappers can be single figures. It is probably only over 30 when mapathons take place. Mapathons usually mean many of new mappers at once and the associated problems. However most of us have likely started with a mapathon, so they are useful events.

For validation training, I suggest you join one of the early in month Missing Maps London on-line mapathons. These are focused on training new mappers with the iD Editor or JOSM. There is usually some validation training depending on numbers and trainer availability.

The mid-month Missing Maps London on-line events focus on experienced mappers and potentially more challenging or restricted projects. These and other mapathons can also be found on the OpenStreetMap Calendar.

There are also recorded webinars and tutorials on the HOT YouTube channel to help mappers and validators. Some of the videos are short, some are very long! Additional webinars may be notified on the HOT Slack Channel from time to time. Some mappers and validators also respond to questions asked on the HOT Slack Channel and past conversation may be searched.

Hopefully we’ll see you on-line. Happy Mapping!

Email icon Bluesky Icon Facebook Icon LinkedIn Icon Mastodon Icon Telegram Icon X Icon

Discussion

Comment from rphyrin on 25 November 2025 at 08:50

Well, it seems that I have been a type 3 validator since the beginning.

At first, I wanted to be type 2, but I didn’t have the means to properly show other mappers where the errors that needed fixing were located (i wish we had this kind of feature built directly into the Tasking Manager). Before such a feature exists – for the sake of “developing mapping skills” – I think it is better for now to communicate that feedback outside the OSM/Tasking Manager platform instead of rejecting tasks one by one.

At first, I wanted to be type 2, but it actually resulted in the project stalling, as other mappers left and never returned. Regarding the idea of “improving mapping skills in others,” some people simply do not see mapping as a valuable skill in the first place, and this might be the last time they ever participate in a mapathon.

There is also probably a type 0 validator who rejects tasks purely for the sake of “stealing” someone else’s points, usually in the context of gamified mapathon scenarios (especially when those points directly correlate with monetary rewards).

Comment from SColchester on 25 November 2025 at 10:14

Very insightful Adrian!

Comment from Adrian Shobrooke on 25 November 2025 at 10:50

Hi rphyrin,

You’re not alone in wishing for direct mapper communication in TM. I’ve also directly communicated with mappers and encouraged them to join the HOTOSM Slack channel. This is for those that appear to have better general skills and have responded to validation comments. It does actually happen, sadly rarely.

Yes, many mappers never return, but some do. If I can improve those, I think I’ve got a win.

Re stalling projects, perhaps some of the stall is due to repeat invalidation cycle. I’ve just been part of a meeting which may result in some simple TM improvements, but we’ll have to wait a while after implementation to have enough data to prove there has been a beneficial change.

Yes Type 0 likely exists, but I don’t know how to identify them. Poor validation is always a risk. I know some is picked up prior to project archiving, but that will be items like unsquared buildings, feature clashes, shared nodes. Simple mapping errors not picked up.

Gaming the system. Sadly that will always happen in a personal Key Performance Indicators (KPI) environment. KPIs are usually developed to measure behaviour change, but frequently poor KPIs develop the wrong behaviours. I’ve seen a mapper split tasks mutiple times, creating a larger tasks mapped count. I’ve thought about sum of area mapped instead of task count. Splitting tasks would not change the total area.

I have listed your GeoScribble suggestion in a list of features to be added to TM to aid communication. Unfortunately I think it is unlikely to happen as much of the mapping environment is outside of TM. I can’t see how to get information back from JOSM into TM, iD editor might be better as it is wrapped by TM and maybe a separate layer could be captured into the TM database instead of OSM.

PS. How did you come across diary my entry? I’ve no idea if/where new diary entries are highlighted.

Comment from SeverinGeo on 28 November 2025 at 15:52

I agree that the TM is meant to organize the mapping and allow for some validation steps, but it does not embed a validation process or checks. Some validation steps also need to be done on a larger scale than the tasks, even if you can select several of them. From my experience, I would also like to be able to grant or restrict access to specific users independently of their mapping experience. This experience should not simply be calculated from a number of changesets (if I am not mistaken). Communication within the TM is good when you use the @username feature to notify a mapper. I tried sending feedback to every mapper participating in my projects, and I agree that receiving an answer is a win.

Regarding advanced validation, I created a specification validation course in the UN Maps Learning Hub (it’s free, but you have to sign up for Moodle). Last year, I organized a workshop consisting of ten two-hour sessions to cover the course. The workshop was open to anyone interested, and the videos are available on YouTube. There are 66 videos because I prefer to cut the recordings so that each one covers a specific topic rather than having ten two-hour-long videos. Buildings may not be covered tough.

Comment from Adrian Shobrooke on 28 November 2025 at 17:57

Thank you for responding Severin,

I frequently wish the a Validator was a specific role with separate access controls, but I think that would require significant TM code changes. The role can be managed reasonably well by the use of teams, but that requires good project management authoring. Which is itself relatively rare.

I dislike the changeset approach to estimating mapper capability, but It’s what we have to live with. A good validator should also have good communication skills. Being factually correct with feedback can be off putting - I know as I’m likely to be quite direct, I don’t do shades of grey very well.

I will look up your videos. I think you in advance for breaking them up into smaller chunks. A 2 hours video of listening to others telling me something without me practising the skill is not my idea of a good way to learn.

Comment from Johnwhelan on 30 November 2025 at 15:53

Your dairy entry hit osmweekly. https://www.weeklyosm.eu/archives/18306

Based on experience validation feedback works best the faster it is given. Catch someone within twenty minutes and they’re more likely to act on the feedback. They also are more likely to complete more tasks on the poject, so sitting on a project and validating it quickly means it is more likely to get completed. Tone on validation comments makes difference as well. Two projects very similar, I validated one and another mapper who was a bit black and white the other. Similar rapid feedback, mine got completed in half the time.

After a month forget it, they may have changed their practices anyway.

This morning I received a comment on a changeset from 2019, apparently it didn’t match the latest ideas of tagging.

Be aware that HOT validation doesn’t quite follow OSM rules where the process is changeset comment, wait for feedback, then correct. Validators tend to just correct possibly with feedback.

Projects with buildings, I don’t bother validating them. It’s a waste of my time. I can draw a new building correctly tagged in JOSM with the buildings_tool in two or three clicks of a mouse. Cleaning up a misshapen building drawn in iD takes more mouse clicks than to redraw it.

There is another type of validation which is load up a fairly large area of the map then check for duplicate buildings with the tool. Run JOSM validation on the area, it picks up a fair number of errors that can be corrected. You really need ECC memory to avoid errors and a fairly quick processor. dellrefurbished.com 5820 workstations work well.

The downside is the errors caught are not fed back to the project, but the quality of the map is improved.

Another way to improve building quality is using JOSM and the buildings_tool plugin. There was a mapathon held in Calgary some years ago that mapped buildings that got negative feedback for the quality of buildings mapped. For once in my life I actually organised a mapathon, we had five new mappers and one experienced mapper. It took a bit of time to install JOSM and the buildings_tool but once done they mapped very quickly. We mapped around three times the number of buildings that the first mapathon mapped and there was no issue on the quality of the mapping. I’ve noticed in a mapathon quite often a mapper will only map a handful of buildings.

The other advantage of JOSM is it does a fair bit of validation on the upload so untagged ways etc. get warned against. I think iD does some but JOSM is stricter and gives immediate feedback which feeds into more tiles mapped.

Oracle’s java is sometimes prohibited on corporate machines but Microsoft’s OPEN JDK works fine. You can also run JOSM from a USB stick if need be.

Have fun

John

Comment from Adrian Shobrooke on 30 November 2025 at 19:10

Thank you John for your thoughts. I agree that very prompt validation is best. If a mapper has badly mapped a few tasks, it is very hard to get them on track. If they come back for more mapping.

I’ll correct a couple of buildings, but more than that I won’t. A waste of my time as you say.

It took me a while to install JOSM, only on my hope PC. Many work environments are risk averse. Most users cannot install anything. Sadly many work mapathons based on iD editor lead to a lot of poor mapping. OSM rate limiting goes some way to prevent these issues. Mapathons are also a good way to introduce mappers to the wider OSM environment, so keeping up the communication with new mappers is important to encourage new blood.

Have fun? Oh yes, been my signature for years.

Adrian

Comment from Patrik_B on 1 December 2025 at 10:06

Great diary entry Adrian!

@SeverinGeo thank you for sharing your training materials! I will check that out

Log in to leave a comment