OpenStreetMap
Although the raster maps at maps.atownsend.org.uk support zoom levels up to 24, until recently I’ve not made much use of that for showing extra data. However, there is a lot of room at these high zoom levels - as the picture above shows, enough for all the directions on this signpost here.
The name is shown as normal at lower zoom levels, then as you zoom in the directions are shown too, as you can see here.
A major rewrite here takes into account tags such as natural, reef, wetland, surface and tidal before deciding how to show wetland areas. See the picture above, which is here, and here in OSM.
The beach between high and low tide can be clearly seen here (in OSM, here). There are blue dots in the sand rather than black. A deliberate decision was taken to show more detail for areas above low tide. See e.g. here between Wales and the Wirral (see here in OSM).
There’s lots of software around to work with OpenStreetMap data - querying it, creating maps from it, using it for navigation. One area that there has historically been less support is analysing OSM changeset data. As an example, here is one of mine.
However, many years ago Toby Murray wrote ChangesetMD. That can be used to maintain a database of changeset information (tags, discussions, etc.). Martijn van Exel updated that to a recent version of Python and I made a couple more changes to support very recent Python versions and made the replication more scriptable.
Over on the planet.osm.org site, there’s a download for all the changeets generated at the same time as the weekly “planet” file is created, and also one for all the changeets and discussions as well. The changesets-only one is just over 6GB (bzipped), and the one with discussions as well is only about 250MB more, because most changesets don’t have discussions.
If you’re not interested in changeset data dating back to 2012 you don’t need to download this file; you can just start replicating from whatever point in the past you are intereted in starting from.
If you’re on Debian or Ubuntu, you’ll likely want to follow these instructions from a non-root user account of your choice (substitute that account for “youruseraccount” in the instructions).
From the virtual account created by the installation, which you can reenter by running
cd /home/${youruseraccount}/src/ChangesetMD
source .venv/bin/activate
Run something like this
python changesetmd.py -d changesets -c -g
The “-g” is optional and initialises the database to story geometric data. “changesets” in the line above is just the database name.
To load the database with changesets and discussions:
python changesetmd.py -d changesets -g -f ~/data/discussions-latest.osm.bz2
For many years I’ve used Garmin devices for both for navigation and for collecting data for OSM. As well as lots of premade maps in different styles it’s actually pretty easy to tinker with the map style yourself, although the OSM wiki makes it seem much more complicated than it actually is. It’s also pretty easy to see what OSM keys map to what values - there’s a file for points, one for lines and one for polygons that shows what OSM key and value corresponds to what Garmin feature.
I also maintain a web map style that tries to be much more inclusive than other web maps in terms of what it shows (have a look around the map legend for that), and thought that it’d be great to do the same for Garmin maps too. The web map style uses a lua script to preprocess OSM data before the CartoCSS code deals with it, which makes the latter much, much simpler. It turns out that exactly the same approach works when creating Garmin maps too, as described on this page.
I’ve added a new page to the switch2osm guide: “Manually building a tile server (Debian 12)”.
Mostly it’s minor changes only from Debian 11 and Ubuntu 22.04 - new versions of software, that sort of thing. Where new manual steps have to be carried out, the guide describes them. Debian 12 itself does have some differences - “syslog” is no longer created by default, so debugging changes slightly.
I’ve also updated a couple of other pages such as the replication ones to reflect that Debian 12 is now a supported operating system.
Separately to that, I’ve moved map.atownsend.org.uk (which uses a related may style) from an Ubuntu 22.04 server to a Debian 12 one. There was nothing wrong with the old one, but the hosting provider I use was offering higher spec ARM servers for essentially the same price as the old AMD one I was using, so it made sense to move.
Most hedges have been added to OpenStreetMap as lines. Most renderers show something like this:
and everyone’s happy.
However, from the beginning of OSM, people have mapped some hedges as areas, so for example something like this:
As there have been lots of updates recently, I’ve made a new release of all of the code behind map.atownsend.org.uk
There are releases for four repositories - the lua code that does all the hard work, the map style that determines what it looks like, the map legend, and the website for the map itself.
Here’s a summary of what’s new:
Rocky features are now shown in more detail:
The new forum “community.osm.org” has been going for a while now, so maybe it’s useful to have a look at how things are going.
There’s obviously lots that goes into creating that forum as a site where people can share ideas - there’s the forum software itself, and the people looking after the technical administration of the site, the migration of the old forum (which has just happened) and the help site (planned for later), the various implementation decisions that got us to here, and also the people looking after content moderation (which is more actively managed than before). Of all of these, this diary entry is really only about the Discourse software itself.
I’ll not comment here about the future migration of https://help.openstreetmap.org/ to Discourse and the work required within Discourse to support that, since it would be unfair to judge something that has not happened yet.
Full disclosure - I’m one of the moderators of a couple of categories in the forum, but this is very much a personal view.
The software is actively maintained, unlike the old FluxBB forum software, or OSQA, which is used for the help site.
It’s working! Some communities that might have been a bit quiet or spread over private forums are now able to talk together much easier than before.
Searching works, with some caveats around the UI (see below for that). This may sound obvious, but mailing lists search at for example talk-gb can be a bit of a pain to use - a page such as this only shows the subject and the name of the poster, not the date of the message.
You can avoid “me too” answers (but see “reaction icons” below).
There’s a translate button on every post that supports the most common languages. This reduces the “echo chamber” effect that some forums had previously (and some other OSM channels still do now).
This isn’t directly related to OSM, but many times in “software” or “development” OSM channels someone has asked “how do I do X … on Windows 10?”.
The reason that’s often a question is that, for historical reasons, much of the OSM software used to do things has been developed on Linux. That’s great for people working on Linux themselves, and the Apple Macintosh users can normally jury-rig something to run that software too.
Windows users can do the same too, by installing “Windows Subsystem for Linux”. Within Windows, search for “Windows Features”:
In Windows Features, tick the box for WSL and click OK:
Just after the New Year, there were a few reports of people getting stuck and having to be rescued while hiking in the English Lake District.
There are writeups of this in “The Great Outdoors” magazine here and on Alex Roddie’s blog here. They really are worth reading - the consensus on IRC, Mastodon etc. was that the quality of the article was streets ahead of what you might expect to see in the generalist national press.
Lots has been written about how map and app developers can try and convey information to a user beyond “this is a way of getting from A to B” (including an OSM diary entry that I wrote in December 2022), so I won’t go over that again.
However, one particular quote from the Great Outdoors article did stand out for me, and it’s this:
A spokesperson for Keswick Mountain Rescue Team, which carried out the rescues, said: “There is no path via this route – only a scramble of loose scree which also requires the walker to negotiate the rocky outcrop of Slape Crag. It’s the scene of previous callouts.”
The reason why that stood out is that “path” can mean different things to different people:
an actual signed path suggesting that people on foot are encouraged to go via a particular route. In England and Wales there’s the concept of “public footpaths” (and public bridleways), which is a legal right of foot (and horse) access across what might be otherwise private land.
some indication on the ground that people often go via a particular route
Often the two coincide, and “where you’re supposed to go” matches “where people do actually go” correspond. Sometimes, however they don’t.
(or, what’s wrong with the general purpose map on my phone that just seems designed just to show me adverts?)
I decided to write this after reading some of the comments on the wiki here and in the community forum here.
People tend to view “things that appear on a map” as “somewhere that they can go”. For example here:
someone who didn’t do their research properly might think that they can just wander along that path to the top of Everest from the south, despite the underlying data suggesting “difficult alpine hiking”. This diary entry is an attempt to try and avoid such misunderstandings.
The map at map.atownsend.org.uk shows linear paths and tracks as a single line, It shows legal rights of way (some special categories in England and Wales) and also tries to give some indication of what a walker might expect on that path.
Historically, the map at map.atownsend.org.uk didn’t change the URL as you moved around, and you had to press a “permalink” button to save the current location. This wasn’t ideal, so I had a look around for alternatives. Helpfully, the Leaflet site has a list of suitable plugins. Of those, leaflet-hash was suggested elsewhere. That seems to be still in use, although it hasn’t been updated for a while. It is spectaculary easy to use, and “does what it does” with no problems.
The challenge is that it stores the current location (for example “https://map.atownsend.org.uk/maps/map/map.html#11/54.4900/-3.1098”) but doesn’t store the current layer. That map currently has a choice of 5 base maps (test instances have 6) and 5 overlays:
The OSM wiki contains links to various downloads for Garmin maps, as well as some information about creating your own from OSM data.
For many years now I’ve used maps on Garmin devices that I’ve created myself. I’ve done this so that I show things like England and Wales public footpaths and bridleways that I’m not aware that other maps can show. Until now, I’ve never quite got around to making the mechanism publicly available, or indeed the maps themselves. I’ve finally got around to fixing that.
The “readme” here explains how to run the script yourself, if you want to do that. If you don’t, you can download the latest files for Great Britain. New maps should appear there about weekly.
Differences between these maps and other maps for Garmin devices include:
Since the release of Ubuntu 22.04 LTS, it’s now much easier to keep a tile-serving database updated. This version of Ubuntu ships with much more recent versions of osm2pgsql and osmium, so it’s easier to set up updates using those.
First things first, the new “setting up a rendering server” instructions for Ubuntu 22.04 are here. These are much shorter than they used to be because most of the software setup is done automatically as it is installed.
It’s been a while since I wrote about updates here.
Map.atownsend.org.uk is a map style that started out as a fork of OSM Carto in 2014, when it was obvious that the direction of travel for that style was towards “a map that was nice to look at” rather than “a map that was useful”.
For example, here’s the area that I linked to then in OSM Carto now:
Over at map.atownsend.org.uk I use a wider variety of icons for different sorts of objects within OSM. I previously explained here how simple changes to a basic icon make it still recognisable as the basic object (“this is a pub”) but also convey more information (“this particular pub is wheelchair accessible, and has a beer garden”).
The same principle can be used with “shop” icons - a selection of different types of shops can be shown differently if they have one of the tags commonly used in OSM to indicate “zero waste” or “eco” shops. This is done here in the lua file that processes all OSM data before loading into the rendering database. In the case of “zero waste supermarkets”, the code looks like this:
In England, the Environment Agency has an extensive network of river level monitors that are used to provide flood warnings such as (at the time of writing) this one.
However, these alerts tend to be only issued when land or property is likely to flood, and there are many paths that flood much earlier. An example is here, where if the downstream station reads just over 1m, you’ll get your feet wet. The Environment Agency doesn’t issue a warning at that level because it’s well within the normal range, and the river’s still well within the levees that have been provided for it for the last 50 years.
I’ve added a new page to the switch2osm guide: “Manually building a tile server (Ubuntu 21.04)”. The work that the Debian developers have done to include the required software inside Debian 11 is now in Ubuntu 21.04, so there’s actually less setup work needed than in previous Ubuntu versions.
Ubuntu 21.04 isn’t an LTS release, so Debian 11 or Ubuntu 20.04 are still the recommended options for servers that are expected to stay around for a while. One of the regular faux pas with the Natural Earth download site has just been and gone, but the guide includes a workaround in case problems reoccur there.
Previously I wrote about checking National Parks in Britain and Ireland for breakage, by counting the number of valid polygons in a rendering database.
A similar approach can also be used to check line features as well as polygons. It’s possible to count the number of contiguous sections of a given polygon. For example, here’s the Lyke Wake Walk, which is in one piece:
psql -d gis -c "SELECT count(*) FROM planet_osm_line WHERE (osm_id = '-1996318');"
count
-------
2
(1 row)
and here is the Sabrina Way, which isn’t:
psql -d gis -c "SELECT count(*) FROM planet_osm_line WHERE (osm_id = '-537430');"
count
-------
12
(1 row)
If something is changed in OSM that introduces an extra piece, or fixes a gap, that number will change. This script can be used to count the change in the number, like this:
Change in relation components for Sabrina_Way 537430
3c3
< 12
---
> 13
Taginfo is a way of finding out what tags and values are used within OpenStreetMap. There’s a worldwide instance here, and other instances exist, including various Geofabrik ones such as Antarctica here.
As well as describing keys and values used within OpenStreetMap, the worldwide instance of Taginfo also holds details of what other projects use particular OSM tags. The Taginfo/Projects page explains the update process.
Each project submits a .json file containing details of key and value usage, such as this one for an OSM Mug.
The one for the map style that I look after is here. Currently I’ve just submitted a list of keys, not values, but I have included a description for each one, as can be seen here for “designation”.
The first challenge was to create the initial list. There are references to tags in three main places in my map style - the lua style file, the carto .mml file and the individual carto .mss files such as this one. A bit of investigation (creating a unique sorted list of strings within style.lua on one hand and .mml and .mss files on the other found that although some keys do pass through style.lua unchanged, almost no keys that are later used in carto files aren’t mentioned in style.lua at all. That means that I only needed to worry about keys in style.lua. I created an initial .json in the correct format and then wrote a shell script to generate the “tags:” section of the .json file. I then filled in the descriptions where the shell script had just created placeholders, and checked it in.
Next, I wanted to be able to easily update the taginfo.json file after changes to the map style. Because I’ve also checked the previous generated file into github I can easily create a new one and compare the two:
sh -x ./taginfo_lua_genkeylist.sh > q.txt
diff taginfo_lua_nodesc.txt q.txt
for the most recent change the output of that is: