Hi dieterdreist, we’ve posted an update to Daylight in @migurski/diary/394109 where we commit to exposing the specific OSM tags considered for distribution in Daylight. For Facebook’s use, we’re required to meet certain legal standards in jurisdictions around the world so we don’t use OSM boundary definitions.

We’ve published an update to Daylight, more information in this diary entry.

Allan, thanks for starting your time on the board with this sustained effort to reach distant corners of the community. It’s inspiring to see you take your chairmanship responsibility so seriously and to gather so much input beyond just the visible contributions.

This point about obstacles to joining a “do-o-cracy” is especially significant and I’d love to have seen it emerge as a more widely-shared concern:

Respondents in Africa and Asia underscored the cost of volunteering, noting that in lower-income countries the cost of Internet access and need to work more than one job to support a family constrains time devoted to volunteer mapping. This is an obstacle to geographic diversity.

I wanted to summarize some questions and responses from other channels here.

• #daylightdistro_feedback channel on OSMUS Slack
• Responses to post on Talk@

Is Facebook planning to create new planet files like this frequently for internal use?

We are unsure yet about the public release schedule we want to commit to because a lot of it depends on community feedback.

Is Facebook fixing data errors on OSM?

We do correct OSM upstream for the errors that we find. When our human review process catches a map error, we do two things:

1. Hold it back from release to our display maps
2. Fix the error upstream in OSM.org

What’s the breakdown of the 8GB size difference between Daylight planet and OSM planet?

Probably the most significant source of that difference is that we completely ignore & exclude tags that aren’t relevant for our display map needs.

So DLD is a sort of friendly fork for FB and others to ‘read’ from and where ‘writing’ is selectively pulling from upstream?

Not a fork. But friendly, yes. We’ve observed that many companies using OSM are uneasy about the potential for bad edits. Oversight from the community fixes most issues in time, where Daylight can pick up just the good fix and not the original error.

Will information about rejected changes be released to the community?

We’re discussing internally how we might go about doing this.

Is there any plan to publish sub-region data?

Not at this time, but the PBF release format makes Daylight compatible with major OSM tools that can be used to generate regional extracts.

That’s a great suggestion Mikel. I’ll try to learn more about what we’ve done and planned with OSMCha so far.

Hi Samith, I don’t know the answer to your question but here’s a group who’s done this before: osm.wiki/San_Francisco_Building_Height_Import

Hi Bill! We haven’t done any routing-specific checks for this distro. Could you say more about what you’re looking for?

Thanks for your feedback, Luc! For the time being, the data is being provided “as-is” since it’s a snapshot of a hybrid human/machine process with a lot of individual judgement calls. When we see that there’s demand for Daylight we may decide to release it on a regular basis and further open up its process.

I’ve followed up on this and a few other talks from Matt Broberg, thanks for sharing. They’re excellent. He made some interesting scale comparisons from his IRC talk:

Thanks Tom. I’m still picking at Vagrant/Virtualbox to see if I can figure anything out there but the logfiles aren’t yielding anything I recognize as a clue.

Hey @scruss, that’s true. I believe Heather is recommending a paid community manager for OSM as well.

Heather, thank you for posting this. I appreciate your sustained push for better representation is the OSM community, and I agree that active community management is valuable. It would be interesting to migrate selected official communications away from un-moderatable channels. For example, legal-talk@ and osmf-talk@ could be moved to forum.openstreetmap.org topics where active moderation (including deletion of messages) is technically possible. For a comparable illustration, here’s a thread discussing Metafilter’s approach to moderation:

In our reading, moderators engage with egregiously offensive behaviour (e.g., name-calling, personal derails, trolling, *-isms) by deleting the respective comments. In other cases, they remind members within threads of best practices (e.g., put in NSFW links when appropriate) and site guidelines (e.g., no rehashing of previously discussed topics) in a preventative mode.

Moderators also engage with the community in more pro-active ways. They shape the discussions by bringing in their individual perspectives and they care. They point out that people should look for help if needed and where to find it, they show solidarity and they appreciate content posted by others.

…

We chose to look at MetaFilter because the quality of discourse here has been applauded when it comes to people discussing controversial topics on the internet and the moderators appear to be a core reason for the constructive conversations on the site.

Hi Baloo, I appreciate your comment here.

You raise an excellent point, which ultimately comes down to trust and predictability: are commercial interests in opposition to those of OpenStreetMap? I wouldn’t be doing any of this if I didn’t think the two were basically aligned behind the creation and distribution of free geographic data for the world for people to use in creative, productive, or unexpected ways.

Hey Paul, thanks for posting. Are you asking because you’re curious, or because you’re trying to communicate to me that permission from the board is required for doing board-related work on paid time? I have permission and support from my employer to participate in the board, and I intend to follow any conflict of interest rules that apply.

Thanks for this feedback, Frederick!

Pure separation of personal, community, and participant goals is not in OSM’s best interests: the map has always had a job to do in the world beyond just existing and commercial use is a part of that. I’d like to see a future more like that of Linux: a healthy, stable organization and a range of diverse participants acting in concert without privileging narrow forms of participation over others. I imagine this is also important to you as an owner of a company similarly reliant on OpenStreetMap data.

You linked to my Talk@ post but I’ll mention it again: I’ve set up a short series of open office hours in a wide range of time zones to answer any community questions that might come up. There are three left: tomorrow, Sunday, and Tuesday. So far I’ve had one very excellent and wide-ranging conversation about communication venues in OSM which reminded me that forum.openstreetmap.org exists as an interesting potential place for the OSMF board to make itself available for community feedback and conversation. Since you’re still a prominent member of the board, would you be open to using some of your remaining time to make communications channels between leadership and the community more explicit?

Hey Ben, thanks for this detailed response to my post!

I’ll just write a few words because I’m about to get on a plane. I think you’re pretty close to the point I am hoping to convey, but maybe coming at it from a different angle? I don’t believe OSM is irrelevant now (as evidenced by attendance at conferences like SOTM and the interest of the robot and crisis communities), but I think its leadership and the Foundation are stuck in a holding pattern that I trace back through the history of the project. Briefly, because the OSMF has been unable to adapt to the emerging needs of new members which I connect to CoC’s and improved license communication, those new members find reasons to limit their involvement in OSM and ultimately hesitate to buy in.

Money and all that it represents is a shorthand for this. The companies and consortiums who could conceivably provide this money find OSM unapproachable for the reasons I’ve described, and so they either get lightly involved or steer clear entirely. I’ve spoken about OSM with numerous people from large businesses and funding organizations who either muse about creating their own less-hostile alternatives, or dip their toes in tentatively before drawing them back. OSM’s unwillingness to open the door to participation on unfamiliar terms blocks the project from access to the kinds of funding you describe.

I understand that a number of people have taken offense at the term “craft”, and I am also aware that I’m making a disprovable prediction regarding project doom. I think it has been necessary to shake the tree enough to flush these topics into the open.

Also, BushmanK, I’d love to hear more about the proven fact that HOT involvement has not succeeded in sustaining the map!

It’s a less important point, but the XML output uses seven digits of precision after the decimal point, and for data sets like land use it might be more appropriate to use 3-4. Less important, though.

One interesting outcome will be to gauge the suitability of the OSM tool chain for data at a different scale, like forest outlines vs. roads. How many of the tools assume street-scale in all things, and enforce that in minimum zoom levels for editing (iD) or high-precision data output (planet XML)?

On further thought, it's probably worth noting that Mapnik's processing pipeline, the way that styles must be specified before geometry is passed through, is a major constraint here. I could imagine a future modification to Mapnik where it behaves more like HTML + CSS and applies the rules in-flight as they are needed, and removes the need to prepare all possible combinations. I expect this is many versions off, though.

Hi asciiphil,

Cascadenik is my project, thanks for your comments. The combinatorial explosion issue is very much something I've run into when developing stylesheets, and like you I've found myself pushing a lot of logic in SQL to fix the problem. Here, for example, is a SELECT statement from one recent project: http://dpaste.com/hold/190143/

The "is_tunnel" and "is_bridge" parts help handle the various different ways of saying "yes", and the "kind" result column groups together many different types of roads. There's also a subselect in there you might have noticed: it groups the lines from each layer into single MULTILINE geometries, which helps deal with certain casing issues. Ultimately this is the way I've made stylesheets workable for myself - keep the ability to separate line casings and widths and colors and zoom levels, and use the expressiveness of SQL to munge the planet database into something more directly applicable to rendering. I know that Cloudmade have done similar things, including I believe downsampling the detail level of the road vectors for certain rendered zoom levels.