Everything about OpenStreetMap.
OSM's data model can be split into OSM elements and OSM changeset.
Changeset is a special OSM entity for metadata about data edits, including id fields (id, user, uid), two timestamps (created_at, closed_at), special attributes (open, comments_count) and custom attributes via tags. Actual edits corresponding to a changeset is not part of the OSM data model, and is typically derived from OSM files using custom software.
A simplified example of OSM data represented in XML
(the entire file should be enclosed in an
<osm> tag pair).
<node id="42437959" visible="true" version="12" changeset="13150866" timestamp="2012-09-18T02:14:34Z" user="RoadGeek_MD99" uid="475877" lat="40.7229823" lon="-73.9885488"> <tag k="highway" v="traffic_signals"/> </node> <node id="42443513" visible="true" version="5" changeset="5155439" timestamp="2010-07-07T00:47:51Z" user="Dylan Semler" uid="31855" lat="40.7232770" lon="-73.9884510"> <tag k="highway" v="traffic_signals"/> </node> <way id="5672851" visible="true" version="37" changeset="41811184" timestamp="2016-08-30T20:49:09Z" user="infinitesunrise" uid="4434293"> <nd ref="42437959"/> <nd ref="42443513"/> <tag k="highway" v="primary"/> <tag k="name" v="1st Avenue"/> <tag k="oneway" v="yes"/> </way> <relation id="1077653" visible="true" version="3" changeset="21582260" timestamp="2014-04-09T04:29:57Z" user="lxbarth" uid="589596"> <member type="way" ref="22898643" role="from"/> <member type="way" ref="34080174" role="to"/> <member type="node" ref="42428201" role="via"/> <tag k="restriction" v="only_straight_on"/> <tag k="type" v="restriction"/> </relation>
An OSM changeset represented in XML.
<changeset id="1077653" user="aeonesa" uid="61599" open="false" comments_count="0" created_at="2009-05-04T18:36:38Z" closed_at="2009-05-04T19:36:38Z"> <tag k="created_by" v="Potlatch 0.11a"/> </changeset>
A simple OSM edit in osmChange format (
<osmChange version="0.6" generator="acme osm editor"> <modify> <node id="1234" changeset="42" version="2" lat="12.1234567" lon="-8.7654321"> <tag k="amenity" v="school"/> </node> </modify> </osmChange>
OSM Cheat Sheet: tags and keyboard shortcuts.
name, the primary tag used for common names;
ref, reference numbers or codes.
alt_name, less common names.
highway is the main tag (confusingly) used to classify
any kind of road, street or path:
motorway(one-directional by default);
motorway_link(speed depends on curvature).
trunk(high capacity road without limited access),
residential(in residential area),
unclassified(minor public roads, non-residential);
living_street(slow traffic has absolute right-of-way),
service(in private property or parking lots),
pedestrian(normally forbidden for motor vehicles),
Other tags that can classify ways as part of a road network:
lanes:backward) indicate number of lanes (in a given direction);
turn:lanes(left, through, right) and
parking:laneindicate turning lanes and parking lanes;
tunnelfor bridges and tunnels;
junction=roundabout, a single lane roundabout (one-directional);
accessadditionally describes legal access to an entity via all or particular forms of transport.
maxspeed, maximum legal speed limit (default in kilometers per hour).
oneway(yes, no, -1).
hgv, heavy goods vehicle.
place(continent, country, state, region, county, city, town, village, hamlet, suburb, island);
barrier, a physical structure that blocks or impedes movement (toll_booth, etc).
Turn restriction: member roles
via, and tag
except, exception by vehicle type.
Multipolygon: member roles
inner, and tag
Other relations: route (a collections of ways)
.osm, data file with at most one version per object (may also be history file);
.osh, history file, multiple versions of an object are allowed;
.osc, osmChange file.
.pbf, the primary binary format based on protocol buffers.
.o5mfor data files,
.o5cfor osmChange files.
OSM data files are almost always ordered in a specific way: nodes, ways, relations; each group ordered by ascending ID. History files additionally order each element by ascending version. Change files are usually ordered by changeset ID.
Protocol buffers is a language-neutral, platform-neutral extensible mechanism for serializing structured data. Delta coding and variable-byte coding are applied throughout.
Characteristics of OSM PBF:
API v0.6 calls:
/capabilities(0.25 sq degree, 300 seconds timeout);
/permissions(not authorized, Basic Auth, OAuth);
/userquery by id with endpoints for the current authenticated user (gpx_files, details, preferences);
/notesto edit (create, comment, close, reopen) and read (by bbox and text search);
/<type>/<id>to edit (create, update, delete) and read (history or specific version) individual element; fetch referencing (ways, relations) or referenced (full) elements; fetch multiple elements with query "(node|way|relation)=id[,id][&...]".
/mapwith query "bbox=left,bottom,right,top";
/changeset/<id>to edit (create, upload, update, expand bounding box, close), read (get, download), or discuss (comment, subscribe, unsubscribe) individual changeset; fetch multiple
/changesetswith query (user, display_name; bbox, time, open/closed; changesets)
/trackpointsto query GPS track points (bbox, page) as GPX files;
/gpxto upload (create) and download (details, data).
Overpass API is a read-only HTTP API for extracting elements from an OpenStreetMap database. Two query languages are available for the Overpass API: Overpass XML and Overpass QL. Overpass Turbo is a Web frontend of Overpass API. Mapbox's list of useful queries.
planet.osm.org provides weekly dumps of the entire OpenStreetMap database
in one file commonly referred to as Planet.osm.
OSM data is provided in text (
and binary (
Diffs/change files produced with
osmosis are organized under
state.txt provides metadata about the most recent change file
/AAA/BBB/CCC.osc.gz is the change file with sequence number
You should use osmosis to retrieve and apply all replication diffs since its last run
Metadata of changes are at
/planet/changesets-latest.osm.bz2 and under
Planet.osm has several mirrors.
Regional extracts are available and periodically updated from third parties. GeoFabrik.de provides daily extracts for continents, countries and sub-country regions; also available in Shapefile. BBBike.org provides weekly extracts for cities and regions as shapefile, CSV, SVG, OPL and formats for offline navigation (Garmin, Navit, maps.me, OsmAnd, mapsforge), also offers custom bounding box or polygon extracts within 24M sq km and 768MB file size. BBBike.org extracts set version of all map entities to -1 and all timestamps to 1969, which can cause trouble. Metro Extracts by Mapzen provides weekly bounding box extracts of popular cities and regions as Shapefile and GeoJSON files split by geometry type of features (lines, points, polygons) or by logical groups (Roads, etc) of OpenStreetMap tags (no relation); custom bounding box extracts with update support is available using Mapzen API; coastline geometries of each area are also available in Shapefile.
OSM turn restrictions:
osmosis is a Java program for processing OSM data, which can extract data inside a bounding box or polygon.
osmupdatecan create planet change files and update OSM data files with them; faster than
osmosisbut skips history diffs and cannot update databases.
osmconvertcan convert and process OSM files faster than
osmosisbut has less functionality; it has a few special functions (--all-to-nodes, --complex-ways and --out-statistics).
osmfilterfilters OSM data files for specific tags.
osmium is the command line tool of
the Osmium library.
Besides all OSM file formats, osmium also supports a "debug" format (write-only) and the OPL format,
and compression by gzip or bzip2.
Osmium is memory conscious in design (uses index), and much faster than
derive/apply/merge changeset, validate, and extract historical view.
showload content in pager;
getid, get objects by ID;
extractby bounding box, polygon, or multi-polygon;
changeset-filter, filter OSM files;
check-refscheck referential integrity (whether all referenced objects are present in the file);
sortby type (node, way, relation), ID, version (and timestamp for change files);
diffdisplay content differences;
derive-changescreate a change file;
merge, merge OSM extracts (objects are unique by type, ID, and version; contents are not unchecked), do not work on OSM data files from different points in time;
apply-changes, merge/apply change files;
add-location-to-waysdereference way geometries;
exportOSM topology into geometry formats (currently supports GeoJSON);
JOSM, Java OpenStreetMap, is the offline editor of OSM data, with many plugins. JOSM plugins for fixing missing turn restrictions: ScoutSigns (road signs by Scout users), ImproveOsm (missing geometries and turn restrictions, by Scout), turnrestrictions (by skobbler).
Merkaartor is another OpenStreetMap editor, written in C++.
KeepRight, data consistency checks (quality assurance) for OSM.
osm2pgrouting, imports OSM topology to PostgreSQL database;
other java programs.
Programming Frameworks for accessing, processing, map rendering (static, interactive), geocoding, and navigation with OpenStreetMap data. Debian GIS Blend meta-package for OpenStreetMap already include many of these tools and libraries.
osm-common(Java): accessing, processing, geocoding; supports Overpass.
osmapi(Python): Python wrapper for the OpenStreetMap API.
osmaR(R): access OpenStreetMap data from file or API, and convert to other classes.
Data processing / parsing:
pyosmiumto Python and
imposm(import OSM to PostgreSQL/PostGIS),
OSMnx, construct street networks and compute density/routing/topological measures [@Boeing2017];
Geocoding: Gisgraphy (Java)