The makemap tool uses rules written in a small XML-based language to control the import of data from OSM and other sources. The idea is that an input data set is supplied containing a set of attributes and values (a row in the map database), and a program made up from a series of statements uses it to create one or more output map objects.
makemap uses a standard set of rules. You can write them out using makemap (see makemap's usage message for details) and edit them to create your own rules.
Thus a single input data set (database row) can create more than one map object, but the converse is not true.
The program is executed in its entirety once for each input data set.
File format
Files have the extension .makemap and are used by the makemap tool. They can be placed on the makemap command line using either /input=, or as the main input file.
A <CartoTypeImportRules> element contains the whole file.
It can contain:
- <file_type> elements
- <file> elements
- commands
The <file_type> elements contain commands to be applied to a certain file type or sett of files, specified using a name containing wild cards. For example, <file_type name='*.osm'> ... </file_type> specifies a program to be applied to all OSM (OpenStreetMap data) files; the program is applied to OSM files placed later on the makemap command line.
The <file> elements contain commands to be applied to a named file, and cause that file to be loaded and processed. The <file> command is usually used for SHP files.
Any commands outside any <file_type> or <file> elements are treated as if they are in an implicit <file_type name='*.osm'> ... </file_type> element.
Summary
An import rules program is run once for every row in the input database. For example, when reading OSM data it is run once for every node, way and relation.
The statements in the program can set output map object attributes like the layer name, the name of the object and other string attributes, and the integer attribute. The <commit/> statement creates a new output object.
The interpreter state
It consists of a stack of output objects reflecting the dynamic nesting of <group> elements (see below), and a list of output data objects for conversion to map objects.
The current input object represents a single database row. When processing OSM data, it contains the data from a <node>, <way> or <relation> element. When expressions use an ordinary variable, it refers to one of the attributes of the current input object; in OSM terms, that would be a element with k and v attributes giving the key and the value respectively; the k attribute is the variable name, and the v attribute is its value.
The current output object is the one on the top of the stack. When the <commit/> statement is executed, it is copied to the list of output data objects, and each one of these becomes a map object after the import rules program has been fully executed.
Output object data in detail
Each output object that is committed by a <commit/> statement contains the following information:
The layer name: a string. If the layer is empty, the map object is used only if it is part of an OSM relation. Normally the layer must not be empty. Set it using <set_layer>.
The map object type. This should be set using <set_point/>, <set_line/> or <set_polygon/>. If it is not set, a default is used, but it is always best to set it explicitly, especially when processing OSM ways and relations.
The feature info.
The feature info for a route object contains:
- discriminator: ordinary route, bridge or tunnel
- a level in the range -8...+7
- a feature type in the range 0...31; these are the route types, which also act as indexes into speed and bonus arrays in route profiles
- flags for one-way, drive-on-left, roundabout and toll road
- speed limit in kph
- gradient code (used only in internal route calculation layers)
- access flags (wrong-way, pedestrian, cycle, motor, emergency, other)
The feature info for a non-route object contains:
- discriminator: always 0
- a level in the range -8...+7
- a feature type in the range 32...32767
- a sub-type in the range 0...2047; sub-types are used to distinguish island sizes, settlement ranks, etc.
You can set the feature info using the statements <set_feature_type>, <set_one_way>, <set_bridge>, <set_sub_type>, etc., described below. Note that setting a route object to a non-route feature type or vice versa clears other attributes of the feature info.
The string attributes. They are set by the <set> and <copy> statements.
Conditional execution of statements
All statements except <file_type> and <file> may be executed conditionally. That happens if the statement has a non-empty test attribute containing a boolean expression. The test is compulsory in the if statement, and optional for all other statements.
A statement may also contain an else attribute instead of a test attribute, causing it to be executed only if the preceding statement (at the current level of nesting inside <group> or <if> statements) had a test or else with a false result, and if the else attribute had a true result.
example:
<if test='place=="city"'> <commit test='population!=@ and population lt 1000' layer='place/major' feature_type='vil'/> <commit else='population!=@ and population lt 100000' layer='place/major' feature_type='tow'/> <commit else='1' layer='place/major' osm_type='cit'/> </if> <if else='place=="town"'> <commit test='population!=@ and population lt 1000' layer='place/major' feature_type='vil'/> <commit else='1' layer='place/major' osm_type='tow'/> </if> <if else='place=="village"'> <commit test='population!=@ and population lt 250' layer='place/minor' feature_type='ham'/> <commit else='1' layer='place/minor' osm_type='vil'/> </if>
Here, the construct else='1' is used to provide an unconditional 'else', because an ordinary else is an 'else-if'.
Expressions
Expressions can refer to the attributes of the current input object by name, and can test whether the object is in a node, a way or a relation, using the special boolean variables @node, @way and @relation; exactly one of these is true if the data source is OpenStreetMap, otherwise none of them is true. The variable @ means 'undefined'.
More details of expression syntax and operators.
Statements
<file_type name='string'> statements </file_type>
Creates a program defined by statements to be used for the file type given by name. The file type is a file extension like 'osm', without a leading full stop. This statement may not appear inside any other element, but must appear directly inside <CartoTypeinputRules>
<file name='string' { codepage='number' } > statements </file>
Loads the named file and processes it using the program defined by statements. The optional codepage attribute gives the 8-bit character encoding as a codepage number, which is used in DBF files associated with ESRI shapefiles.
example:
<?xml version="1.0" encoding="UTF-8"?> <CartoTypeImportRules> <file name='ne_10m_land.shp'> <commit layer='outline'/> </file> <file name='ne_10m_lakes.shp' codepage='1252'> <copy name='name'/> <commit layer='land/major' feature_type='wat'/> </file> <file name='ne_10m_populated_places.shp' codepage='1252'> <set name='name' value='NAME'/> <set_layer name='place/major'/> <commit test='SCALERANK==1 or SCALERANK==2' feature_type='cit'/> <commit else='SCALERANK gt 3' feature_type='tow'/> </file> <file name='ne_10m_admin_0_countries.shp' codepage='1252'> <set name='name' value='NAME'/> <set_layer name='boundary/major'/> <set_sub_type value='2'/> <commit/> </file> <file name='ne_10m_roads.shp' codepage='1252'> <set name='ref' value='name'/> <set_layer name='road/major'/> <if test='type=="Major Highway"'> <set_feature_type name='Motorway'/> <commit/> </if> <if else='type=="Secondary Highway"'> <set_feature_type name='TrunkRoad'/> <commit/> </if> <if else='type=="Road" or type=="Unknown"'> <set_feature_type name='PrimaryRoad'/> <commit/> </if> </file> </CartoTypeImportRules>
These <file> statements create a world map from Natural Earth data.
<dbf name='string' prefix='string' key='string' { external_key='string' } { codepage='number' } />
The dbf command is legal only inside a file command. It allows an associated DBF file to be loaded while reading an ordinary input file, usually a SHP file.
The name attribute is the path of the associated .dbf file, including its extension.
The prefix attribute is a prefix added to names fields loaded from the associated DBF file, if it's necessary to distinguish those field names from the field names in the main file.
The key is the name of the key field: that is, the field containing the value to be matched with a value in the associated DBF file.
The external_key is optional and is needed only if the key field in the associated file is different from that of the main file. If it is * (a single asterisk) it causes all the records in the associated DBF file to be made available.
Example: assume you have a shapefile main.shp (with its own dbf file main.dbf) and an associated dbf file other.dbf. The field ID exists in both files; in main.dbf it's the unique key, but in other.dbf there is a record containing the same ID. You need to load the matching ID record from other.dbf every time you process a record from main.dbf. You could do it like this
<file name='main.shp'>
<dbf name='other.dbf' prefix='OTHER_' key='ID'/>
<!--- rules -->
</file>
This causes all the fields in other.dbf in the record with ID equal to ID in main.dbf to be loaded and made available, but with OTHER_ prefixed to their field names.
If the matching field in other.dbf is not called ID but, for example, XID, you would use this dbf command:
<dbf name='other.dbf' prefix='OTHER_' key='ID' external_key='XID'/>
If multiple records have the same ID in other.dbf you can use an optional subscript on the field names: for example, OTHER_ID[0], OTHER_NAME[6]. If there is no subscript, the subscript 0 is assumed. There is no way to find out how many matching records there are. For that functionality you need to use embedded ChaiScript code.
<group> statements </group>
Pushes a new output data object on the stack, copying its values from those of the top of the stack. At the end of the group the top-of-stack object is popped and discarded. Thus the <group> must contain at least one <commit> statement to have any useful effect.
example:
<group test='highway!=@'>;
opens a new group if the 'highway' attribute is defined.
<if {test|else}='expression' > statements </if>
Tests the expression and executes the contained statements if it is true. The expression may refer to attributes of the current input object by their ordinary names. The difference between <if> and <group> is that <if> does not push a new output data object on to the stack.
example:
<if test='highway=="motorway"'> <set_road_type name='Motorway'/> <commit layer='road/major'/> </if>
tests whether the 'highway' attribute has the value 'motorway' and if so sets the road type to Motorway and commits a map object to the layer 'road/major'.
Note that all other statements may also contain a test or an else, so <if> exists to provide a tested grouping of statements; there is no need to use an <if> ... </if> construct around a single statement.
<commit { layer='string' type='string' feature_type='string' }/>
Creates a new output data object by copying the top-of-stack values, and adds it to the list of output data objects to be made into map objects.
If layer is set, it is used as the layer for this object instead of the layer set by <set_layer>.
If type is set, it is used as the map object type instead of the type set by <set_point>, <set_line> or <set_polygon> .
If feature_type is set, it is used as the feature type instead of the value set by <set_feature_type>.
<break/>
Breaks out of the current <group> or <if>.
<exit/>
Terminates execution of the whole program.
<set name='string' value='expression'/>
Sets the string attribute name to value.
example:
<set name='_ele' value='ele'/>
sets the string attribute '_ele' to the value of the input attribute 'ele'.
You can use '$' as the name attribute to set the label, which doesn't have an attribute name:
<set name='$' value='name'/>
Subscripts enable you to select a particular token from a value. This feature is available from release 8.8 onward. To select the first token (for example, to select the string 'New Zealand' from the name attribute 'New Zealand / Aotearoa') use two subscripts. The first one, [0], selects the first token (tokens are numbered from zero) and the second, ["/"], specifies the delimiter:
<set name='$' value='name[0]["/"]'/>
<copy name='string'/>
Sets the string attribute(s) named by key, which may contain wild cards, to the same string attributes in the output object. The name 'name' copies the 'name' attribute to the unnamed output attribute, which by convention is the name.
example:
<copy name='name:*'/>
copies all attributes matching 'name:*', like 'name:en', 'name:de', etc., to output string attributes of the same names.
You can also use the prefix attribute to add a prefix to the output string attributes:
<copy name='*' prefix='_'/>
The above example copies all attributes to output attributes, prefixing the output attributes with an underscore. Attributes starting with an underscore are not indexed for text searching. Therefore this example allows you to import all OpenStreetMap attributes without unnecessarily enlarging the text index. When a prefix is used, attributes which have already been set, even without the prefix, are not duplicated by new attributes with a prefix. This is in general what you want to do; if you have imported 'addr:street' explicitly, <copy name='*' prefix='_'/> does not create a new attribute called '_addr:street'.
<set_feature_type name='string'>
Sets the feature type to 'name'. Names are either full words or phrases like 'trunk road', used for route types, or three-letter abbreviations like 'stn' meaning a railway station. Letter case and spaces are ignored when comparing names: 'Motorway' is the same as 'motorway'. Route types must be one of the following: motorway, motorway link, trunk road, trunk road link, primary road, primary road link, secondary road, secondary road link, tertiary road, unclassified road, residential road, track, service road, pedestrian road, vehicular ferry, passenger ferry, living street, cycleway, path, footway, bridleway, steps, road, unpaved road, railway, light railway, subway, aerial way, ski downhill, ski nordic, waterway, unknown route.
example:
<set_feature_type name='motorway'/>
sets the feature type to 'motorway'.
<set_level value='expression'/> sets the level to the value of expression, which must be in the range -8 ... 7.
<set_sub_type value='expression'/> sets the sub-type of a non-route object to the value of expression.
<set_speed_limit value='expression'/> sets the speed limit of a route object to a value in the range 0...255.
<set_osm_speed_limit name='tag'/> sets the speed limit of a route object by interpreting the value of the OpenStreetMap tag named by tag, which is in the OpenStreetMap format for a speed limit. The tag is normally 'maxspeed'.
<set_one_way/> sets the one-way state to 'forwards'
<set_two_way/> sets the one-way state to 'two-way'
<set_one_way_forward/> sets the one-way state to 'forwards'
<set_one_way_backward/> sets the one-way state to 'backwards'
<set_roundabout/> sets the roundabout flag
<set_public_access value='expression'/> sets public access to the value of expression, taken as a boolean value.
<set_vehicle_access value='expression'/> sets vehicle access to the value of expression, taken as a boolean value.
<set_pedestrian_access value='expression'/> sets pedestrian access to the value of expression, taken as a boolean value.
<set_cycle_access value='expression'/> sets cycle access to the value of expression, taken as a boolean value.
<set_motor_vehicle_access value='expression'/> sets motor vehicle access to the value of expression, taken as a boolean value.
<set_emergency_vehicle_access value='expression'/> sets emergency vehicle access to the value of expression, taken as a boolean value.
<set_other_access_restricted value='expression'/> sets the 'other access restricted' flag to the value of expression, taken as a boolean value.
<set_feature_info_raw_value value='expression'/> sets the raw integer value of the feature info.
<set_toll/> sets the toll-road flag
<set_bridge/> sets the bridge flag
<set_tunnel/> sets the tunnel flag
<set_layer name='string'/>
Sets the layer to name.
<set_point/>
Sets the map object type to 'point'.
<set_line/>
Sets the map object type to 'line'.
<set_polygon/>
Sets the map object type to 'polygon'.
<script>
A <script> ... </script> element contains program code in the ChaiScript language. You can use a <script> element anywhere convenient (in fact, anywhere that the <commit> command is legal) if you need the power of a programming language when importing data.
