It's possible to use the makemap tool to create very large CartoType maps (CTM1 files), up to and including a map of the whole world using the full OpenStreetMap data file planet.osm. The easiest and fastest way is to use lots of RAM.
CTM1 file size limit: use /largefile=yes
The largest possible CTM1 file is one terabyte (2^40 bytes; 1024Gb). That limitation is caused by the use of 40-bit numbers to store file positions. Note, however, that by default the makemap tool uses 32-bit numbers for file positions, giving a limit of 4 gigabytes. You can enable 40-bit file positions using the command-line argument /largefile=yes with makemap.
RAM versus swap space
Using a large amount of RAM is much faster than using swap space. For example, a map of North America took over 40 hours to create using 32Gb of RAM on Windows, which allocated swap space automatically. Using 64Gb of RAM on Linux, plus 32Gb of swap space, it took five and a half hours. Using 96Gb of RAM on Linux brought the time down to just over five hours.
The actual command line used for the map of North America was
osmconvert north_america.pbf | ~/ctm1/makemap -largefile=yes +proj=lcc +lat_1=20 +lat_2=60 +lat_0=23 +lon_0=-96 +x_0=0 +y_0=0 +ellps=WGS84 +datum=WGS84 +units=m -extent=-179,18,-50,75 ~/ctm1/coastlines.makemap -urbanareas=yes -.osm north_america.ctm1
and it used a PBF data file created using this command
osmconvert planet.pbf --verbose -b=-179,18,-50,75 --complex-ways -o=north_america.pbf
The resulting file was 14.8Gb in size.
Creating a map of the whole world
You'll need at least 128Gb of RAM to create a map of the whole world from planet.pbf. A map of the world was created by a CartoType user on a Windows computer with 128Gb of RAM and an Intel Xeon E5-2697v3 Haswell-EP processor with 14 cores, running at 2.6GHz. The process took about 12 hours.
Another example: Germany
Creating a map of Germany on Linux took 1hr 30min using this command line:
osmconvert germany.pbf | ~/ctm1/makemap -largefile=yes -project=laea -urbanareas=yes -extent=5.45,47.19,15.2,55.15 -usgs=~/new-height-data ~/postcodes/DE.txt ~/ctm1/coastlines.makemap -.osm -output=germany.ctm1
and extracting the data using this command:
osmconvert planet.pbf --verbose -b=5.45,47.19,15.2,55.15 --complex-ways -o=germany.pbf
and it produced a file of 9.2Gb
Computer specifications
The computer used for the Linux examples above has 94Gb of RAM and an Intel Xeon Silver 4110 processor with 8 cores and 16 threads, running at 2.1GHz. Multiple cores are very helpful now that makemap performs many operations in parallel. The computer also has a 256Gb SSD and a 3Tb hard drive. Mass storage capacity is less important than RAM.
Keeping your copy of the OpenStreetMap planet file up to date
If you often need to create large maps it's a very good idea to maintain your own local copy of planet.pbf. Use planet.pbf, not planet.osm, which is larger. Once you've downloaded this enormous file, you should update it about once a week using osmupdate:
osmupdate --verbose --day --drop-author planet.pbf planet_new.pbf
and then you can rename the files and archive the old one as convenient.
Routing with large maps: reduce RAM footprint by using /route=ch
You may find that creating routes causes problems with large maps because of the size of the route data. Standard route data is loaded into RAM, and can take several Gb on large maps. For example, the drawing a map of Germany without routing took about 500Mb RAM in a recent test; creating a route cause nearly 6Gb of additional RAM to be used. If you need to lower the RAM footprint, at the cost of restricting routing to a sinnge route profile, you should use the contraction-hierarchy routing system, which uses only a few 100Mb of RAM at run-time. Use the command-line option /route=ch to create contraction-hierarchy data instead of the normal routing data. Note that this may add some hours to the time it take to create the map; creating a contraction hierarchy is slow.
