Using the FreeBSD operating system, a version of Unix, you can create larger CTM1 files than is usually possible under Windows, depending on how much RAM you have.
Installing FreeBSD
Basic Installation
It's assumed you know how to install an operating system, so this part will be brief. You can download the latest version of FreeBSD from freebsd.org, burn it on a CD and boot from it. Make sure you download the AMD64 version, not the Intel386 as the latter is only 32-bit. The AMD64 instruction set appeared before Intel's own 64-bit processors so the 64-bit version is named for them (which must annoy Intel).
Different versions of FreeBSD ask different install questions but you can leave it to the default options selected most of the time.
Swap Space
You may want to increase the size of the swap partition when you are partitioning the disk, especially if it does not have its full complement of RAM installed yet. Make sure the swap partition is three times the size of RAM you intend to install. Assuming you are using the 'easy' disk partition option you will have to delete the existing swap partition AND the partition before it, and then create one of the correct size. For example, if your motherboard supports 8Gb of RAM you should ideally have a 24Gb swap partition, and you can specify this with '24G'in the appropriate field.
On a high-performance installation you will probably want to use a separate swap drive or drives, and you can add to swap space later, so it's not a disaster if you get the swap space wrong.
You may wish to format the disk using the GUID system rather than MBR, especially if you have large disks (>2Tb). It's beyond the scope of this document to go in to the pros and cons; if in doubt stick to MBR.
Other Install Options
As of FreeBSD 9.0, the installation program is a lot simpler and has fewer choices. In case you are using a version with different defaults, select the options to install the ports tree (not strictly necessary if you plan to download the latest later), and probably the system source code. You may want the 32-bit emulation libraries but they're not needed for CartoType at present.
You must obviously configure the network to suit your local requirements. DHCP configuration is easiest if available, but you will have to find the IP address so you can talk to the box later.
You should add a user to log in remotely with. As you go through the questions it will ask if you want to invite the user in to other groups. Don't select the default by pressing Enter. At that prompt, type 'wheel'. This is the group you need to be in if you want to switch to the root user (Administrator) after logging in remotely. It's a security feature; if you're not in wheel there is no way it will let you switch to root.
Configuring for CartoType use
It is easier to work with FreeBSD using an ssh terminal client like PuTTY from Windows, or simply ssh from a UNIX-like box.
For more information on any of this, or if you're new to FreeBSD or UNIX, you should read the handbook on www.freebsd.org. To get help on a particular command or function, type 'man commandname' to get the appropriate manual page.
Logging in
Start PuTTY or the ssh client of your source at the IP address of your FreeBSD box and you'll receive a login prompt. Don't enter 'root' “ it won't let you! Use the user and password you created during the install, the one you added to the 'wheel' group. Then, once you're logged in, type'su'. You'll be promoted for the root password, sometimes whether you set one or not. Enter it (obviously) and the prompt will change – you're now root, and able to do some of the most powerful and destructive things possible with a computer. So be careful.
FreeBSD Ports Tree
The FreeBSD ports tree allows you to download and automatically patch the source code for many things, then build and install them. FreeBSD does also have a pre-compiled package system, similar to Linux, but we want maximum performance and code compiled explicitly to take advantage of your CPU. The first thing you need to do with the ports tree is update it.
Since the Summer of 2012 the project has moved to Subversion rather than CVS, and CVS was phased out completely in early 2013. The easy way to update your ports is to log in as root and type:
portsnap fetch
portsnap extract
portsnap update
Unfortunately, portsnap is friendly but a bit dim – installing the ports collection from the DVD won't stop it wanting to install them all again. So only do this if you have a fast Internet connection.
Installing Applications
The main CartoType utility for generating maps is called gmdt1, or generate_map_data_type1.exe on Windows. It's written in ‘C++' and need to be compiled from source. This is easy, but you'll probably want to pull the latest version down from source repository using the Mercurial version control system, which we must install first. You may also need Osmosis to work with OSM data, and that's written in Java so we'll need that too.
Install Mercurial (hg)
You may wish to pull source code from the repository using Windows and a graphical shell around Mercurial, but you can also pull it directly. If you don't plan to do this, don't bother with compiling mercurial.
/usr/ports/devel/mercurial
make
make install
You'll be asked to specify various options – leave them as the default. Mercurial is written in Python, so it will download and install that first, as well as a number of other things Python might need.
Installing Java
You'll probably want to use Osmosis, and this is written in Java so we'll need that first. Rather than messing with the run-time only we'll download and install the whole development kit. The version that's fully tested is diablo-jdk16, but OpenJDK may be used in the future. It's been around since August 2012.
cd /usr/ports/java/diablo-jdk16
make
Here we hit an interesting licencing problem – FreeBSD, being free, can't distribute Java. Although it's also free, it's a different kind of free. You will probably receive a message like:
Because of licensing restrictions, you must fetch the distribution manually, and you will have to go to Google for this, because the way to do it changes.
Please place the downloaded file(s) in /usr/ports/distfiles.
Basically, do what it says – probably using a machine with a web browser and copying it on to the FreeBSD box in the directory specified. 'distfiles' is where the latest source code is always downloaded to. In the web browser you will have to agree to the Oracle license agreement.
When the file is in place and you return to the directory and type 'make', it will all happen.
If you want to try OpenJDK instead:
cd /usr/ports/java/openjdk7
make
make install
Java seems to require just about all the open source software in the universe before it will compile, so this is going to be a long job. Unfortunately you can't go out for lunch as it will ask you to confirm options periodically – just accept the defaults, except when it asks if you want X11 support, in which case the answer is probably 'no'.
Installing Osmosis
For some reason, Osmosis is filed under 'Astronomical ports', but don't let that put you off.
cd /usr/ports/astro/osmosis
make
make install
This will be quick – it's just downloading and copying in to place (as long as Java is already there).
Installing PostgreSQL
You don't necessarily need the PostgreSQL database but if you're using one, it's the best for these purposes. It's the DBMS of choice for OpenStreetMap as has GIS extensions available. Assuming you are more familiar with Oracle or MySQL, this goes it to a bit more detail. Postgre is unusual as it does not store meta information (accounts and so on) in a table.
First off, install PostgreSQL server:
cd /usr/ports/databases/postgresql93-server
make
make install
(Note this installs 9.3 Beta as at August 2013 – you may prefer 9.2, but no earlier if you want postgis 2.0)
If you want the server started when you boot, execute the following:
echo 'postgresql_enable="YES"' >>/etc/rc.conf
Otherwise you can run it with 'service postgresql onestart'
HOWEVER, you must initialise the database before you start it. The following command works for me:
service postgresql oneinitdb
This is easier than the procedure described in the manual. You're also going to have to create a user, and to do it you have the be the UNIX PostgreSQL user (pgsql).
su pgsql
createuser -sdrP yourname
exit
This creates a super-user. It would make life a lot easier if 'yourname' above was your UNIX login name. The Postgres accounts are different to the UNIX accounts, but it often uses your UNIX name to attempt to login.
Now we can test it all with:
createdb testdb
psql testdb
This should give you a prompt which suggests you type 'help' for help. It's lying (as at 9.3 Beta) . 'help;' works a lot better, in fact it will ignore everything until you add a semi-colon.
My favourite option is \q for 'quit'. One you've done that you can install postgis:
service postgresql stop
cd /usr/ports/databases/postgis20
make
make install
This also pulls in far more external source than you might expect, and therefore takes a long time to compile.
Operating System Enhancements
There are a number of things you may wish to do with the FreeBSD environment to make it more useful. Unfortunately there isn't time to go in to great detail, but this section is intended to give you ideas and tell you where to look for more information.
A Word About File Systems
Natively, FreeBSD formats disks using UFS, the Unix File System. It's as old as UNIX and perfectly good. Linux using its own format called EXT, which is currently on version 3. The only apparent reason for Linux not using UFS was fear of patent difficulties, but it's left a situation where you can't swap Linux drives with any standard UNIX. FreeBSD can mount non-native file systems including EXT, FAT, NTFS, HPFS, CDFS but you would be safer mounting them as read-only.
UFS is starting to show its age (30+) and a new system called ZFS, developed by Sun, is set to take over. Its advantages are that it includes a lot more error checking, data duplication and automatic recovery from hardware failures – and it can address a mind-bogglingly large amount of disk space. The ‘Z' in the name stands for Zettabyte, which is big. 1K Terabytes make a Petabyte, 1K Petabytes make an Exabyte and 1K Exabytes make a Zettabyte. And ZFS can handle 256 quadrillion Zettabytes. This should be plenty for your OSM project.
Although UFS can handle file systems greater than 2Tb, things like the native disk mirroring can't (always). ZFS has built in RAID and a volume manager, so it's the way to go if any of your drives are more than 2Tb, or if you have more than about eight drives and you no longer care exactly which one your file is on.
Geom Mirroring
There plenty of ways to get RAID or disk mirroring on FreeBSD using UFS, but the easy way to do is using the Geom Mirror system, known as gmirror. For instructions on use, see the FreeBSD Handbook. It's safe, tried and tested and very reliable. It can also speed up reads considerable, as it will balance the load between multiple drives.
Sharing files on a Windows network
Windows (including OS/2 and LAN Manager) use a system of file sharing known as SMB, which has become known to some as CIFS (Common Internet File System) when Microsoft decided to market it seriously in 1996. Don't let the name fool you.
UNIX uses an older and cleaner system called NFS (Network File System).
There are two ways of networking Windows and UNIX machines. You can install an NFS client on Windows. This has to be downloaded for Windows XP, but is built-in from Windows 7 onwards – except it's not installed by default and only available on the Enterprise or Ultimate edition. It was, I'm told, added on Windows Server 2003 R2 (note R2, not SP2)
If you want to install/enable the Windows 7 NFS client, go to "Programs and Features" in the Control Panel. In the left hand column, you'll see a link called "Turn Windows features on or off".
Select that and it will open a list box that shows all of the optional components built in to Windows 7. Some are already activated. Expand the entry for "Services for NFS". There are two check boxes under that. Select both and hit OK. Windows will install those components and reboot. To use it, go to Administrative Tools->Services for NFS to configure it, or use 'nfsadmin' from the command line.
Nonplussed by Microsoft's attitude, the University of Michigan wrote their own NFS client:
http://www.citi.umich.edu/projects/nfsv4/windows/readme.htm
However, rather than fighting with Windows, you can install the SMB service on FreeBSD – it's called SAMBA and can be found in /usr/ports/net/samba36 (as of the current release). Samba 4, which is normally present in the ports collection, can be problematic. Samba4 is supposed to provide a Windows Domain Controller, Active Directory. If you just want to share files and directories, it's probably best to stick with Samba 3.
FTP Access
There is a base system FTP server controlled by inetd (see /etc/inetd.conf). All you need to do is enable it by uncommenting the line in front of the FTP server, adding the line 'inetd_enable=YES' to /etc/rc.conf and getting inetd to re-read its config file with 'service inetd reload'.
However, pure-FTP in /usr/ports /ftp/pure-ftpd is a lot better.
Rsync
The standard way of synchronising files on UNIX machines, especially across a network, is to use rsync. It's available in /usr/ports/net/rsync.
