August 2014

Time Machine backups using FreeBSD ZFS

In this blog article, I described a way to use Netatalk3 to do Time Machine backups on FreeBSD.  This approach worked well, but it had some problems: The Time Machine backups are in every user’s home dir.  That’s messy and there is the potential that they’ll accidentally delete the backup. If I put the backups on a ZFS disk, I can compress them I would like the potential to use ZFS snapshots down the road I would prefer to have all the backups in one directory, rather than scattered across user profiles So, here is a new, better recipe.  Note that this recipe will only work with Netatalk 3.1.2 or better.  The current FreeBSD port is version 3.1.3, so that helps.  Firstly, as in the other recipe, the first step is to install netatalk3, and nss_mdns pkg install netatalk3 pkg install nss_mdns avahi needs mDNS, so that needs to be configured in /etc/nsswitch.conf.  Ensure that this line exists: hosts: files dns mdns Next create a ZFS dataset for the backups.  my zpool is /storage, and the Time Machine backups will be in /storage/timemachine.  Notice that I’ve turned on compression. zfs create storage/timemachine zfs set compression=gzip storage/timemachine zfs get all storage/timemachine zfs list I want to grant the ability to do time machine backups to a FreeBSD group, so I’ll make that, and add the users.  This group will be referred to in afp.conf. pw groupadd timemachine pw groupmod timemachine -m tom pw groupshow timemachine After that, we need to create user directories, one for each time machine user. mkdir storage/timemachine/tom chown tom:timemachine /storage/timemachine/tom chmod 700 /storage/timemachine/tom chmod 777 /storage/timemachine So, now that I have a compressed ZFS dataset to store the backups on, backup directories created, and a FreeBSD group created, I can create afp.conf.  I’ve chosen to limit Time Machine to 500GB space for each user. ; ; Netatalk 3.x configuration file ; [Global] vol preset = default_for_all_vol log file = /var/log/netatalk.log log level = default:warn  hosts allow = 192.168.77.0/24 mimic model = TimeCapsule6,116 disconnect time = 1 vol dbpath = /var/netatalk/CNID/$u/$v/ [default_for_all_vol] file perm = 0640 directory perm = 0750 cnid scheme = dbd [Homes] basedir regex = /storage/home #500 GB (units of MB) vol size limit = 512000 [TimeMachine] time machine = yes path=/storage/timemachine/$u valid users = @timemachine #500 GB (units of MB) vol size limit = 512000 You should notice that the path of the [TimeMachine] share is set to path=/storage/timemachine/$u This means that for each logged in user, $u is substituted wth the name of the user. So, when Time Machine logs in as “tom”, the data is stored at /storage/timemachine/tom. Only members of the group “timemachine” will be able to use Time Machine. This line vol dbpath = /var/netatalk/CNID/$u/$v/ Ensures that each user as a CNID database for each volume.  Without this line, there is a single CNID database which is shared for all users who are using TimeMachine.  That generally results in corrupting the CNID database.  By specifying /$u/$v, we get a CNID database for each user for each volume, which is much more reliable. By specifying a disconnect time of one hour: disconnect time = 1 We can disconnect orphaned sessions and hopefully avoid the dreaded “volume in use” error in TimeMachine. The rest of the steps are exactly as in the previous blog entry. You’ll need to start dbus, avahi, and netatalk, like this: /usr/local/etc/rc.d/dbus onestart /usr/local/etc/rc.d/avahi-daemon onestart /usr/local/etc/rc.d/netatalk onestart The next step takes place on your OS X client machine.  On each host that will perform backups, enable Time Machine to see non-TM volumes: defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1 Then, mount your user’s share using afp://.  This will make the share visible to TimeMachine. After this, you should be able to see your Netatalk shares in Time Machine, and perform backups

TimeMachine backups on FreeBSD-10

I usually work on a Macbook, and I use keyless ssh with a cron job to do nightly backups.  It works very well.  However, I decided to try backups with Time Machine, using my FreeBSD-10 server as the host. The first step is to install netatalk3, and nss_mdns pkg install netatalk3 pkg install nss_mdns avahi needs mDNS, so that needs to be configured in /etc/nsswitch.conf.  Ensure that this line exists: hosts: files dns mdns Finally, configure netatalk.  My configuration in /usr/local/etc/afp.conf looks like [Global] vol preset = default_for_all_vol log file = /var/log/netatalk.log hosts allow = 192.168.77.0/24 mimic model = TimeCapsule6,116 [default_for_all_vol] file perm = 0640 directory perm = 0750 cnid scheme = dbd [Homes] basedir regex = /storage/home time machine = yes You’ll need to start dbus, avahi, and netatalk, like this: /usr/local/etc/rc.d/dbus onestart /usr/local/etc/rc.d/avahi-daemon onestart /usr/local/etc/rc.d/netatalk onestart The next step takes place on your OS X client machine.  On each host that will perform backups, enable Time Machine to see non-TM volumes: defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1 Then, mount your user’s share using afp://.  This will make the share visible to TimeMachine. After this, you should be able to see your Netatalk shares in Time Machine, and perform backups

Building U-boot 2014 on FreeBSD

I’m upgrading the U-boot version used for Wandboard by Crochet-BSD.    The first step is to upgrade the Device Tree Compiler (DTC); there are instructions here. Once you have the U-boot 2014 source, the build is trivial: gmake SED=gsed HOSTCC=cc CROSS_COMPILE=armv6-freebsd- wandboard_quad_config gmake SED=gsed HOSTCC=cc CROSS_COMPILE=armv6-freebsd-  

Building u-boot for Chromebook

I got interested in building Crochet images for Chromebook.  The first step, naturally is to build U-boot for Chromebook. In order to build U-boot, you’ll need to have already installed gmake gcc You can get these from the ports tree.  You’ll also need xdev, the FreeBSD cross-compiler toolkit. I started with the latest U-boot 2014 source, which I got here.  The latest U-boot requires DTC 1.4.  I’m on FreeBSD 10, which has a much older DTC. To build the latest DTC: git clone git://git.kernel.org/pub/scm/utils/dtc/dtc.git cd dtc gmake CC=gcc cp dtc /usr/bin Once we have DTC 1.4, we can build U-boot. wget ftp://ftp.denx.de/pub/u-boot/u-boot-2014.07.tar.bz2 tar zxvf u-boot-2014.07.tar.bz2 cd u-boot-2014.07 gmake SED=gsed HOSTCC=cc CROSS_COMPILE=armv6-freebsd- snow_config gmake SED=gsed HOSTCC=cc CROSS_COMPILE=armv6-freebsd- The U-boot binary image is “u-boot.bin”. To burn the U-boot binary to a SD card: sudo dd if=u-boot.bin of=/dev/rdisk1 The next challenge is the file system. The file system on the Chromebook looks like this: start size part contents 0 1 PMBR (Boot GUID: E780FED6-7991-B94B-B622-FA7541BE9FBB) 1 1 Pri GPT header 2 32 Pri GPT table 8671232 21295104 1 Label: “STATE” Type: Linux data UUID: D5488177-EB19-3243-8D7C-D6E4F01DB5B3 20480 32768 2 Label: “KERN-A” Type: ChromeOS kernel UUID: 47715F23-7DD6-5545-93DC-3807D1CD4AFD Attr: priority=0 tries=0 successful=0 4476928 4194304 3 Label: “ROOT-A” Type: ChromeOS rootfs UUID: BE1064FF-EC28-E64B-B3A1-A5C176D57C0F 53248 32768 4 Label: “KERN-B” Type: ChromeOS kernel UUID: 5D200D96-1F52-804C-83A7-AD8081E8ACD4 Attr: priority=1 tries=0 successful=1 282624 4194304 5 Label: “ROOT-B” Type: ChromeOS rootfs UUID: D0795A2A-1D0D-4140-9F57-F87AEF399C44 16448 1 6 Label: “KERN-C” Type: ChromeOS kernel UUID: E8A703B8-E0A5-964D-9508-D88DE336D003 Attr: priority=0 tries=15 successful=0 16449 1 7 Label: “ROOT-C” Type: ChromeOS rootfs UUID: 7A947ABA-E723-8440-9A07-0EC394C85195 86016 32768 8 Label: “OEM” Type: Linux data UUID: 173B3C97-CDCF-F849-99E7-7614C6C0F9FD 16450 1 9 Label: “reserved” Type: ChromeOS reserved UUID: CD83D43A-4FC8-4849-B0DD-368E9C3819BA 16451 1 10 Label: “reserved” Type: ChromeOS reserved UUID: 344D3149-A5C2-B44B-A65C-2DFD59758FC5 64 16384 11 Label: “RWFW” Type: ChromeOS firmware UUID: D9B79232-94AD-A143-9C49-F947BFA2491E 249856 32768 12 Label: “EFI-SYSTEM” Type: EFI System Partition UUID: E780FED6-7991-B94B-B622-FA7541BE9FBB 30777311 32 Sec GPT table 30777343 1 Sec GPT header There is a good description of the Chromium OS file system here.  The size parameter is in sectors, which are usually 512 bytes each.  So, for example the state partition is 10GB in size.  KERN-A is 16MB in size, and ROOT-A is 2GB. The partition ids for ChromeOS kernel and rootfs, are “fe3a2a5d-4f32-41a7-b725-accc3285a309” and “3cb8e202-3b7e-47dd-8a3c-7ff2a13cfcec” respectively.  The partition ids for ChromeOS are documented here.