Time Machine backups using FreeBSD ZFS

tom@khubla.com
August 25, 2014
15 Comments
0

In this blog article, I described a way to use Netatalk3 to do Time Machine backups on FreeBSD.  This approach worked well, but it had some problems:

  • The Time Machine backups are in every user's home dir.  That's messy and there is the potential that they'll accidentally delete the backup.
  • If I put the backups on a ZFS disk, I can compress them
  • I would like the potential to use ZFS snapshots down the road
  • I would prefer to have all the backups in one directory, rather than scattered across user profiles

So, here is a new, better recipe.  Note that this recipe will only work with Netatalk 3.1.2 or better.  The current FreeBSD port is version 3.1.3, so that helps.  Firstly, as in the other recipe, the first step is to install netatalk3, and nss_mdns

pkg install netatalk3
pkg install nss_mdns

avahi needs mDNS, so that needs to be configured in /etc/nsswitch.conf.  Ensure that this line exists:

hosts: files dns mdns

Next create a ZFS dataset for the backups.  my zpool is /storage, and the Time Machine backups will be in /storage/timemachine.  Notice that I've turned on compression.

zfs create storage/timemachine
zfs set compression=gzip storage/timemachine
zfs get all storage/timemachine
zfs list

I want to grant the ability to do time machine backups to a FreeBSD group, so I'll make that, and add the users.  This group will be referred to in afp.conf.

pw groupadd timemachine
pw groupmod timemachine -m tom
pw groupshow timemachine

After that, we need to create user directories, one for each time machine user.

mkdir storage/timemachine/tom
chown tom:timemachine /storage/timemachine/tom
chmod 700 /storage/timemachine/tom
chmod 777 /storage/timemachine

So, now that I have a compressed ZFS dataset to store the backups on, backup directories created, and a FreeBSD group created, I can create afp.conf.  I've chosen to limit Time Machine to 500GB space for each user.

;
; Netatalk 3.x configuration file
;

[Global]
vol preset = default_for_all_vol
log file = /var/log/netatalk.log
log level = default:warn 
hosts allow = 192.168.77.0/24
mimic model = TimeCapsule6,116
disconnect time = 1 
vol dbpath = /var/netatalk/CNID/$u/$v/ 

[default_for_all_vol]
file perm = 0640
directory perm = 0750
cnid scheme = dbd

[Homes]
basedir regex = /storage/home
#500 GB (units of MB)
vol size limit = 512000

[TimeMachine]
time machine = yes
path=/storage/timemachine/$u
valid users = @timemachine
#500 GB (units of MB)
vol size limit = 512000

You should notice that the path of the [TimeMachine] share is set to

path=/storage/timemachine/$u

This means that for each logged in user, $u is substituted wth the name of the user. So, when Time Machine logs in as "tom", the data is stored at /storage/timemachine/tom.

Only members of the group "timemachine" will be able to use Time Machine.

This line

vol dbpath = /var/netatalk/CNID/$u/$v/

Ensures that each user as a CNID database for each volume.  Without this line, there is a single CNID database which is shared for all users who are using TimeMachine.  That generally results in corrupting the CNID database.  By specifying /$u/$v, we get a CNID database for each user for each volume, which is much more reliable.

By specifying a disconnect time of one hour:

disconnect time = 1

We can disconnect orphaned sessions and hopefully avoid the dreaded "volume in use" error in TimeMachine.

The rest of the steps are exactly as in the previous blog entry.

You'll need to start dbus, avahi, and netatalk, like this:

/usr/local/etc/rc.d/dbus onestart
/usr/local/etc/rc.d/avahi-daemon onestart
/usr/local/etc/rc.d/netatalk onestart

The next step takes place on your OS X client machine.  On each host that will perform backups, enable Time Machine to see non-TM volumes:

defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1

Then, mount your user's share using afp://.  This will make the share visible to TimeMachine.

After this, you should be able to see your Netatalk shares in Time Machine, and perform backups

15 Responses to "Time Machine backups using FreeBSD ZFS"

  • John
    January 1, 2015 - 7:42 pm Reply

    Thank you. This worked great and was exactly what I needed.

  • Kim Pedersen
    January 31, 2015 - 4:21 pm Reply

    Very nice tutorial!

    only things i experienced was :

    I got no mountpoint when doing zfs create. I had to “zfs create -o mountpoint=/path/to/whatever /pool/whatever

    Then in my /var/log/netatalk.log i got ALOT of errors concerning “vol dbpath = /var/netatalk/CNID/$u/$v/” – but we are only two users so i commented it out.

    And last – on my macbook air (Yosemite) i had to mount the timemachine folder before the share showed up in the TM-options.

    But all in all – very nice and clean tutorial. Thanks alot!

    //Kim

  • Guilherme
    April 19, 2015 - 10:16 pm Reply

    Very nice, thanks a lot! 🙂

  • Notes
    May 20, 2015 - 3:51 am Reply

    Note that you must manually create /var/netatalk/CNID/

    Additionally, on my 10.1-RELEASE, Time Machine does not detect the disk at the end of the process.

  • Nick
    May 21, 2015 - 8:54 am Reply

    Hello!
    Thank you for tutorial!
    One question:
    You sure about option: “vol dbpath = /var/netatalk/CNID/$u/$v/”

    I am install netatalk and see on /var/netatalk/CNID folder for each user who connect to TM.
    I think to use this option is overkill.

  • Matt Martin
    February 2, 2016 - 2:17 pm Reply

    Thanks for the article, it was helpful. I am trying to solve a problem or two that I am having. I am using
    FreeBSD 10.1-RELEASE running current netatalk. The other end is mac OS X 10.11.3, the latest and greatest.

    I can successfully mount a test volume (ZFS on the BSDbox) shared by netatalk on the mac, read and write
    to it. Works great.

    Another share on the BSDbox is a “time machine” enabled share. I can mount it as well on the mac. But the mac will NOT see this mounted share when trying to setup the Time Machine disk. I have run the “defaults” command, and have verified that it is in force. This is the major problem.

    The other thing is there seems to be a permission issue (?) with the cnid_metad trying to create the
    subdirectories in /var/netatalk/CNID — the $u/$v/ directory specified in the afp.con file. It is a good
    idea, can’t figure out why it can’t “mkdir” as reported as the permissions on the CNID directory seem
    right.

    Otherwise, thanks again. If I can figure out what I am doing wrong here, then I can have the network
    backup for the three apples around the house.

  • Chris Ross
    April 5, 2017 - 8:16 am Reply

    I’m following this guide, and other similar, and am having an authentication problem. I can see the server appear on my Mac (FreeBSD 10.3-STABLE, netatalk 3.1.11; Mac is 10.11.8). But, if I try to mount it, it won’t accept my credentials. I tried my full name, my username (same on mac and server), and password. I set up the /usr/local/etc/pam.d/netatalk as I saw elsewhere, but something isn’t working.

    Anyone else seen this “unable to authenticate to mount volume” issue, or know where on the FreeBSD server to look for errors in a log?

    • Chris Ross
      April 5, 2017 - 3:03 pm Reply

      Ignore the above. Turns out that the prebuilt binary for netatalk3 doesn’t build with PAM support. I was able to determine this by looking in /usr/local/libexec/netatalk-uams, where I noted the three sym-links were to the _passwd variants, and the _pam variants weren’t present. Building and installing from ports fixed this issue, and it’s all working as expected now.

      • roens
        July 31, 2017 - 9:32 pm Reply

        Chris…

        I’m also experiencing inability to log in. (FreeBSD 11.0-RELEASE-p9; netatalk 3.1.11; MacOS 10.11.6)

        But as the afp.conf docs indicate, _clrtxt (which equates to both _pam and _password) results in plaintext authentication.

        • roens
          July 31, 2017 - 9:41 pm Reply

          … and as it turns out, the fix was to update libgcrypt!

          After turning on debug logging to a separate file (in afp.conf):
          log file = /var/log/netatalk.log
          log level = default:debug

          … I found complaint about libgcrypt:
          afpd[75054] {uams_dhx2_passwd.c:89} (info:UAMS): PAM DHX2: libgcrypt versions mismatch. Need: 1.8.0

          And, indeed, `pkg info libgcrypt` showed that I still had 1.7.8 installed. And after upgrading it with `pkg install libgcrypt`, and restarting netatalk, I’m now able to auth! And it’s working using the default: _dhx & _dhx2

  • trosdejos
    March 7, 2018 - 6:54 am Reply

    As roens said, updating libgcrypt resolves this issue. It worked for me.
    Great tutorial and great help from roens

  • Andrew
    July 10, 2018 - 8:20 pm Reply

    Thanks for this!

    It worked directly for me — just had to `mkdir /var/netatalk/CNID/(username)`, comment out the Homes section of `afp.conf` (not relevant for me), and then explicitly mount the TimeMachine share on the Mac.

    All easy, up and running in 10mins!

    FREEBSD-11.1-RELEASE, macOS 10.13.4 High Sierra

Leave a Reply