Bare Metal coding on FreeBSD

I recently got interested in the technical details of how ARM OS's work, so I decided to try my hand at writing a simple one.  This blog post is not about the OS itself, but about setting up the development environment.

In my case, I'm developing in a terminal session, on FreeBSD 10 on an AMD-64 host, so I'll need to cross-compile all my code.  Luckily, the ports tree includes gcc-arm-embedded a port of the launchpad ARM cross tools.  It's easy to install:

pkg install gcc-arm-embedded

This package includes all the tools which are needed:

-rwxr-xr-x 1 root wheel 711488 Oct 3 11:17 arm-none-eabi-addr2line
-rwxr-xr-x 2 root wheel 740040 Oct 3 11:17 arm-none-eabi-ar
-rwxr-xr-x 2 root wheel 1298680 Oct 3 11:17 arm-none-eabi-as
-rwxr-xr-x 2 root wheel 620816 Oct 3 11:17 arm-none-eabi-c++
-rwxr-xr-x 1 root wheel 710528 Oct 3 11:17 arm-none-eabi-c++filt
-rwxr-xr-x 1 root wheel 620608 Oct 3 11:17 arm-none-eabi-cpp
-rwxr-xr-x 1 root wheel 29416 Oct 3 11:17 arm-none-eabi-elfedit
-rwxr-xr-x 2 root wheel 620816 Oct 3 11:17 arm-none-eabi-g++
-rwxr-xr-x 2 root wheel 620608 Oct 3 11:17 arm-none-eabi-gcc
-rwxr-xr-x 2 root wheel 620608 Oct 3 11:17 arm-none-eabi-gcc-4.8.4
-rwxr-xr-x 1 root wheel 24480 Oct 3 11:17 arm-none-eabi-gcc-ar
-rwxr-xr-x 1 root wheel 24448 Oct 3 11:17 arm-none-eabi-gcc-nm
-rwxr-xr-x 1 root wheel 24448 Oct 3 11:17 arm-none-eabi-gcc-ranlib
-rwxr-xr-x 1 root wheel 271072 Oct 3 11:17 arm-none-eabi-gcov
-rwxr-xr-x 1 root wheel 3992568 Oct 3 11:17 arm-none-eabi-gdb
-rwxr-xr-x 1 root wheel 776672 Oct 3 11:17 arm-none-eabi-gprof
-rwxr-xr-x 4 root wheel 1025912 Oct 3 11:17 arm-none-eabi-ld
-rwxr-xr-x 4 root wheel 1025912 Oct 3 11:17 arm-none-eabi-ld.bfd
-rwxr-xr-x 2 root wheel 722928 Oct 3 11:17 arm-none-eabi-nm
-rwxr-xr-x 2 root wheel 906848 Oct 3 11:17 arm-none-eabi-objcopy
-rwxr-xr-x 2 root wheel 1123424 Oct 3 11:17 arm-none-eabi-objdump
-rwxr-xr-x 2 root wheel 740056 Oct 3 11:17 arm-none-eabi-ranlib
-rwxr-xr-x 1 root wheel 365208 Oct 3 11:17 arm-none-eabi-readelf
-rwxr-xr-x 1 root wheel 712976 Oct 3 11:17 arm-none-eabi-size
-rwxr-xr-x 1 root wheel 712080 Oct 3 11:17 arm-none-eabi-strings
-rwxr-xr-x 2 root wheel 906864 Oct 3 11:17 arm-none-eabi-strip

Additionally, an ARM simulator such as QEMU will be needed.  FreeBSD also include that port:

pkg install qemu-devel

I can easily use BSD Make, however I prefer GNU Make, so I've installed that too

pkg install gmake

With these tools installed, I have enough to cross-compile ARM assembler and C code, link it, and run it in QEMU and debug with GDB.

FemtoOS; a simple bootloader and protected mode kernel

I've recently become interested in how the i386 boot loader works.  There is an excellent example of a boot loader here, and another here.  Some simple protected-mode code which implements a kernel capable of writing a line of text, is here.  FemtoOS, is the culmination of combining code from all those, and building a simple boot loader plus a protected mode kernel, that outputs text.  In my case, I'm compiling the kernel on FreeBSD.

The bootloader is in boot.asm.  Like any i386 boot loader, it's loaded by the BIOS at address 0xC700, in 16-bit real mode.

It starts by clearing the screen, outputting a message, and then loading kernel.bin from the disk.  kernel.bin is loaded at address 0x1000.  The bootloader then enters protected mode, sets up a GDT and passes control to kernel.bin at address 0x1000.  Looking at the documentation for BIOS interrupt 13 here, it's clear that a single sector has 512 bytes.  boot.bin is exactly 512 bytes long and the floppy image was created using this code from Makefile.  Therefore kernel.bin starts at the second sector on the disk.

cat boot.bin kernel.bin /dev/zero | dd bs=512 count=2880 of=floppy.img

This code from boot.asm reads 18 sectors, starting at sector 02, into RAM at 0x1000.  The largest kernel.bin can be, therefore is 512*18=9KB.

 mov ax, 0                              
 mov es, ax                              
 mov bx, 0x1000          ; Destination address = 0000:1000
 mov ah, 02h             ; READ SECTOR-command
 mov al, 12h             ; Number of sectors to read (0x12 = 18 sectors)
 mov dl, [drive]         ; Load boot disk
 mov ch, 0               ; Cylinder = 0
 mov cl, 2               ; Starting Sector = 3
 mov dh, 0               ; Head = 1
 int 13h                 ; Call interrupt 13h

kernel.bin is linked from the object files created from loader.asm, main.c and video.c.  When the bootloader passes control to kernel.bin, it starts at loader.asm which in turn passes control to main() from main.c.  Note that while boot.asm contains both 16-bit and 32-bit code, loader.asm contains only 32-bit code; it is called after boot.asm has put the host in 32-bit protected mode.

This code sets up the GDT

 xor ax, ax              ; Clear AX register
 mov ds, ax              ; Set DS-register to 0 - used by lgdt
 lgdt [gdt_desc]         ; Load the GDT descriptor

and this code puts the machine into protected mode, followed by passing control to loader.asm.  Note that immediately after putting the machine into protected mode a jmp instruction is issued to the label "kernel_segments" which is the first 32 bit instruction executed on boot.  From this point on, we are in 32 bit protected mode.

 mov eax, cr0            ; Copy the contents of CR0 into EAX
 or eax, 1               ; Set bit 0     (0xFE = Real Mode)
 mov cr0, eax            ; Copy the contents of EAX into CR0       
 jmp 08h:kernel_segments ; Jump to code segment, offset kernel_segments
        
[BITS 32]                ; We now need 32-bit instructions
kernel_segments:
 mov ax, 10h             ; Save data segment identifyer
 mov ds, ax              ; Move a valid data segment into the data segment register
 mov ss, ax              ; Move a valid data segment into the stack segment register
 mov esp, 090000h        ; Move the stack pointer to 090000h       
 jmp 08h:0x1000          ; Jump to section 08h (code), offset 01000h

The code in loader.asm that call's the C code main() is pretty simple:

start:
  call main  ; Call our kernel's main() function

There is good documentation for the protected mode text console here.  The simple implementation of this is in video.c.

There is a floppy disk image here, which boots in both qemu and VMWare.

Time Machine backups using FreeBSD ZFS

In this blog article, I described a way to use Netatalk3 to do Time Machine backups on FreeBSD.  This approach worked well, but it had some problems:

  • The Time Machine backups are in every user's home dir.  That's messy and there is the potential that they'll accidentally delete the backup.
  • If I put the backups on a ZFS disk, I can compress them
  • I would like the potential to use ZFS snapshots down the road
  • I would prefer to have all the backups in one directory, rather than scattered across user profiles

So, here is a new, better recipe.  Note that this recipe will only work with Netatalk 3.1.2 or better.  The current FreeBSD port is version 3.1.3, so that helps.  Firstly, as in the other recipe, the first step is to install netatalk3, and nss_mdns

pkg install netatalk3
pkg install nss_mdns

avahi needs mDNS, so that needs to be configured in /etc/nsswitch.conf.  Ensure that this line exists:

hosts: files dns mdns

Next create a ZFS dataset for the backups.  my zpool is /storage, and the Time Machine backups will be in /storage/timemachine.  Notice that I've turned on compression.

zfs create storage/timemachine
zfs set compression=gzip storage/timemachine
zfs get all storage/timemachine
zfs list

I want to grant the ability to do time machine backups to a FreeBSD group, so I'll make that, and add the users.  This group will be referred to in afp.conf.

pw groupadd timemachine
pw groupmod timemachine -m tom
pw groupshow timemachine

After that, we need to create user directories, one for each time machine user.

mkdir storage/timemachine/tom
chown tom:timemachine /storage/timemachine/tom
chmod 700 /storage/timemachine/tom
chmod 777 /storage/timemachine

So, now that I have a compressed ZFS dataset to store the backups on, backup directories created, and a FreeBSD group created, I can create afp.conf.  I've chosen to limit Time Machine to 500GB space for each user.

;
; Netatalk 3.x configuration file
;

[Global]
vol preset = default_for_all_vol
log file = /var/log/netatalk.log
log level = default:warn 
hosts allow = 192.168.77.0/24
mimic model = TimeCapsule6,116
disconnect time = 1 
vol dbpath = /var/netatalk/CNID/$u/$v/ 

[default_for_all_vol]
file perm = 0640
directory perm = 0750
cnid scheme = dbd

[Homes]
basedir regex = /storage/home
#500 GB (units of MB)
vol size limit = 512000

[TimeMachine]
time machine = yes
path=/storage/timemachine/$u
valid users = @timemachine
#500 GB (units of MB)
vol size limit = 512000

You should notice that the path of the [TimeMachine] share is set to

path=/storage/timemachine/$u

This means that for each logged in user, $u is substituted wth the name of the user. So, when Time Machine logs in as "tom", the data is stored at /storage/timemachine/tom.

Only members of the group "timemachine" will be able to use Time Machine.

This line

vol dbpath = /var/netatalk/CNID/$u/$v/

Ensures that each user as a CNID database for each volume.  Without this line, there is a single CNID database which is shared for all users who are using TimeMachine.  That generally results in corrupting the CNID database.  By specifying /$u/$v, we get a CNID database for each user for each volume, which is much more reliable.

By specifying a disconnect time of one hour:

disconnect time = 1

We can disconnect orphaned sessions and hopefully avoid the dreaded "volume in use" error in TimeMachine.

The rest of the steps are exactly as in the previous blog entry.

You'll need to start dbus, avahi, and netatalk, like this:

/usr/local/etc/rc.d/dbus onestart
/usr/local/etc/rc.d/avahi-daemon onestart
/usr/local/etc/rc.d/netatalk onestart

The next step takes place on your OS X client machine.  On each host that will perform backups, enable Time Machine to see non-TM volumes:

defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1

Then, mount your user's share using afp://.  This will make the share visible to TimeMachine.

After this, you should be able to see your Netatalk shares in Time Machine, and perform backups

TimeMachine backups on FreeBSD-10

I usually work on a Macbook, and I use keyless ssh with a cron job to do nightly backups.  It works very well.  However, I decided to try backups with Time Machine, using my FreeBSD-10 server as the host.

The first step is to install netatalk3, and nss_mdns

pkg install netatalk3
pkg install nss_mdns

avahi needs mDNS, so that needs to be configured in /etc/nsswitch.conf.  Ensure that this line exists:

hosts: files dns mdns

Finally, configure netatalk.  My configuration in /usr/local/etc/afp.conf looks like

[Global]
vol preset = default_for_all_vol
log file = /var/log/netatalk.log
hosts allow = 192.168.77.0/24
mimic model = TimeCapsule6,116

[default_for_all_vol]
file perm = 0640
directory perm = 0750
cnid scheme = dbd

[Homes]
basedir regex = /storage/home
time machine = yes

You'll need to start dbus, avahi, and netatalk, like this:

/usr/local/etc/rc.d/dbus onestart
/usr/local/etc/rc.d/avahi-daemon onestart
/usr/local/etc/rc.d/netatalk onestart

The next step takes place on your OS X client machine.  On each host that will perform backups, enable Time Machine to see non-TM volumes:

defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1

Then, mount your user's share using afp://.  This will make the share visible to TimeMachine.

After this, you should be able to see your Netatalk shares in Time Machine, and perform backups

Building U-boot 2014 on FreeBSD

I'm upgrading the U-boot version used for Wandboard by Crochet-BSD.    The first step is to upgrade the Device Tree Compiler (DTC); there are instructions here.

Once you have the U-boot 2014 source, the build is trivial:

gmake SED=gsed HOSTCC=cc CROSS_COMPILE=armv6-freebsd- wandboard_quad_config
gmake SED=gsed HOSTCC=cc CROSS_COMPILE=armv6-freebsd-

 

Building u-boot for Chromebook

I got interested in building Crochet images for Chromebook.  The first step, naturally is to build U-boot for Chromebook.

In order to build U-boot, you'll need to have already installed

  • gmake
  • gcc

You can get these from the ports tree.  You'll also need xdev, the FreeBSD cross-compiler toolkit.

I started with the latest U-boot 2014 source, which I got here.  The latest U-boot requires DTC 1.4.  I'm on FreeBSD 10, which has a much older DTC.

To build the latest DTC:

git clone git://git.kernel.org/pub/scm/utils/dtc/dtc.git
cd dtc
gmake CC=gcc
cp dtc /usr/bin

Once we have DTC 1.4, we can build U-boot.

wget ftp://ftp.denx.de/pub/u-boot/u-boot-2014.07.tar.bz2
tar zxvf u-boot-2014.07.tar.bz2
cd u-boot-2014.07
gmake SED=gsed HOSTCC=cc CROSS_COMPILE=armv6-freebsd- snow_config
gmake SED=gsed HOSTCC=cc CROSS_COMPILE=armv6-freebsd-

The U-boot binary image is "u-boot.bin". To burn the U-boot binary to a SD card:

sudo dd if=u-boot.bin of=/dev/rdisk1

The next challenge is the file system. The file system on the Chromebook looks like this:

       start        size    part  contents
           0           1          PMBR (Boot GUID: E780FED6-7991-B94B-B622-FA7541BE9FBB)
           1           1          Pri GPT header
           2          32          Pri GPT table
     8671232    21295104       1  Label: "STATE"
                                  Type: Linux data
                                  UUID: D5488177-EB19-3243-8D7C-D6E4F01DB5B3
       20480       32768       2  Label: "KERN-A"
                                  Type: ChromeOS kernel
                                  UUID: 47715F23-7DD6-5545-93DC-3807D1CD4AFD
                                  Attr: priority=0 tries=0 successful=0
     4476928     4194304       3  Label: "ROOT-A"
                                  Type: ChromeOS rootfs
                                  UUID: BE1064FF-EC28-E64B-B3A1-A5C176D57C0F
       53248       32768       4  Label: "KERN-B"
                                  Type: ChromeOS kernel
                                  UUID: 5D200D96-1F52-804C-83A7-AD8081E8ACD4
                                  Attr: priority=1 tries=0 successful=1
      282624     4194304       5  Label: "ROOT-B"
                                  Type: ChromeOS rootfs
                                  UUID: D0795A2A-1D0D-4140-9F57-F87AEF399C44
       16448           1       6  Label: "KERN-C"
                                  Type: ChromeOS kernel
                                  UUID: E8A703B8-E0A5-964D-9508-D88DE336D003
                                  Attr: priority=0 tries=15 successful=0
       16449           1       7  Label: "ROOT-C"
                                  Type: ChromeOS rootfs
                                  UUID: 7A947ABA-E723-8440-9A07-0EC394C85195
       86016       32768       8  Label: "OEM"
                                  Type: Linux data
                                  UUID: 173B3C97-CDCF-F849-99E7-7614C6C0F9FD
       16450           1       9  Label: "reserved"
                                  Type: ChromeOS reserved
                                  UUID: CD83D43A-4FC8-4849-B0DD-368E9C3819BA
       16451           1      10  Label: "reserved"
                                  Type: ChromeOS reserved
                                  UUID: 344D3149-A5C2-B44B-A65C-2DFD59758FC5
          64       16384      11  Label: "RWFW"
                                  Type: ChromeOS firmware
                                  UUID: D9B79232-94AD-A143-9C49-F947BFA2491E
      249856       32768      12  Label: "EFI-SYSTEM"
                                  Type: EFI System Partition
                                  UUID: E780FED6-7991-B94B-B622-FA7541BE9FBB
    30777311          32          Sec GPT table
    30777343           1          Sec GPT header

There is a good description of the Chromium OS file system here.  The size parameter is in sectors, which are usually 512 bytes each.  So, for example the state partition is 10GB in size.  KERN-A is 16MB in size, and ROOT-A is 2GB.

The partition ids for ChromeOS kernel and rootfs, are "fe3a2a5d-4f32-41a7-b725-accc3285a309" and "3cb8e202-3b7e-47dd-8a3c-7ff2a13cfcec" respectively.  The partition ids for ChromeOS are documented here.