August 2013

Cross-compiling FreeBSD ARM on AMD-64

I’ve been learning about FreeBSD kernel development, and discovering that, not surprisingly, it’s very, very slow to compile on a Raspberry Pi.  So, I decided to figure out how to compile the Pi kernel on a nice fast AMD-64. Firstly, get the kernel sources to a subdirectory under a non-root user profile: mkdir fbsd cd fbsd svn co http://svn0.us-east.freebsd.org/base/head src/FreeBSD/ cd src/FreeBSD Next, we need to set some environment variables: export MAKEOBJDIRPREFIX=/storage/home/tom/fbsd/obj export TARGET_ARCH=armv6 Then, in order to cross-compile we need the kern build tools, so build those: make kernel-toolchain Then finally, build the kernel make buildkernel KERNCONF=RPI-B The completed kernel should show up in ‘/storage/home/tom/fbsd/obj/arm.armv6/storage/home/tom/fbsd/src/FreeBSD/sys/RPI-B’.  There will be a large number of .ko files in there and of course a file named “kernel”.  Copy everything to the directory ‘/boot/kernel’ on the Raspberry Pi, and reboot.

The FreeBSD FDT, the Raspberry PI SPI

How the FDT works In specific, I’m looking at this file bcm2835.dtsi which contains the hardware definitions for the BCM2835, the SOC which powers a Raspberry Pi. Primarily, i’m interested in how to read it.   To really make sense of this, you should have this pdf at hand; it’s the hardware manual for BCM2835.  If you’re interested the FreeBSD code for OpenFirmware is here. devicetree.org has a pretty good explanation of how to read a dts file, here. Looking at the start of the BCM2835 dts, I see this: cpus { cpu@0 { compatible = “arm,1176jzf-s”; }; }; This defines the CPU with the syntax “<manufacturer, model>.  So the BCM2835 has an ARM CPU and the model is “1176jzf-s”. Next I see: SOC: axi { compatible = “simple-bus”; #address-cells = <1>; #size-cells = <1>; reg = <0x20000000 0x01000000>; ranges = <0 0x20000000 0x01000000>; This defines an axi bus, which is compatible with the FreeBSD simple-bus. These two lines: #address-cells = <1>; #size-cells = <1>; Describe that an address cell is 32 bits wide, and that size cells are also 32 bits wide. This line reg = <0x20000000 0x01000000>; describes that the axi bus is memory mapped to the memory range at address 0x20000000  and that the memory range is 0x01000000 bytes wide. So, looking at some peripherals, I see this for the gpio reg = <0x200000 0xb0>; So the gpio is at address 0x200000 within the range specified for the bus, and is 0xb0 cells wide (176, 32-bit bytes).  This is consistent with page 90-91 of the BCM2835 PDF.  Note that the PDF gives the memory range as starting at 0x7E200000.  The PDF specifies memory addresses as bus addresses, but the FDT specifies them as physical addresses.  Page 5 of the PDF explains the difference with a diagram.  Section 1.2.4 on  page 6 of the PDF explains that bus addresses are used in the pdf. The SPI The SPI base address is specified on page 152 of the PDF.  It’s as bus address 0x7E204000 and its 6 – 32bit cells wide. So, an appropriate declaration for SPI0, in polled mode, could look like: spio0 { compatible = “broadcom,bcm2835-spi”; reg = <0x204000 0x06>; };  

The skeleton of a FreeBSD driver

It’s been a goal of mine for a while to understand how FreeBSD device drivers work.  So, firstly I wrote a simple module; it’s here.  The next step is to write the skeleton of a device driver.  That is, a device driver that does nothing.  The resulting source code is here. The Makefile is very similar to that of a module.  In fact, it’s basically identical: SRCS+=ofw_bus_if.h bus_if.h device_if.h foodev.c KMOD=foodev .include <bsd.kmod.mk> I include the headers I need, to inform the make to build them from .m files, and I include my C file which contains my driver.  The last line includes everything else I need to make a driver. The source code for my driver, for the imaginary “foo” device is here. Firstly, I declared a struct called “foodev_softc”.  This struct defines the state data for my driver.  Every driver in FreeBSD can have a softc, containing the drivers state data. struct foodev_softc { device_t sc_dev; }; In the case of my driver I’m storing only the “device_t” that the kernel passed for this device.  However other drivers store resources here such as memory, irqs or DMA. There are three functions declared “foodev_probe”, “foodev_attach” and  “foodev_detach”.  You can learn about those here.  They are called by the kernel, in order when the device is probed, attached and detached. Next, I declare an array of type “device_method_t”, which contains the entry points of the probe, attach and detach methods for my driver: static device_method_t foodev_methods[] = { DEVMETHOD(device_probe, foodev_probe), DEVMETHOD(device_attach, foodev_attach), DEVMETHOD(device_detach, foodev_detach), DEVMETHOD_END }; device_method_t is defined in “bus.h” Similar to a module, there is a struct that defines a driver.  Here is mine: static driver_t foodev_driver = { “foodev”, foodev_methods, sizeof(struct foodev_softc), }; This struct contains the name of my module, the struct which contains my module’s entry points, and the size of the foodev_softc struct, since the kernel needs to know how many bytes of memory to allocate for my driver’s state data.  “driver_t” is defined in bus.h also. Finally, also similar to a module, I use a macro to declare my driver: DRIVER_MODULE(foodev, simplebus, foodev_driver, foodev_devclass, 0, 0); “DRIVER_MODULE” is defined in bus.h The last piece which remains to be explained is this: static devclass_t foodev_devclass; Every driver defines a static instance of “devclass_t”, and “devclass_t” is defined in bus.h.  A “devclass_t” is a pointer to a “devclass”, which you will find in subr_bus.c.  So, the declaration above simply declares a pointer to a “devclass” which the kernel will allocate and use to store information such as the parent of the device and all devices owned by the device. The results of loading the driver on my Pi look like this: root@raspberry-pi:/home/pi/foodev # kldload ./foodev.ko foodev probe foodev probe foodev attach foodev probe foodev probe foodev attach foodev probe foodev probe foodev attach foodev probe foodev probe foodev attach root@raspberry-pi:/home/pi/foodev # kldstat -v | grep foo 5 1 0xc262b000 9000 foodev.ko (./foodev.ko) 108 simplebus/foodev root@raspberry-pi:/home/pi/foodev # kldunload ./foodev.k foodev.kld foodev.ko* root@raspberry-pi:/home/pi/foodev # kldunload ./foodev.ko root@raspberry-pi:/home/pi/foodev # kldstat -v | grep foo dmesg shows this: foodev0 mem 0x2000b400-0x2000b423 irq 0 on simplebus0 foodev1 mem 0x2000b800-0x2000b84f irq 2 on simplebus0 foodev2 mem 0-0xdec0addd on simplebus1 foodev3 mem 0x3-0xdec0ade0 on simplebus1 foodev0: detached foodev1: detached foodev2: detached foodev3: detached    

Writing a FreeBSD loadable module

In preparation for writing a device driver, I thought I would firstly write a module.  The resulting source code is here.   Firstly the Makefile: SRCS+=bus_if.h device_if.h examplemodule.c KMOD=examplemodule .include <bsd.kmod.mk> The first line defines the sources.   examplemodule.c is my module source.  I needed to include bus_if.h and device_if.h since I’m writing my module for newbus, and those files are generated from bus_if.m and device_if.m, respectively.  Including the .h files as sources tells the FreeBSD make that it needs to generate those from the .m files. The second line simply defines the name I wish to give my module.  This Makefile will generate “examplemodule.ko”.   Finally the last line is the magic the FreeBSD team has defined which contains everything else I need to make my module. The entry point for my module looks like this: static int examplemodule_modevent(module_t mod, int type, void *unused) { switch (type) { case MOD_LOAD: uprintf(“Loaded examplemodule \n”); return (0); case MOD_UNLOAD: uprintf(“Unloaded examplemodule \n”); return (0); } return (EINVAL); } When FreeBSD loads my module, or unloads it, this method will be called.  The body of the module is pretty self-explanatory. There is also this: static moduledata_t examplemodule_mod = { “examplemodule”, examplemodule_modevent, 0 }; This is the definition of my module. It contains the name of my module (“examplemodule”) and the entry point “examplemodule_modevent”.  If you’re curious about “moduledata_t”, it’s a struct defined in sys/module.h Finally a macro is used to declare the module.  Of course, this macro refers to the module definition “examplemodule_mod” DECLARE_MODULE(examplemodule, examplemodule_mod, SI_SUB_DRIVERS, SI_ORDER_MIDDLE); The macro “DECLARE_MODULE” is also defined in sys/module.h Loading the module produces this: root@raspberry-pi:/home/pi/freebsd-module # kldload ./examplemodule.ko Loaded examplemodule root@raspberry-pi:/home/pi/freebsd-module # kldunload ./examplemodule.ko Unloaded examplemodule