FreeBSD

Spelunking Apple DOS 3.3 DSK files

If you happen to have had an Apple][, you might be interested in retrieving your data from old DOS 3.3 floppies.  Or maybe you’re just interested in retrocomputing and downloaded some floppies at Apple2Online.  In my case, I had an ulterior motive.  I’ve been learning about FreeBSD device drivers and was wondering how difficult it would be to add support for Apple][ DOS 3.3 to FreeBSD.  I used Virtual][ as a desktop emulator to verify my DSK files. In short; use the md device to mount a DSK image, which is simply a binary dump of a floppy disk.  This is exactly how FreeBSD hackers build .img files or mount .iso images.  easy. Prior to writing code, the first task was to investigate the DSK image.  I used this image, for nostalgic reasons, however any valid DSK file should work.  I referenced this document for technical details of the VTOC, which I won’t explain here.  It is, important, however, that on a DOS 3.3 disk, the VTOC is located on track 17, sector 0. A DOS 3.3 disk has these characteristics: 35 tracks per disk 16 sectors per track 256 bytes per sector So a simple formula, in base-10, to find the byte offset of the first byte of track T, sector S is: Offset = T*16*256 + S*256 For a typical disk with 35 tracks, the total storage space works out to 35*16*256 = 143360 bytes / disk.  Not surprisingly, thats the byte size of a DSK image.   The first byte of the VTOC by the above formula, is therefore 17*16*256 = 69632. The arrangement of the VTOC, from the manuals is this: Byte        Description 00 Not Used 01 Track number of first catalog sector 02 Sector number of first catalog sector 03 DOS release number 04-05 Not Used 06 Volume number 07 – 26 Not Used 27 Maximum number of track-sector pairs which will fit in a track-sector list 28 – 2F Not Used 30 Last allocated track 31 Track allocation direction 32 – 33 Not Used 34 Tracks / diskette 35 Sectors / track 36 – 37 Bytes / sector 38 – FF Track-Sector allocation bitmaps In my case the first couple bytes of the VTOC are: 04 11 0F 03 00 00 FE 00 Disk catalog is at track 0x11, sector 0x0F.  Not surprisingly track 0x11 is track 17 in decimal. DOS 3.3 disk Volume number 0xFE Once we’ve read the VTOC, we can find the disk catalog, which contains the list of every file on the disk.  In my case the byte offset for the catalog would be 69632 + 10*256 = 72192 (0x11A00) A catalog sector looks like Byte        Description 00 Not Used 01 Track number of next catalog sector 02 Sector number of next catalog sector 03 – 0A Not used 0B – FF File descriptor entries (34 bytes each) There is room for 7 file descriptor entries in a catalog sector, each having this format Byte        Description 00 Track of the first track-sector list entry 01 Sector of the first track-sector list entry 02 File type 03 – 20 File name 21 – 22 File length, in units of sectors The start of my catalog is: 00 11 09 00 00 00 00 00 00 00 00 0F 0C 84 C7 CF D4 C8 C9 C3 AE D3 C5 D4 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 05 The next catalog sector is at track 0x11, sector 0x09 The first track sector list for the first file is at track 0x0F, sector 0x0C The file type is 0x84 (a locked binary file) The file name bytes are “C7 CF D4 C8 C9 C3 AE D3 C5 D4”. The file is 5 sectors in size The A0 character bytes are ASCII extended character codes for spaces.  The file names are ASCII with the high bit set, so the bytes “C7 CF D4 C8 C9 C3 AE D3 C5 D4”, as decimal with the high bit unset are “71 79 84 72 73 67 46 80 69 81” which is “G O T H I C . S E T”.   Therefore the file name is “GOTHIC.SET”. The track-sector list for “GOTHIC.SET” is track 0x0F, sector 0x0C.  The offset, in decimal, is therefore: 15*16*256+12*256 = 61440 + 3072 = 64512 The format of a track-sector list is: Byte        Description 00 Not Used 01 Track number of next track-sector list, or 00 02 Sector number of next track-sector list 03 – 04 Not used 05 – 06 Sector offset 07 – 0B Note used 0C – FF Track and sector pairs for data sectors, or 00 The bytes at this offset are: 00 00 00 00 00 00 00 00 00 00 00 00 0F 0B 0F 0A 0F 09 0F 08 There is no next track-sector list The track and sector pairs are for data for this file are all on track 0x0F, and they are sectors {0x0B, 0x0A, 0x09, 0x08}. So, to read this file, I would read the entire data of those four sectors.

Building FreeBSD-9.2 for Soekris using Crochet-BSD

Crochet-BSD is a github project which supports building FreeBSD disk images for various platforms such a RaspberryPi, BeagleBoard and others.  I had a Soekris 4501 laying around, so I decided to extend Crochet to build images for it. So, firstly, I needed a kernel.  Building the kernel config for Soekris is explained pretty well here.  Essentially: Add support for AMD Geode Remove everything that isn’t needed, for example SCSI support Add in various useful network goodies, such as pf and ALTQ. The resulting kernel configuration is here. The basic configuration for a new Crochet board is very well explained here.  I simply copied that and made modifications as necessary.   My Github fork of Crochet is here.  The end result is a handful of changes which ended up merged into Crochet-BSD. The boot log is here.  If you want to try the image on your Soekris box, it’s here. It turns out that building the kernel for Soekris is quite easy.   There are, however, some gotchas in building a working image: /boot/loader.conf needs to specify that the console device for loader is serial. /boot.config needs to pass “-h” to tell the boot loader to use the serial console /etc/ttys needs to be configured so that FreeBSD will use the Soekris RS-232 port as the system console.

Cross-compiling FreeBSD ARM on AMD-64

I’ve been learning about FreeBSD kernel development, and discovering that, not surprisingly, it’s very, very slow to compile on a Raspberry Pi.  So, I decided to figure out how to compile the Pi kernel on a nice fast AMD-64. Firstly, get the kernel sources to a subdirectory under a non-root user profile: mkdir fbsd cd fbsd svn co http://svn0.us-east.freebsd.org/base/head src/FreeBSD/ cd src/FreeBSD Next, we need to set some environment variables: export MAKEOBJDIRPREFIX=/storage/home/tom/fbsd/obj export TARGET_ARCH=armv6 Then, in order to cross-compile we need the kern build tools, so build those: make kernel-toolchain Then finally, build the kernel make buildkernel KERNCONF=RPI-B The completed kernel should show up in ‘/storage/home/tom/fbsd/obj/arm.armv6/storage/home/tom/fbsd/src/FreeBSD/sys/RPI-B’.  There will be a large number of .ko files in there and of course a file named “kernel”.  Copy everything to the directory ‘/boot/kernel’ on the Raspberry Pi, and reboot.

The FreeBSD FDT, the Raspberry PI SPI

How the FDT works In specific, I’m looking at this file bcm2835.dtsi which contains the hardware definitions for the BCM2835, the SOC which powers a Raspberry Pi. Primarily, i’m interested in how to read it.   To really make sense of this, you should have this pdf at hand; it’s the hardware manual for BCM2835.  If you’re interested the FreeBSD code for OpenFirmware is here. devicetree.org has a pretty good explanation of how to read a dts file, here. Looking at the start of the BCM2835 dts, I see this: cpus { cpu@0 { compatible = “arm,1176jzf-s”; }; }; This defines the CPU with the syntax “<manufacturer, model>.  So the BCM2835 has an ARM CPU and the model is “1176jzf-s”. Next I see: SOC: axi { compatible = “simple-bus”; #address-cells = <1>; #size-cells = <1>; reg = <0x20000000 0x01000000>; ranges = <0 0x20000000 0x01000000>; This defines an axi bus, which is compatible with the FreeBSD simple-bus. These two lines: #address-cells = <1>; #size-cells = <1>; Describe that an address cell is 32 bits wide, and that size cells are also 32 bits wide. This line reg = <0x20000000 0x01000000>; describes that the axi bus is memory mapped to the memory range at address 0x20000000  and that the memory range is 0x01000000 bytes wide. So, looking at some peripherals, I see this for the gpio reg = <0x200000 0xb0>; So the gpio is at address 0x200000 within the range specified for the bus, and is 0xb0 cells wide (176, 32-bit bytes).  This is consistent with page 90-91 of the BCM2835 PDF.  Note that the PDF gives the memory range as starting at 0x7E200000.  The PDF specifies memory addresses as bus addresses, but the FDT specifies them as physical addresses.  Page 5 of the PDF explains the difference with a diagram.  Section 1.2.4 on  page 6 of the PDF explains that bus addresses are used in the pdf. The SPI The SPI base address is specified on page 152 of the PDF.  It’s as bus address 0x7E204000 and its 6 – 32bit cells wide. So, an appropriate declaration for SPI0, in polled mode, could look like: spio0 { compatible = “broadcom,bcm2835-spi”; reg = <0x204000 0x06>; };  

The skeleton of a FreeBSD driver

It’s been a goal of mine for a while to understand how FreeBSD device drivers work.  So, firstly I wrote a simple module; it’s here.  The next step is to write the skeleton of a device driver.  That is, a device driver that does nothing.  The resulting source code is here. The Makefile is very similar to that of a module.  In fact, it’s basically identical: SRCS+=ofw_bus_if.h bus_if.h device_if.h foodev.c KMOD=foodev .include <bsd.kmod.mk> I include the headers I need, to inform the make to build them from .m files, and I include my C file which contains my driver.  The last line includes everything else I need to make a driver. The source code for my driver, for the imaginary “foo” device is here. Firstly, I declared a struct called “foodev_softc”.  This struct defines the state data for my driver.  Every driver in FreeBSD can have a softc, containing the drivers state data. struct foodev_softc { device_t sc_dev; }; In the case of my driver I’m storing only the “device_t” that the kernel passed for this device.  However other drivers store resources here such as memory, irqs or DMA. There are three functions declared “foodev_probe”, “foodev_attach” and  “foodev_detach”.  You can learn about those here.  They are called by the kernel, in order when the device is probed, attached and detached. Next, I declare an array of type “device_method_t”, which contains the entry points of the probe, attach and detach methods for my driver: static device_method_t foodev_methods[] = { DEVMETHOD(device_probe, foodev_probe), DEVMETHOD(device_attach, foodev_attach), DEVMETHOD(device_detach, foodev_detach), DEVMETHOD_END }; device_method_t is defined in “bus.h” Similar to a module, there is a struct that defines a driver.  Here is mine: static driver_t foodev_driver = { “foodev”, foodev_methods, sizeof(struct foodev_softc), }; This struct contains the name of my module, the struct which contains my module’s entry points, and the size of the foodev_softc struct, since the kernel needs to know how many bytes of memory to allocate for my driver’s state data.  “driver_t” is defined in bus.h also. Finally, also similar to a module, I use a macro to declare my driver: DRIVER_MODULE(foodev, simplebus, foodev_driver, foodev_devclass, 0, 0); “DRIVER_MODULE” is defined in bus.h The last piece which remains to be explained is this: static devclass_t foodev_devclass; Every driver defines a static instance of “devclass_t”, and “devclass_t” is defined in bus.h.  A “devclass_t” is a pointer to a “devclass”, which you will find in subr_bus.c.  So, the declaration above simply declares a pointer to a “devclass” which the kernel will allocate and use to store information such as the parent of the device and all devices owned by the device. The results of loading the driver on my Pi look like this: root@raspberry-pi:/home/pi/foodev # kldload ./foodev.ko foodev probe foodev probe foodev attach foodev probe foodev probe foodev attach foodev probe foodev probe foodev attach foodev probe foodev probe foodev attach root@raspberry-pi:/home/pi/foodev # kldstat -v | grep foo 5 1 0xc262b000 9000 foodev.ko (./foodev.ko) 108 simplebus/foodev root@raspberry-pi:/home/pi/foodev # kldunload ./foodev.k foodev.kld foodev.ko* root@raspberry-pi:/home/pi/foodev # kldunload ./foodev.ko root@raspberry-pi:/home/pi/foodev # kldstat -v | grep foo dmesg shows this: foodev0 mem 0x2000b400-0x2000b423 irq 0 on simplebus0 foodev1 mem 0x2000b800-0x2000b84f irq 2 on simplebus0 foodev2 mem 0-0xdec0addd on simplebus1 foodev3 mem 0x3-0xdec0ade0 on simplebus1 foodev0: detached foodev1: detached foodev2: detached foodev3: detached    

Writing a FreeBSD loadable module

In preparation for writing a device driver, I thought I would firstly write a module.  The resulting source code is here.   Firstly the Makefile: SRCS+=bus_if.h device_if.h examplemodule.c KMOD=examplemodule .include <bsd.kmod.mk> The first line defines the sources.   examplemodule.c is my module source.  I needed to include bus_if.h and device_if.h since I’m writing my module for newbus, and those files are generated from bus_if.m and device_if.m, respectively.  Including the .h files as sources tells the FreeBSD make that it needs to generate those from the .m files. The second line simply defines the name I wish to give my module.  This Makefile will generate “examplemodule.ko”.   Finally the last line is the magic the FreeBSD team has defined which contains everything else I need to make my module. The entry point for my module looks like this: static int examplemodule_modevent(module_t mod, int type, void *unused) { switch (type) { case MOD_LOAD: uprintf(“Loaded examplemodule \n”); return (0); case MOD_UNLOAD: uprintf(“Unloaded examplemodule \n”); return (0); } return (EINVAL); } When FreeBSD loads my module, or unloads it, this method will be called.  The body of the module is pretty self-explanatory. There is also this: static moduledata_t examplemodule_mod = { “examplemodule”, examplemodule_modevent, 0 }; This is the definition of my module. It contains the name of my module (“examplemodule”) and the entry point “examplemodule_modevent”.  If you’re curious about “moduledata_t”, it’s a struct defined in sys/module.h Finally a macro is used to declare the module.  Of course, this macro refers to the module definition “examplemodule_mod” DECLARE_MODULE(examplemodule, examplemodule_mod, SI_SUB_DRIVERS, SI_ORDER_MIDDLE); The macro “DECLARE_MODULE” is also defined in sys/module.h Loading the module produces this: root@raspberry-pi:/home/pi/freebsd-module # kldload ./examplemodule.ko Loaded examplemodule root@raspberry-pi:/home/pi/freebsd-module # kldunload ./examplemodule.ko Unloaded examplemodule  

Building a Beaglebone FreeBSD Kernel

Building a FreeBSD kernel for Beaglebone is not terribly difficult.  Firstly, you need the FreeBSD-Current sources: svn co http://svn0.us-east.freebsd.org/base/head /src/FreeBSD/ Next, build the cross-development tools for FreeBSD: cd /src/FreeBSD/head make XDEV=arm XDEV_ARCH=armv6 xdev Then get Crochet-FreeBSD.  This is by far the easiest way to build a FreeBSD kernel for Beaglebone git clone git://github.com/kientzle/crochet-freebsd.git Once you have Crochet, you’ll need u-boot. You can get the latest sources here.  You’ll need to drop them into your crochet directory. Crochet needs the “auditdistd” user, so if you’re not on FreeBSD-10, or CURRENT, you’ll need to add that user: pw useradd auditdistd  -s /usr/sbin/nologin -g wheel Finally, once you’ve editted “config.sh” to set up the source directory for FreeBSD and to select the Beaglebone build, you can run it: sh crochet.sh -c config.sh Once crochet has finished, it will have produced an image file into the /work/ subdirectory of crochet-freebsd.  Write that image file to a sd card. dd if=FreeBSD-armv6-BEAGLEBONE.img of=/dev/rdisk1 bs=1m You’ll likely want to set up a serial console.  In order to do that, you need one of these, a PL2303 serial cable.  You can get one here.  The pin connections are as described at circuitco. Function      Beagleboard    Wire Ground        Pin1           Black Receive       Pin4           Green Transmit     Pin5           White On OS X, you’ll need PL2303 serial drivers.  You can get those here.  You should then have this device on OS X /dev/tty.PL2303-000013FA Once the Beagleboard is plugged in, you can get a serial console like this: screen /dev/tty.PL2303-000013FA 115200 To boot the SD card, follow the instructions here.  For simplicity sake, I have included the instructions: Disconnect the power to the Beagleboard Connect the serial cable to both the Beagleboard and the computer Insert the micro-SD card Hold the Boot switch.  This is the switch at the opposite end of the board from the LEDs, adjacent to the SD card slot Connect the power to the Beagleboard Release the Boot switch In my case, I had connected the serial cable and brought up screen previously; that way I could see the boot-up messages. If you want to try it for yourself, the .img file is here.  

Beaglebone Serial Console on OS X

I recently got a Beaglebone Black.   I was very excited; it’s similar to a Pi, but has 60 GPIO ports! Getting a serial console up on it was not quite a simple as I thought, since the BeagleBone uses FTDI.  Following the instructions here I installed the OS X serial driver for FTDI.   After plugging in the device, I had this new serial device: /dev/tty.usbmodemfa133 So to get a serial console up I typed: screen /dev/tty.usbmodemfa133 115200 Once I had the Beaglebone started and had a network connection, the next step was to update the package manager.  If you’re interested, the default OS on the Beaglebone is Angstrom Linux, and the package system is Itsy.  You can view the available packages here. opkg update opkg upgrade Since I had a USB Wifi Adapter, it seemed to make sense to use it.  My adapter is an PCUSBW1150. opkg list ‘linux-firmware-rt*’opkg install linux-firmware-rtl8192cu After a reboot, I now have a wlan0 device: wlan0 Link encap:Ethernet HWaddr 00:02:72:C3:AB:72 UP BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) Angstrom uses connman to manage network settings.  My connman configuration for my wifi is in the file ‘/var/lib/connman/wifi.config’ and looks like: [service_home] Type=wifi Name=shrek2.5 Security=wpa Passphrase=passpasspass Once connman is configured, restart it systemctl restart connman.service Finally, some additional configuration. I installed nano opkg install nano To change the hostname, edit the file ‘/etc/hostname’  

The mystery of uart0

The FDT file for Raspberry Pi, declares UART.  I’ve had quite a bit of trouble understanding how declaring a uart in the FDT results in /dev/tty0 and /dev/cuau0 appearing in /dev/ Here are the relevant lines from the FDT: uart0: uart0 { compatible = “broadcom,bcm2835-uart”, “broadcom,bcm2708-uart”, “arm,pl011”, “arm,primecell”; reg = <0x201000 0x1000>; interrupts = <65>; interrupt-parent = <&intc>; clock-frequency = <3000000>; /* Set by VideoCore */ reg-shift = <2>; }; Eventually this uart device gets bound to the sio driver, and exposed as /dev/cuau0 and /dev/tty0. From the FDT declaration I can see that: There are some declarations of the devices that this uart driver is compatible with It uses irq 65, and the parent interrupt controller is “intc” which is declared in the same FDT There are some registers declared It’s also interesting to note that the uart device is not a child of the gpio device, it’s a child of axi, the bus. There is a clue, in the kernel conf for Pi, here; it declares a uart device and pl011 device uart device pl011 The source tree for the uart devices is at /sys/dev/uart ad pl011 is at /sys/dev/uart/uart_dev_pl011.c. Therefore: The kernel conf file specifies that uart and pl011 need to be compiled into the kernel The FDT file declares that there is a uart and that it’s compatible with pl011. The file /sys/dev/uart/uart_bus_fdt.c, is the uart bus driver for devices with a FDT . When the kernel boots, it will call “probe” on this device.  The code below probes the FDT (via Open Firmware) and will detect that a pl011 is compatible. static int uart_fdt_probe(device_t dev) { struct uart_softc *sc; phandle_t node; pcell_t clock, shift; int err; sc = device_get_softc(dev); if (ofw_bus_is_compatible(dev, “ns16550”)) sc->sc_class = &uart_ns8250_class; else if (ofw_bus_is_compatible(dev, “lpc,uart”)) sc->sc_class = &uart_lpc_class; else if (ofw_bus_is_compatible(dev, “fsl,imx-uart”)) sc->sc_class = &uart_imx_class; else if (ofw_bus_is_compatible(dev, “arm,pl011”)) sc->sc_class = &uart_pl011_class; else if (ofw_bus_is_compatible(dev, “cadence,uart”)) sc->sc_class = &uart_cdnc_class; else return (ENXIO); node = ofw_bus_get_node(dev); if ((err = uart_fdt_get_clock(node, &clock)) != 0) return (err); uart_fdt_get_shift(node, &shift); return (uart_bus_probe(dev, (int)shift, (int)clock, 0, 0)); } So, the UART bus device knows to bind the device that was specified in the FDT, because it matches this string “arm, pl011”. So, the mystery is solved this way: The kernel conf ensures that the uart and pl011 devices are compiled in The FDT declares the uart and declares that its compatible with “arm, pl011” which is a string that pl011 recognizes The hardware configuration that the pl011 needs, it reads via Open Firmware, which in turn reads from the FDT blob If you look at the datasheet for the BCM2835, you will see that there is indeed a Primecell pl011 UART on the chip.   On page 177 of the datasheet there is this: The PL011 USRT is mapped on base adderss 0x7E20100. This address mapping of “0x7e20100” matches what was declared int the FDT above. reg = <0x201000 0x1000>; The reason for the difference between 0x7E20100 and 0X201000 is the I/O mapping on the ARM chip, described on page 6 of the datasheet.