Bootable FreeBSD image on CompactFlash Card

One of the hacking toys I have is an Alix 1.C board which runs FreeBSD (9.1 at the time of writing). I started out net-booting the board but I thought it would be nice to boot the Alix board from a CompactFlash (CF) card I had laying around, too. In fact, I wanted to create a bootable image for the CF card which I could transfer to the CF card via dd(1).

Pre-requisites

Before the show could start, I needed to find out the size of the CF card I had. I did this by writing all 0s to the card.

root@T61p:~# dd if=/dev/zero of=/dev/sdb bs=512
dd: writing `/dev/sdb': No space left on device
1000945+0 records in
1000944+0 records out
512483328 bytes (512 MB) copied, 1099.51 s, 466 kB/s

I did this by inserting the card in a laptop through an PCCard adapter. The laptop on which I do my hacking runs Linux (Ubuntu 12.10 at the time of writing), so the above is the output of a Linux command. So what this revealed is that the card has 1,000,944 blocks of 512 Bytes which is a total capacity of 512,483,328 Bytes or 488.74 MBytes.

Setting up the Partition Table

Now that we know the size of the CF card, we can create the bootable image. The first step is to create a file of exactly the size as the CF card. This can be done simply by running a command like this:

[phs@freebsd ~]$ dd if=/dev/zero of=freebsd-alix.img bs=512 count=1000944
1000944+0 records in
1000944+0 records out
512483328 bytes transferred in 18.620906 secs (27521933 bytes/sec)

Next, we need a virtual disk based on that image to work with. The solution is to create memory disk as described in the FreeBSD handbook’s section 19.13 on virtual disks. As root, do this:

freebsd# mdconfig -a -t vnode -f /home/phs/freebsd-alix.img -u 0

This creates /dev/md0. Of course, nothing’s on that disk, yet. The first thing to do is creating a partition table.

freebsd# gpart create -s gpt md0
md0 created

This creates an empty UEFI-compatible GPT partition table, but no partitions, yet. You can examine the partition table like this:

freebsd# gpart show
(...)
=>     34  1000877  md0  GPT  (488M)
       34  1000877       - free -  (488M)

In order to boot FreeBSD from the image later, at least two partitions are required: A partition of 512K containing the boot code, and one partition containing the actual operating system. Because I wrote an I2C device driver for the Alix 1.C, I also want a swap partition so I have some space for kernel dumps.

freebsd# gpart add -t freebsd-boot -l boot -s 512K md0
md0p1 added
freebsd# gpart add -t freebsd-swap -l swap -s 64M md0
md0p2 added

Before adding the third partition for the FreeBSD operating system, find out how much space is left on the device by examining the partition table:

freebsd# gpart show md0
=>     34  1000877  md0  GPT  (488M)
       34     1024    1  freebsd-boot  (512k)
     1058   131072    2  freebsd-swap  (64M)
   132130   868781       - free -  (424M)

Ok, now create the data partition:

freebsd# gpart add -t freebsd-ufs -l rootfs -s 868781 md0
md0p3 added

Embedding the boot code

Now that the partition structure is laid out, it’s time to make the image bootable. FreeBSD’s gpart(8) manual page offers great information on what those files are and do: pmbr is a "protective MBF" which allows legacy, BIOS-based systems to boot the image. The MBR code searches the GPT for a freebsd-boot partition and starts the secondary boot code from there. In this case, that’s gptboot which searches the GPT for a freebsd-ufs partition from which it loads /boot/loader.

The following command writes the boot code to the image. Note that I’m using the files from /usr/obj, i.e. the ones most recently build from source and not the one’s currently installed.

freebsd# gpart bootcode -b /usr/obj/usr/src/sys/boot/i386/pmbr/pmbr
  -p /usr/obj/usr/src/sys/boot/i386/gptboot/gptboot -i 1 md0
bootcode written to md0

Initializing the root file system partition

The third partition which is supposed to store the operating system must be initialized with a file system before it can be written.

freebsd# newfs -L FreeBSD -U /dev/gpt/rootfs 
/dev/gpt/rootfs: 424.2MB (868776 sectors) block size 32768, fragment size 4096
        using 4 cylinder groups of 106.06MB, 3394 blks, 13696 inodes.
        with soft updates
super-block backups (for fsck_ffs -b #) at:
 192, 217408, 434624, 651840

This creates an empty UFS2-based file system with soft-updates enabled on the partition which can be mounted:

freebsd# mount /dev/md0p3 /mnt
freebsd# df -h
Filesystem     Size    Used   Avail Capacity  Mounted on
(...)
/dev/md0p3     410M    8.0k    377M     0%    /mnt

Installing FreeBSD to the bootable image

Installing the operating system is as easy as this:

[phs@freebsd /usr/src]$ sudo make installkernel KERNCONF=ALIX1C DESTDIR=/mnt
--------------------------------------------------------------
>>> Installing kernel ALIX1C
--------------------------------------------------------------
(...)
[phs@freebsd /usr/src]$ sudo make installworld DESTDIR=/mnt
(...)
[phs@freebsd /usr/src]$ sudo make DESTDIR=/mnt distrib-dirs distribution
(...)

Before the image is really useful, you probably want to adjust a few config files here and there. Especially etc/fstab should be configured so that the root file system is found and mounted when booting:

[phs@freebsd /mnt]$ cat etc/fstab 
# Device        Mountpoint  FSType  Options Dump    Pass
/dev/ada0p3     /           ufs     ro      0       0
/dev/ada0p2     none        swap    sw      0       0

In my case, the Alix 1.C board has a serial console but no screen. In order to get a login prompt on the serial console, etc/ttys needs to contain a line like this one:

ttyu0   "/usr/libexec/getty std.115200" vt100   on  secure

Finally, a root password must be set:

# pw -V /mnt/etc usermod root -h 0

When done, transfer the image to the CF card using dd(1).

Update Jan 24th, 2013: There are a few more files than just /etc/fstab that should be adjusted – added a few words about those.

Booting a PowerBook from a LiveCD image on an USB-Stick

I have an old Apple PowerBook G4 with a broken CD/DVD drive. For most practical purposes, the broken drive is no issue. However, if you’re going to re-install the laptop, it becomes one. Luckily, the PowerBook is able to boot from USB…

At first, I tried following the instructions on "LiveUSB on PPC" found in Gentoo’s Wiki, but that didn’t work out at first. I then found a blog entry titled "Creating a bootable USB Stick with Mac OS X in 10 easy steps&quot. Combining the two lead to success, so here’s what I did:

  • I downloaded the latest PowerPC release of Finnix, a &quot self-contained, bootable Linux CD distribution&quot from the project’s front page.

  • I re-named the ISO image from finnix-ppc-105.iso to finnix-ppc-105.dmg. Also, I displayed the file’s information in Finder by right-clicking on the file icon and selecting "Show Information&quot. I doubt that this step is required but it certainly didn’t do any harm.

  • From a shell, aka "Terminal Window&quot, I used the command diskutil list to find the device path to my USB drive. In my case, it was /dev/disk6.

  • I then unmounted the drive by running

    $ diskutil unmountDisk /dev/disk6

  • Using good, old dd(1), I wrote the disk image to the USB drive:

    $ sudo dd if=finnix-ppc-105.dmg of=/dev/disk6 bs=1m
  • Finnally, I unmounted the drive by running:

    $ diskutil eject /dev/disk6

In order to boot the PowerBook from the USB drive, I had to drop into Open Firmware. In case you didn’t know it, this is done by holding down Cmd+Option+o+f right after the computer is turned on.

The next step was to find the device node of the bootable USB drive. To do this, I browsed the device tree for any USB node that had a disk child node.

> dev /
> ls

In my case, the USB drive was at /pci@f2000000/usb@1b,1/disk@1.

The instruction found on the Gentoo wiki assign the cd alias to that node, so I did that, too, by running:

> devalias cd /pci@f2000000/usb@1b,1/disk@1

This allowed me to finally boot from the USB drive like this:

> boot cd:,\:tbxi

Ubuntu 11.10 on Lenovo W520

At work, I got a new laptop — a Lenovo W520. It came with Ubuntu 11.10 ("Oneiric Ocelot") pre-installed by the support team. My first impression was that it worked pretty well, but I quickly discovered that I couldn’t change the brightness of the display through the Fn+Home/End keys.

The W520 uses this “Optimus” technology with an integrated on-board graphics card plus a separate NVIDIA card, where both cards can be switched on-the-fly — on Windows. The default installation used the high performance card and I suspected that the video card driver was keeping me from adjusting the brightness. As it turns out, after switching to the on-board Intel card, things worked fine. Here’s what I needed to do:

First, I needed to disable the NVIDIA card in “the BIOS” and switch to the on-board card. There’s an option somewhere under “Setup”, then “Display”, if I recall correctly.

Next, I changed the video card section in my /etc/X11/xorg.conf to read this:

Section "Device"
  Identifier "Device0"
  Driver "intel"
  Option "Shadow" "True"
  Option "DRI" "True"
EndSection

In fact, I added the two Option lines later on and only changed the Driver line at first. I then discovered that most of the little try icons in the upper right corner of the Gnome desktop wouldn’t show anymore. A look at /var/log/Xorg.0.log turned up some errors and running glxinfo yielded lines like these:

Xlib:  extension "GLX" missing on display ":0.0".

Luckily, someone else ran into the same issues over at http://theiszm.wordpress.com/2010/06/27/glx-missing-on-display/. As indicated there, I also ran these commands:

$ sudo apt-get purge nvidia*
$ sudo apt-get install --reinstall xserver-xorg-video-intel 
    libgl1-mesa-glx libgl1-mesa-dri xserver-xorg-core
$ sudo dpkg-reconfigure xserver-xorg
$ sudo update-alternatives --remove gl_conf /usr/lib/nvidia-current/ld.so.conf

The first command hinted that some "ubuntu-desktop" package would also be removed. I don’t know what that is, but I don’t miss it, yet. Anyways, after a final reboot, the brightness adjustment now works and all my tray icons are back in place.

CMake and C++ “Compile Time” Polymorphism

For a recent project of mine, I wanted to use what some people call “Compile Time Polymorphism” in C++. Here’s how I implemented it.

Polymorphism in the context of programming computers usually refers to the ability to tread objects of a different data type through the same interface. In C++, this is often implemented through class inheritance and the use of virtual functions. The text book example of this concept is two classes, Cat and Dog, that inherit from a common super class Animal. Animal has a method makeSound() that is implemented by each subclass accordingly. In real software projects, polymorphism is used to hide multiple implementations behind a uniform interface. Here’s an example of how this concept is usually used in C++.

class Animal {
public:
 void makeSound(void) = 0;
};

class Cat : public Animal {
public:
 void makeSound(void);
};

class Dog : public Animal {
public:
 void makeSound(void);
};

The issue with this code is that it requires the use of virtual functions which means you need a vtable for the concrete subclasses. Usually, as a programmer, you don’t need to worry about vtables as the compiler takes care of that for you. But let’s take a look at how this works anyways. A vtable is basically a table of function pointers. For each of the concrete classes shown above, the vtable contains a pointer to the respective makeSound method. Also, each object carries a pointer to the vtable. At runtime, when a virtual method of an object is called, the pointer to the vtable is resolved to the actual vtable. From there, the address of the method is loaded and the call to it is made indirectly. So the use of virtual methods not only increases the size of your code, but also the size of your objects. In addition to that, it forces the compiler to use indirect function calls through pointers which are usually slower than direct function calls. Again, the compiler takes care of all of that, so this is purely informational.

All of the above is okay and in fact required if you don’t know the concrete type of an object until the software actually runs. Also, in most software projects, the drawbacks don’t matter and aren’t even noticeable. But there are situations where you may not want to pay the price of virtual methods, e.g. in a resource limited embedded system.

Also, there are situations where it is known at compile time what the concrete implementation of an interface will be. This is true for example when you have an abstraction of an interface that is specific to a certain operating system: When you compile the software, you already know what the target operating system will be, so you can simply use and link to the right implementation of the interface, instead of post-poning the decision to runtime.

So how would you use polymorphism in C++ without the use of virtual methods?

Here’s how you could do it:

typedef enum OperatingSystemImpl {
 Darwin,
 FreeBSD,
 Linux
} OperatingSystemImpl_t;

template struct OperatingSystemChoice;

class DarwinImpl;
class FreeBSDImpl;
class LinuxImpl;

template<> struct OperatingSystemChoice {
 typedef DarwinImpl m_type;
};

template<> struct OperatingSystemChoice {
 typedef FreeBSDImpl m_type;
};

template<> struct OperatingSystemChoice {
 typedef LinuxImpl m_type;
};


struct OperatingSystemService {
 typedef OperatingSystemChoice< ... >::m_type m_type;
};

Of course, the ellipsis must be expanded, but more on that later. What’s important is how software using this construct would use the code:

OperatingSystemService::m_type OsServiceObj;

The snipped above would create an object of the correct type, dependend on what the ellipsis expands to. The neat thing is that the template compiler ensures that the ellipsis is expanded to a valid “type” as defined in enum OperatingSystemImpl. Also, it is made sure that the actual, underlying class is declared, e.g. class DarwinImpl.

In other words: If you tried to compile the software with the ellipsis expanded to Windows, you’d get a compilation error. If you had implemented this using classic polymorphism, you’d probably have some code that dynamically created the right object depending on what input is given. That means, you have to test your compiled code, feeding it an invalid type. This mean you must run your software. I’m convinced that finding problems earlier is better, so finding an issue when code is compiled is better than finding issues when code is run.

So back to how the ellipsis is expanded. Here’s how CMake, a build system, comes into play. CMake uses input files that describe how the software needs to be compiled. Those input files, as with other build systems, are able to define compiler flags. Also, CMake defines a variable that contains the operating system’s name. I suspect it’s the output of uname. So here’s what I added to my top level CMakeList.txt file:

add_definitions("-DOPERATING_SYSTEM=${CMAKE_SYSTEM_NAME}")

This makes the OPERATING_SYSTEM macro known to the pre-processor. So the code with the ellipsis can be re-written like this:

struct OperatingSystemService {
 typedef OperatingSystemChoice::m_type m_type;
};

Et voilĂ , the right type is picked when the code is compiled.

Here are the nice things about this: There is no need for virtual methods, eliminating the need for vtables. Also, invalid or unsupported operating system types will be found at compile time vs. at runtime while the code for all supported operating systems will still be always compiled (just not used).

One downside may seem that you no longer have an enforced interface like when using purely virtual classes, i.e. the compiler may not tell you that you forgot to implement a newly added method to one of your implementation classes. However, this is more of a minor issue: You will still get a compilation error in that case, but only if you’re compiling for the target system where you forgot to implement the newly added method.

TianoCore and coreboot, again

It has been over two years since the last time I worked on TianoCore for coreboot. These days I had some free time to spend and I used it to continue the project. I updated my code to the latest versions of coreboot and TianoCore and got it to work again in QEMU.

Here’s a "screenshot" and a few words on what it’s all about.

ImageMagick, libjpeg, etc. on Mac OS X

Here’s how I got ImageMagick with JPEG support to compile and run on Mac OS X 10.6 (Intel).

First, I got the ImageMagick Source Code via Subversion, per the instructions from http://www.imagemagick.org/script/subversion.php. Short version:

$ svn co  
https://www.imagemagick.org/subversion/ImageMagick/branches/ImageMagick-6.6.5  
ImageMagick-6.6.5

Then, I pulled libjpeg from the Independent JPEG Group. I had to extract the source code to a subdirectory of the ImageMagick directory called jpeg, i.e. /path/to/ImageMagick-6.6.5/jpeg.

Before I could compile any of the source code, I had to set three environment variables per this thread on the ImageMagick forums:

$  export CFLAGS="-isysroot /Developer/SDKs/MacOSX10.6.sdk 
  -arch ppc -arch i386"
$ export CXXFLAGS="-isysroot /Developer/SDKs/MacOSX10.6.sdk 
  -arch ppc -arch i386"
$  export LDFLAGS="-Wl,-syslibroot,/Developer/SDKs/MacOSX10.6.sdk 
  -arch ppc -arch i386"

Then, I compiled libjpeg via the via the standard ./configure and make dance. I used these commands:

$ cd jpeg
$ ./configure --prefix=/opt --disable-shared 
  --disable-dependency-tracking
$ make -j 16

Now, I was able to configure ImageMagick:


Be aware that the LDFLAGS path is different than the include path! If everything went well, you can now go on to build the imagemagick suite:

$ ./configure --prefix=/opt --without-x --without-perl --with-jpeg 
   --disable-shared --disable-dependency-tracking 
   --enable-delegate-build --enable-osx-universal-binary
$ make -j 16

This gave me statically linked binaries of the ImageMagick tools I was able to run on my Mac. I also tried to build dynamically linked binaries but failed. Because I don’t need the dynamically linked version, I gave up after a while.

Testing with Google’s C++ Test Framework (gtest)

The other day I was playing around with Google’s C++ Testing Framework (a.k.a. gtest). I tried to build the Code with Visual Studio 2008 Express and got some strange linker errors. I was able to solve the errors thanks to this site. In essence, I had to change the “Runtime Library” Setting in the C/C++ Code Generation options to “Multi-Threaded” for a release build and to “Multi-Threaded Debug” for a debug build.

Linker Sets

Reminder: When placing something in a dedicated section using __attribute__((section("foobar"))), the GNU toolchain will automatically add a symbol __start_foobar at the beginning and a symbol __stop_foobar at the end of the section.

However, you will need a reference to that symbol in order to prevent the linker from optimizing the symbol away. In other words, you need to declare something like extern void *__start_foobar; and use it.

When using the Microsoft toolchain, the symbols need to be added explicitly. To do that, one can make use of the fact that when the Microsoft linker encounters several sections with a “$_” in their name, it will merge the contents into one final output section. The name of the output section will be the common prefix of the declared sections. The beauty is that the contents are in the order of the section names.

Here’s an example: Supposed you placed something into a section called “foobar$_something”. You can then add a variable __start_foobar into a section “foobar$_a” and a variable __stop_foobar into a section “foobar$_z”. The resulting binary will have one section “foobar” with the contents of variable __start_foobar placed at the beginning, followed the contents of everything in section “foobar$_something” and the contents of the variable __stop_foobar at the end.

Fix Windows Full Text search

I’ve recently noticed that using the Windows full text search may not always turn up the expected results. Apparently, Windows requires a program to install a search filter for a given file type. There is some plain text filter available by default, but it’s only registered for some endings. Source code files, e.g. Groovy files, will not be included, even though they contain nothing but plain ASCII text. Anyways, there’s a fix available, but it’s nowhere near intuitive…