“Parallels” for Linux

Ben has an interesting post on how to boot Windows XP using KVM on Fedora Core 9. The interesting part is that Windows XP is installed on the host’s hard disk. His instructions almost work verbatim, but there’s one exception. Since I’m using KVM-73, the QEMU command is:

$ qemu-system-x86_64 -hda /dev/sda -net nic -net user -m 1024 
    -cdrom fixntldr.iso -boot d -std-vga 

This will also give the guest system access to the network.

Encrypted Devices/Filesystems on Linux

Yesterday I tried to encrypt a complete USB Stick under Linux. I followed this tutorial and it worked quite well.

Mounting the encrypted device isn’t as obvious as could be, so here it goes:

$ cryptsetup create <symbolic name> <device name>
$ mount /dev/mapper/<symbolic name> <mountpoint>

The beginnings of coreboot and TianoCore

In order to create a UEFI payload for coreboot, I’ve started a coreboot platform as part of the TianoCore EDK II. The sources for the platform can be obtained here. Note that the CorebootPkg directory must be placed in the TianoCore $WORKSPACE directory.

To build the package on FreeBSD, a GNU toolchain from vendor sources must be used. This is because the TianoCore tools use some compiler/linker flags unknown to the toolchain included in the FreeBSD base system. The path as well as the names of the toolchain binaries must be adjusted in Conf/tools_def.txt. Because I built the toolchain according to these instructions, the preprocessor will not look in /usr/include for headers which causes errors in the ProcessorBind.h header when it attempts to include stdint.h. This patch can be applied to fix this.

Note that the build process still cannot complete as the tools producing the final Firmware Volume (FV) cannot cope with the ELF binaries produced by the GNU toolchain.

TianoCore and the Python-based Build Process, Part 3

This is part III of my attempts to build the TianoCore EDK II with the Python-based tools. In order to circumvent the error that stopped me in part II, the build process needs to be taught to use GNU make, i.e. gmake, on FreeBSD instead of make, which is BSD make. This can be done by editing the *_ELFGCC_*_MAKE_PATH variable in Conf/tools_def.txt.

The tools_def.txt file is automatically copied from a template part of the BaseTools sources. This patch fixes the template so the changes described above do not have to be applied manually.

At this point, the build process starts and does actually build some modules. However, the UnixPkg cannot be built completely on FreeBSD. This is because the code makes some assumptions only true on Linux, e.g. the presence of the sys/vfs.h header.

TianoCore and the Python-based Build Process, Part 2

So here’s the "sequel" to Part One. This time I’m trying to actually build a Firmware Volume with the Python-based tools.

Prerequisites for build.py

The core of the tools is build.py, a Python script which invokes the tools in order to build a Firmware Volume (FV). On FreeBSD, build.py cannot be run until the following requirements are met:

  • SQLite3 for Python, which can be installed through the databases/py-sqlite3 port.
  • The Python module for ANTLR, a parser generator.
  • Installing the module mentioned above requires EasyInstall, or rather: I don’t know how it can be done otherwise.

Because I could not find a port for EasyInstall, I did this to install the script on FreeBSD:

$ fetch http://peak.telecommunity.com/dist/ez_setup.py
$ chmod +x ez_setup.py
$ ./ez_setup.py

Note that this isn’t the whole truth as the path to the interpreter in the script, i.e. the first line aka "shebang", must be adjusted to /usr/local/bin/python before the script can be executed.

After that, the easy_install command is available and the ANTLR module can be installed by running this:

$ eazy_install 
  http://www.antlr.org/download/Python/antlr_python_runtime-3.0.1-py2.5.egg

Running build.py

In theory, running build.py and thus building a Firmware Volume should be as easy as this:

$ cd path/to/edk2
$ export PYTHONPATH=/path/to/basetools/Source/Python
$ . edksetup.sh BaseTools
$ python $PYTHONPATH/build/build.py

Unfortunately, the last step initially aborted with this error:

build...
 : error 5000: Please execute /home/phs/sandbox/basetools/Bin/FreeBSD-i386:/sbin: 
/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin: 
/home/phs/bin/edksetup.bat to set /home/phs/sandbox/basetools/Bin/Freebsd7 in 
environment variable: PATH!



- Failed -

After some try’n’error time, I think that the above error is caused by user error: I had previously copied the compiled C programs from Source/C/bin to bin/FreeBSD-i386 (paths relative to /path/to/basetools). After removing bin/FreeBSD-i386, I created a link to BinWrappers/PosixLike at the same location:

$ cd /path/to/basetools
$ ln -s BinWrappers/PosixLike Bin/FreeBSD-i386

I then re-ran build.py (see above) and it produced some output that didn’t look like errors:

00:44:09, Sep.21 2008 [FreeBSD-7.1-PRERELEASE-i386-32bit-ELF]

WORKSPACE                = /usr/home/phs/sandbox/edk2
EDK_SOURCE               = /usr/home/phs/sandbox/edk2/EdkCompatibilityPkg
EFI_SOURCE               = /usr/home/phs/sandbox/edk2/EdkCompatibilityPkg
EDK_TOOLS_PATH           = /home/phs/sandbox/basetools

TARGET_ARCH              = IA32
TARGET                   = DEBUG
TOOL_CHAIN_TAG           = ELFGCC

Active Platform          = UnixPkg/UnixPkg.dsc
Flash Image Definition   = UnixPkg/UnixPkg.fdf

Processing meta-data . . . . . . .

Unfortunately, though, right after the dots, an error occured:

build...
UnixPkg/UnixPkg.dsc(...): error 4000: Instance of library class [NetLib] is not found
        in [MdeModulePkg/Universal/Network/ArpDxe/ArpDxe.inf] [IA32]
        consumed by module [MdeModulePkg/Universal/Network/ArpDxe/ArpDxe.inf]
 

- Failed -
00:44:17, Sep.21 2008 [00:08]

Fixing the UnixPkg

The UnixPkg part of the EDK II seems to be broken as the error above indicates a dependency error between modules which is caused by an incorrect platform definition file (*.dsc). Applying this patch fixes the problem.

The patch ensures that all dependencies are met, but the build process still fails with this error:

Processing meta-data . . . . . . . . done!
make: don't know how to make pbuild. Stop


build...
 : error 7000: Failed to execute command
        make pbuild [/usr/home/phs/sandbox/edk2/Build/Unix/DEBUG_ELFGCC/IA32/MdePkg/Library/BaseTimerLibNullTemplate/BaseTimerLibNullTemplate]

Waiting for all build threads exit...
make: don't know how to make pbuild. Stop


build...
 : error 7000: Failed to execute command
        make pbuild [/usr/home/phs/sandbox/edk2/Build/Unix/DEBUG_ELFGCC/IA32/MdePkg/Library/BaseLib/BaseLib]


build...
 : error F002: Failed to build module
        MdePkg/Library/BaseTimerLibNullTemplate/BaseTimerLibNullTemplate.inf [IA32, ELFGCC, DEBUG]

- Failed -
01:01:43, Sep.21 2008 [00:09]

Oh, well, to be continued…

TianoCore and the Python-based Build Process

Now that I can use coreboot and libpayload on FreeBSD, it’s time to try the new Python-based build process for the TianoCore EDK II on FreeBSD.

Prerequisites are:

Note that Subversion access requires a user account at the TianoCore project.

Installing the e2fs-libuuid port

This is trivially easy, just do:

$ cd /usr/ports/misc/e2fs-libuuid
$ sudo make install

That’s all. The headers and libraries are installed under /usr/local.

Building the Base Tools

Compiling the Base Tools, i.e. the Python-based TianoCore build tools, isn’t complicated but doesn’t work out of the box, either. First, these two patches (patch 1, patch 2) must be applied:

$ cd /path/to/basetools
$ patch -p0 < basetools_include.diff
$ patch -p0 < basetools_make.diff

The first patch adjusts some include paths so that /usr/local/include is searched, too, which is required in order to find the uuid/uuid.h header. The second patch replaces invocations of make to use the $(MAKE) variable which holds the name of invoked the make binary. This is required as in FreeBSD (and other BSDs), make is not GNU make, however the latter is required to build the Base Tools. Consequently, when building the project, make sure that gmake is used:

$ gmake

Compiling the EDK II

To be continued...

Hacking coreboot and libpayload

After some quiet time, I picked up a project I started a while ago: Hacking coreboot and libpayload on FreeBSD.

Building coreboot

On FreeBSD, building coreboot requires a toolchain built from the GNU sources. If the stock toolchain is used, the build process dies with this error:

CC      build/stage0.init
/usr/home/phs/sandbox/coreboot-v3/build/arch/x86/stage0_i586.o(.text+0xf): In function `_stage0':
: relocation truncated to fit: R_386_16 gdtptr
gmake: *** [/usr/home/phs/sandbox/coreboot-v3/build/stage0.init] Error 1

The solution is to build a compiler as described here and here and then set the $CROSS environment variable accordingly, e.g. like this:

$ export CROSS=/opt/bin/i386-unknown-linux-gnu-

Note that the above requires that this patch is applied. After that, build the bios.bin binary using GNU make.

$ gmake menuconfig
$ gmake

Building libpayload

Compiling libpayload is trickier than building coreboot as the build files assume that the world is Linux. First, sh is not always bash. Second, the header and library search paths are screwed up as they don’t include /usr/local subdirectores. Third, the gettext library is installed as libgettextlib.so on FreeBSD and must be linked against the program explicitly. And finally, the install(1) tool has different parameters than on Linux. Oh, and there are no stdarg.h and stddef.h headers.

I’ve hacked around those issues, the Mercurial repository is available at https://phs.subgra.de/hg/libpayload.

Hacking SLOF Clients

SLOF includes a few clients. Especially the net-snk client is interesting as it allows one to add applications which can be started by specifying their name on the command line when booting the kernel.

In order to hack SLOF client applications, the sec-client target must be built in the clients/net-snk subdirectory of the SLOF source tree. Before this can succeed, the CPUARCH environment variable must be set, like this:

$ export CPUARCH=ppc970

Latest Version of KOMAscript

The Tex Live distribution Mac OS X is getting more outdated by the day. Maybe there’s a way to update it, but I didn’t bother to check.

For a reasonably good layout of letters without a footer I found that a recent version of KOMAscript is required. I’m using Version 2.98 obtained from http://dante.ctan.org/.

Installing the new version proved to be quite easy:

$ cd /usr/local/texlive/2007/texmf-dist
$ sudo unzip komascript.tds.zip
$ sudo texconfig rehash

Cross compiling the FreeBSD Kernel

Wow, two post in one day already 😉

There are two things I’d like to note. First, I noticed that cross-compiling seems a major issue for me. Don’t know why that is.

Second, I need to remember Warner Losh’s post on cross compiling FreeBSD. Essentialy, the procedure is:

$ export TARGET=powerpc
$ export TARGET_ARCH=powerpc
$ make kernel-toolchain
$ make buildkernel

Addendum: Unfortunately, this procedure works only on architectures already supported by FreeBSD and its build system. Therefore, it doesn’t work for me. So here’s the full story on how I got FreeBSD to at least compile.

Building the Toolchain

Building the toolchain is pretty straight forward. I’ve already written about how to build a cross compiler. On FreeBSD however, four things are different.

  • The target is powerpc64-unknown-freebsd. I don’t know if powerpc64-unknown-elf would have worked, too.
  • The target is new, so a patch to the binutils sources is required.
  • The GNU MP Bignum Library (GMP) is required. I used version GMP 4.2.1 and installed it in $PREFIX
  • Finally, the MPFT Library needs to be built. I used version MPFT 2.3.0 and installed it in $PREFIX

Note that those steps have to be performed before the compiler is built. Since I didn’t install the libraries in the standard locations, the LD_LIBRARY_PATH variable needs to be set before the compiler can be used:

$ export LD_LIBRARY_PATH=$PREFIX/lib

Building the Kernel

The basic procedure of building a kernel is outlined in the FreeBSD Developer’s Handbook. Provided that the cross compiler has been installed in $PREFIX, these steps are required:

$ export MACHINE=powerpc64
$ export MACHINE_ARCH=powerpc64
$ export CC=${PREFIX}/${TARGET_PREFIX}-gcc
$ export NM=${PREFIX}/${TARGET_PREFIX}-nm
$ export LD=${PREFIX}/${TARGET_PREFIX}-ld
$ export SYSTEM_LD=${LD}
$ export OBJCOPY=${PREFIX}/${TARGET_PREFIX}-objcopy
$ cd /usr/src/sys/powerpc64/conf
$ config KERNCONF
$ cd ../compile/KERNCONF
$ make cleandepend && make depend
$ make kernel

Oh, of course this doesn’t work with the stock FreeBSD sources. Instead, my FreeBSD 64-Bit PowerPC Mercurial repository needs to be used. Note that for convenience reasons, that tree includes a crossenv script which, when sourced, sets up the required environment variables.