TianoCore on QEMU

There is a bios.bin binary file for use with QEMU available at http://fabrice.bellard.free.fr/qemu/efi-bios.tar.bz2. It is meant to be used as a BIOS replacement for QEMU and provides an EFI interface. It is compiled from the TianoCore sources, at least that’s what the QEMU homepage suggests.

The problem with this file is that it can only be used with very few versions of QEMU, that’s why I’m writing this.

I’ve had success with version 0.9.0 when the patches linked from the coreboot wiki were applied. I’ve also had success with a CVS snapshot from July 3rd, 2007. Version 0.9.1 or the stock 0.9.0 do not work.

TianoCore on IBM’s OpenClient

I’ve started work at IBM yesterday. Today I found a room and got all my user accounts and passwords set up so I can actually start to work.

I’m trying to build the TianoCore EDKII (SVN Revision 4792) on IBM’s own Linux distribution, called OpenClient. The distribution sucks but it’s the only kind of Linux we’re allowed to use around here. It’s based on Red Hat (I think), but it feels "different".

To bootstrap the TianoCore Build Environment, I sort of followed my FreeBSD and Fedora Core notes. Here’s what I did:

  • Installed Java JDK 6 Update 4 (JDK 1.6.0_04) for Linux through the self-extracting binary file (not the RPM) that Sun provides. I placed the JDK in /opt.
  • Installed the binary distribution of Apache Ant 1.7.0, Saxon 8.1.1, XMLBeans 2.1.0 as well as ant-contrib 1.0b3 and placed all of them in /opt.
  • Created the symlinks:
    $ cd /opt/apache-ant-1.7.0/lib
    $ sudo ln -s /opt/ant-contrib/ant-contrib-1.0b3.jar ant-contrib.jar
    $ sudo ln -sf /opt/saxonb8.1.1/saxon8.jar /opt/xmlbeans-2.1.0/lib/saxon8.jar
    

I then created a small script that automates the build tool bootstrapping process:

export JAVA_HOME=/opt/jdk1.6.0_04
export XMLBEANS_HOME=/opt/xmlbeans-2.1.0
export ANT_HOME=/opt/apache-ant-1.7.0
export WORKSPACE=/home/phs/Sources/edk2
export PATH=$PATH:$XMLBEANS_HOME/bin:$ANT_HOME/bin
. edksetup.sh ForceRebuild

Sourcing the script successfully builds the build tools. Then, in the file $WORKSPACE/Tools/Conf/target.txt, two settings need to be adjusted:

ACTIVE_PLATFORM=EdkUnixPkg/Unix.fpd
TOOL_CHAIN_TAG=ELFGCC

Of course, the previously mentioned patch needs to be applied. After that, the EDKII Unix Emulation Package can be built and run as described in the tutorial:

$ cd $WORKSPACE/
$ build
$ cd Build/Unix
$ . run.cmd

I found that the IBM OpenClient distribution already includes the e2fsprogs-devel package as well as the relevant X11 development packages. Please also note that it is not neccessary to build an PE32+ cross compiler on Linux.

TianoCore on Fredora Core 8

My attempts building anything of the TianoCore EDK2 codebase on FreeBSD/amd64 have been extremely frustrating so far. I guess trying to build on an unsupported operating system and on an unsupported architecture may have been too much of a hurdle for the beginning. So I figured I’d try building the EDK Unix Simulator (Trunk Revision 4679) on Fredora Core 8. Here’s what I’ve done.

I pretty much followed this tutorial. It’s for Gentoo, so a little tweaking was needed.

First, I downloaded and installed the JDK 6 from SUN via the RPMs they provide. The JDK ends up in /usr/java/jdk1.6.0_04, so the JAVA_HOME environment variable needs to be set to that.

Second, I didn’t bother installing the needed (Java) tools through the Fedora Package Manager but instead downloaded the files manually and placed them under /opt. This is similar to what I did for my FreeBSD attempts. I needed two symlinks like the tutorial says:

$ cd /opt/apache-ant-1.7.0/lib
$ sudo ln -s /opt/ant-contrib/ant-contrib-1.0b3.jar ant-contrib.jar
$ sudo ln -sf /opt/saxonb8.1.1/saxon8.jar /opt/xmlbeans-2.1.0/lib/saxon8.jar

Third, I needed to install the e2fsprogs-devel package. The e2fsprogs package (without the “-devel” suffix) isn’t enough. Also, I had to install the X development packages. I don’t know what exact package was needed, but the Fedora Core 8 package manager has this option that lets you install some pre-selected packages related to X development.

Forth, I had to apply the following patch (the tutorial mentions this):

Index: EdkModulePkg/Bus/Pci/PciBus/Dxe/PciHotPlugSupport.c
===================================================================
--- EdkModulePkg/Bus/Pci/PciBus/Dxe/PciHotPlugSupport.c (Revision 4679)
+++ EdkModulePkg/Bus/Pci/PciBus/Dxe/PciHotPlugSupport.c (Arbeitskopie)
@@ -21,7 +21,7 @@

 --*/

-#include "Pcibus.h"
+#include "pcibus.h"
 #include "PciHotPlugSupport.h"

 EFI_PCI_HOT_PLUG_INIT_PROTOCOL  *gPciHotPlugInit;

Finally, contrary to what the tutorial says, I used the following script to set up the environment. Note that I didn’t include the TOOL_CHAIN line and that I didn’t build the PE/COFF capable GCC.

export JAVA_HOME=/usr/java/jdk1.6.0_04
export XMLBEANS_HOME=/opt/xmlbeans-2.1.0 
export ANT_HOME=/opt/apache-ant-1.7.0
export WORKSPACE=/home/phs/edk2/edk2
export PATH="$PATH:$XMLBEANS_HOME/bin:$ANT_HOME/bin" 

The build went smoothly after that and I was able to use the EDK Unix environment. Note that the thing should not be called “Unix” Package, since it heavily assumes that it runs on Linux in some areas.

TianoCore on FreeBSD/amd64, Take 2

Now that I have build a cross compiler, I’m trying to build the TianoCore EDK again.

In this post I list a few environment variables that need to be set in order to build the EDK. Because the build process needs to use the cross compiler, another environment variable is needed:

$ export CC=/opt/i386-tiano-pe/bin/gcc

Since I’m lazy and all, I put everything in a little script called env.sh:

export WORKSPACE=/home/phs/edk2
export JAVA_HOME=/usr/local/diablo-jdk1.5.0
export ANT_HOME=/opt/apache-ant-1.6.5
export XMLBEANS_HOME=/opt/xmlbeans-2.1.0
export PATH=$PATH:$ANT_HOME/bin:$XMLBEANS_HOME/bin
export CC=/opt/i386-tiano-pe/bin/gcc

Also, the EDK build notes mention that the default build target is the Windows NT Emulation environment. However, that target cannot be built using GCC, so I needed to edit the file Tools/Conf/target.txt and change the ACTIVE_PLATFORM to:

ACTIVE_PLATFORM       = EdkUnixPkg/Unix.fpd

Now, I can tip of the build process as follows:

$ cd ~/edk2
$ . env.sh
$ . edksetup.sh newbuild

Note that the build script must be “sourced”, not executed. Unfortunately, the build process still fails, but now with a different error:

       [cc] 1 total files to be compiled.
       [cc] In file included from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.h:3,
       [cc]                  from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:17:
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:27:20: error: jni_md.h: No such file or directory
       [cc] In file included from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.h:3,
       [cc]                  from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:17:
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:45: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jsize'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:104: error: expected specifier-qualifier-list before 'jbyte'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:193: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1834: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1842: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1851: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1888: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1927: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1930: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1933: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1937: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1940: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'void'
       [cc] In file included from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:17:
       [cc] /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jbyteArray'
       [cc] /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:29: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jbyteArray'

BUILD FAILED
/home/phs/edk2/Tools/build.xml:22: The following error occurred while executing this line:
/home/phs/edk2/Tools/CCode/Source/build.xml:254: The following error occurred while executing this line:
/home/phs/edk2/Tools/CCode/Source/CompressDll/build.xml:45: gcc failed with return code 1

The root cause of this is that the compiler can’t find the jni_md.h header which is located in $JAVA_HOME/include/freebsd (at least on my system). I worked around this problem by editing $JAVA_HOME/include/jni.h as follows:

--- jni.h.orig  2008-02-06 11:50:05.000000000 +0100
+++ jni.h       2008-02-06 11:50:16.000000000 +0100
@@ -24,7 +24,7 @@
 /* jni_md.h contains the machine-dependent typedefs for jbyte, jint
    and jlong */
 
-#include "jni_md.h"
+#include "freebsd/jni_md.h"
 
 #ifdef __cplusplus
 extern "C" {

Now I’m stuck with yet another compilation error:

init:
     [echo] Building the EDK Tool Library: CompressDll

Lib:
       [cc] 1 total files to be compiled.
       [cc] /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c: In function 'Java_org_tianocore_framework_tasks_Compress_CallCompress':
       [cc] /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:57: warning: overflow in implicit constant conversion
       [cc] Starting link
       [cc] /usr/bin/ld: /home/phs/edk2/Tools/CCode/Source/Library/libCommonTools.a(EfiCompress.o): relocation R_X86_64_32S can not be used when making a shared object; recompile with -fPIC
       [cc] /home/phs/edk2/Tools/CCode/Source/Library/libCommonTools.a: could not read symbols: Bad value

BUILD FAILED
/home/phs/edk2/Tools/build.xml:22: The following error occurred while executing this line:
/home/phs/edk2/Tools/CCode/Source/build.xml:254: The following error occurred while executing this line:
/home/phs/edk2/Tools/CCode/Source/CompressDll/build.xml:45: gcc failed with return code 1

Well, I guess we’ll see where this ends…

Building a Cross Compiler on FreeBSD

I’m currently trying to build a cross compiler (and other required tools) on FreeBSD. The compiler will run on FreeBSD/amd64 and should produce i386 binaries. This wouldn’t be too hard since that task can easily be accomplished by using the FreeBSD source tree. However, I need the toolchain to produce binaries in the PE/COFF format instead of the default ELF format.

Building the toolchain is somewhat tricky, at least I found it to be poorly documented. But maybe I just looked in the wrong places. Building the toolchain requires:

  • Building binutils.
  • Locating some header files.
  • Building the compiler.
  • Building a C Library.
  • Building the compiler again so it uses the C library that was just built.

For my needs, I found that the last two steps weren’t needed. I wrote a script that downloads the sources, extracts the archives and builds the toolchain. Here’s the script in full quote (I really wish there was a way to upload files to this thing):

#/bin/sh

#
# Copyright (c) 2008 Philip Schulz 
#

#
# This script downloads, builds and installs GCC and GNU Binutils that can
# produce x86 binaries in PE/COFF Format. The cross toolchain needs some headers
# that aren't usually present on the host system. However, those headers can be
# obtained from the cygwin sources, that's why a snapshot of the cygwin sources
# is downloaded.
#
# After the script finishes, the tools will be located at
# ${PREFIX}/${TARGET_ARCH}/bin. Some other binaries will be installed in
# ${PREFIX}/bin with their own prefix of ${TARGET_ARCH} but I don't know that
# they are for.
#

# Prefix where the Cross-Tools will live
PREFIX="${PREFIX:-/opt}"

# Target architecture.
TARGET_CPU="${TARGET_CPU:-i386}"
TARGET_ARCH=${TARGET_CPU}-tiano-pe

# Program that can fetch the files.
FETCH_COMMAND="/usr/bin/ftp -V"

# GNU Make
GNU_MAKE=`which gmake`

################################################################################
#
# GCC settings.
#
################################################################################
# What version of GCC will be fetched, built and installed 
GCC_VERSION=gcc-4.2.3
# What mirror to use.
GCC_MIRROR=ftp://ftp-stud.fht-esslingen.de/pub/Mirrors/ftp.gnu.org
# File name of the GCC sources. Should probably not be changed.
GCC_ARCHIVE=$GCC_VERSION.tar.bz2
# Where the GCC Sources can be fetched from. Should probably not be changed.
GCC_URL=$GCC_MIRROR/gcc/$GCC_VERSION/$GCC_ARCHIVE
# Arguments for the GCC configure script. Should probably not be changed.
GCC_CONFIGURE_ARGS="--prefix=${PREFIX} --target=${TARGET_ARCH} "
GCC_CONFIGURE_ARGS+="--with-gnu-as --with-gnu-ld --with-newlib "
GCC_CONFIGURE_ARGS+="--disable-libssp --disable-nls --enable-languages=c "
GCC_CONFIGURE_ARGS+="--program-prefix=${TARGET_ARCH}- "
GCC_CONFIGURE_ARGS+="--program-suffix=-4.2.3 "


################################################################################
#
# Binutils settings.
#
################################################################################
# What version of the GNU binutils will be fetched, build and installed
BINUTILS_VERSION=binutils-2.18
# What mirror to use.
BINUTILS_MIRROR=ftp://ftp-stud.fht-esslingen.de/pub/Mirrors/ftp.gnu.org
# File name of the binutils sources. Should probably not be changed.
BINUTILS_ARCHIVE=$BINUTILS_VERSION.tar.gz
# Where the GCC Sources can be fetched from. Should probably not be changed.
BINUTILS_URL=$BINUTILS_MIRROR/binutils/$BINUTILS_ARCHIVE
# Arguments for the GCC configure script. Should probably not be changed.
BINUTILS_CONFIGURE_ARGS="--prefix=${PREFIX} --target=${TARGET_ARCH} "
BINUTILS_CONFIGURE_ARGS+="--disable-nls "

################################################################################
#
# Cygwin settings.
#
################################################################################
CYGWIN_SNAPSHOT=20080129
CYGWIN_ARCHIVE=cygwin-src-${CYGWIN_SNAPSHOT}.tar.bz2
CYGWIN_MIRROR=http://cygwin.com/
CYGWIN_URL=${CYGWIN_MIRROR}snapshots/${CYGWIN_ARCHIVE}
CYGWIN_DIR=cygwin-snapshot-${CYGWIN_SNAPSHOT}-1

################################################################################
#
# Batch code.
#
################################################################################
#
# Fetches the files.
#
do_fetch() {
        if [ ! ( -f $GCC_ARCHIVE ) ] ; then
                echo "Fetching ${GCC_URL}"
                ${FETCH_COMMAND} ${GCC_URL}
        else
                echo $GCC_ARCHIVE already locally present.
        fi

        if [ ! ( -f $CYGWIN_ARCHIVE ) ] ; then
                echo "Fetching ${CYGWIN_URL}"
                ${FETCH_COMMAND} ${CYGWIN_URL}
        else
                echo $CYGWIN_ARCHIVE already locally present.
        fi

        if [ ! ( -f $BINUTILS_ARCHIVE ) ] ; then
                echo "Fetching ${BINUTILS_URL}"
                ${FETCH_COMMAND} ${BINUTILS_URL}
        else
                echo $BINUTILS_ARCHIVE already locally present.
        fi
}

#
# Extracts the archives.
#
do_extract() {
        # Remove already extracted files first.
        rm -rf ${GCC_VERSION}
        rm -rf ${CYGWIN_DIR}
        rm -rf ${BINUTILS_VERSION}

        # Extract the archives
        if [ -f $GCC_ARCHIVE ] ; then
                echo "Extracting ${GCC_ARCHIVE}"
                tar -jxf ${GCC_ARCHIVE}
        fi

        if [ -f $CYGWIN_ARCHIVE ] ; then
                echo "Extracting ${CYGWIN_ARCHIVE}"
                tar -jxf ${CYGWIN_ARCHIVE}
        fi

        if [ -f $BINUTILS_ARCHIVE ] ; then
                echo "Extracting ${BINUTILS_ARCHIVE}"
                tar -xzf ${BINUTILS_ARCHIVE}
        fi
}


BUILD_DIR_PREFIX=build-

#
# Builds Binutils.
#
do_binutils_build() {
        BUILD_DIR_BINUTILS=${BUILD_DIR_PREFIX}binutils-${TARGET_ARCH}

        # Remove dir if it exists.
        if [ -d $BUILD_DIR_BINUTILS ] ; then
                rm -rf $BUILD_DIR_BINUTILS
        fi

        echo "Building binutils..."

        # Changing directory, so use a sub-shell (?)
        (
                # Create a the build directory.
                mkdir ${BUILD_DIR_BINUTILS} && cd ${BUILD_DIR_BINUTILS};
                # Configure, build and install binutils
                ../${BINUTILS_VERSION}/configure ${BINUTILS_CONFIGURE_ARGS} &&
                ${GNU_MAKE} -j 12 -w all && ${GNU_MAKE} -w install
        )

        # Remove build dir
        rm -rf $BUILD_DIR_BINUTILS

        echo "Binutils Build done."
}

#
# "Builds" cygwin. Actually, it only copies some headers around.
#
do_cygwin_build() {
        HEADERS=${PREFIX}/${TARGET_ARCH}/sys-include

        mkdir -p $HEADERS  &&
        cp -rf ${CYGWIN_DIR}/newlib/libc/include/* $HEADERS &&
        cp -rf ${CYGWIN_DIR}/winsup/cygwin/include/* $HEADERS
}

#
# Builds GCC
#
do_gcc_build() {
        BUILD_DIR_GCC=${BUILD_DIR_PREFIX}gcc-${TARGET_ARCH}

        # Remove dir if it exists.
        if [ -d $BUILD_DIR_GCC ] ; then
                rm -rf $BUILD_DIR_GCC
        fi

        echo "Building GCC..."

        # Changing directory, so use a sub-shell (?)
        (
                # Create a the build directory.
                mkdir ${BUILD_DIR_GCC} && cd ${BUILD_DIR_GCC};
                # Configure, build and install GCC.
                ../${GCC_VERSION}/configure $GCC_CONFIGURE_ARGS &&
                ${GNU_MAKE} -j 12 -w all && ${GNU_MAKE} -w install
        )
        rm -rf $BUILD_DIR_BINUTILS

        echo "GCC Build done."
}

do_fetch
do_extract
do_binutils_build
do_cygwin_build
do_gcc_build

Unfortunately, the gcc binary built by the script, located in /opt/i386-tiano-pe/bin, can’t produce binaries. Invoking the compiler on a source file (“Hello, World!” program) dies with:

$ /opt/i386-tiano-pe/bin/gcc main.c -o main
/opt/lib/gcc/i386-tiano-pe/4.2.3/../../../../i386-tiano-pe/bin/ld: crt0.o: No such file: No such file or directory
collect2: ld returned 1 exit status

I assume this is because I skipped the last two steps in the list at the beginning of this post. However, using the compiler to generate an assembler file (parameter -S) and then running the assembler on that file to produce an object file does indeed produce a PE/COFF object file.

$ cd /opt/i386-tiano-pe/bin
$ ./gcc -S ~/main.c
$ ./as -o main.o main.s
$ file ./main.o
./main.o: MS Windows COFF Intel 80386 object file

TianoCore on FreeBSD/amd64

I’m attempting to build the TianoCore code base on FreeBSD/amd64. Here’s what I did so far.

  • In order to be able to check out the EDK2 sources, I installed the devel/subversion port. To check out the source tree, I did this in my home directory:

    $ svn co https://edk2.tianocore.org/svn/edk2/trunk/edk2 edk2
    
  • I installed the java/diablo-jdk15 port.
  • Downloaded Apache Ant 1.6.5. I didn’t install it through the port but instead downloaded the binary distribution and extracted the archive under /opt since the EDK2 build framework requires very specific versions of the tools.
  • Did the same thing with Ant-Contrib 1.0b3, XMLBeans 2.1.0 and Saxon 8.1.1.
  • I created a symbolic link at /opt/xmlbeans-2.1.0/lib to /opt/saxon-8.1.1/saxon8.jar. The build notes for the EDK said a copy was needed, but a symbolic link works just as good. I guess they were running Windows or didn’t know about links. Whatever.
  • Then I set up the environment for the build process as described in the build notes. This is what I did (note that my shell is bash):

    $ export WORKSPACE=/home/phs/edk2
    $ export JAVA_HOME=/usr/local/diablo-jdk1.5.0
    $ export ANT_HOME=/opt/apache-ant-1.6.5
    $ export XMLBEANS_HOME=/opt/xmlbeans-2.1.0
    $ export PATH=$PATH:$ANT_HOME/bin:$XMLBEANS_HOME/bin
    
  • I kicked off the build process with this command:

    $ bash edksetup.sh newbuild
    

    Unfortunately, the build fails with an error:

    BUILD FAILED
    /usr/home/phs/edk2/Tools/build.xml:22: The following error occurred while executing this line:
    /usr/home/phs/edk2/Tools/CCode/Source/build.xml:247: The following error occurred while executing this line:
    /usr/home/phs/edk2/Tools/CCode/Source/PeCoffLoader/build.xml:68: ar failed with return code 139
    /phs/edk2/Tools/CCode/Source/PeCoffLoader/build.xml:68: ar failed with return code 139
    

    This error can be solved by using a GCC that produces PE/COFF binaries instead of the default ELF images.