Cell SDK 3.0 on IBM’s OpenClient Distribution

Today I was finally able to build a simple HelloWorld-Application with and within the TianoCore build environment. This is good news, as it leads me to the next task: Building the same HelloWorld-Application, but this time for Linux on 64-Bit PowerPC.

Since I do not yet have access to a 64-Bit PowerPC machine running Linux, I’m going to use the Cell SDK 3.0 for now. It can be used from an i386 machine and includes a toolchain as well as the full system simulator. The toolchain includes a cross-compiler that is cabable of producing binaries for the Cell BE’s PPU, which is essentially a 64-Bit PowerPC processor. The system simulator simulates a Linux-installation running on Cell.

I’m still on IBM’s OpenClient Linux distribution, which is apparently based on RHEL 5.1, at least according to /etc/redhat-release

Red Hat Enterprise Linux Client release 5.1 (Tikanga)

This is good on one hand, but made things slightly more complicated on the other hand. But first things first. Here’s what I did to prepare the Cell SDK installation:

  • I went to the developerWorks download page for the Cell SDK 3.0 and downloaded the RHEL 5.1 packages.
  • I had to download the "basic libraries and headers for cross-compiling to Cell Broadband Engine’s PPU", both the 32-Bit version and the 64-Bit version, from the Barcelona Supercomputer Center (BSC). Note that I could have built those RPMs myself, but only if I had a few other required RPMs like e.g. a glibc for PowerPC. Apparently those required RPMs are provided on the RHEL installation CDs, however, I’m on IBM’s OpenClient and thus do not have access to the installation CDs. The good thing is, the Fedora RPMs provided by the BSC turned out to work just fine.
  • For the full system simulator, I had to download the sysroot image from the BSC website.

So that’s it for the preparation part, now to actually installing the SDK.

  • I installed the installer RPM like this:
    # rpm -ivh cell-install-3.0.0-1.0.noarch.rpm

    This installs the installer to /opt/cell.

  • Now I needed to install the cross-compilation libraries and headers:
    # rpm -ivh ppu-sysroot-f7-2.noarch.rpm
    # rpm -ivh ppu-sysroot64-f7-2.noarch.rpm
  • Next I ran the installer as instructed by the installation manual:
    # cd /opt/cell
    # ./cellskd –iso /home/phs/Downloads install

After successfully running the installer, I found a functioning cross-compiler in /opt/cell/toolchain/bin.

For the system simulator, I had to install the sysroot imae RPM like this:

# rpm -ivh sysroot_image-3.0-7.noarch.rpm

Unfortunately, I wasn’t able to make the system simulator work because of a missing dependency on a simulation library.

By the way, there’s also official documentation available here.

Setting up a Subversion Server

Ok, there’s probably a million Blog posts, tutorials and HowTos on how to do this already. Yet I still find it hard to find short instructions on how to set up a subversion server quickly. I just had to do it again, and it took me longer than expected, so here it goes.

My requirements are pretty basic:

  • WebDAV so I can access the repository over HTTP.
  • I don’t need too much security, so SSL won’t be needed.
  • I want SVN:Web, a web front end to the repositories.

Here’s what I did for the Subversion+WebDAV part. Note that I’m currently on IBM’s OpenClient Linux Distribution which is based on RedHat.

  • I made sure that the httpd package is installed. I didn’t have to install it, so I guess it’s installed by default.
  • I had to install the mod_dav_svn package with yum:
    $ sudo yum install mod_dav_svn
  • The package installs a config file at /etc/httpd/conf.d/subversion.conf. By default, the <Location> tag is commented out. I just copied it, removed the comment signs and adjusted the values to my needs. This is what it now looks like:
    LoadModule dav_svn_module     modules/mod_dav_svn.so
    LoadModule authz_svn_module   modules/mod_authz_svn.so
    
    <Location /repos>
       DAV svn
       SVNParentPath /var/www/svn
    
       <LimitExcept GET PROPFIND OPTIONS REPORT>
          AuthType Basic
          AuthName "Authorization Realm"
          AuthUserFile /var/www/passwd
          Require valid-user
       </LimitExcept>
    </Location>
    
  • Now the Apache HTTP Server will serve the contents of /var/www/svn via WebDAV (I think), but it will query for a valid user entry. The entry must be created as follows:
    # cd /var/www
    # htpasswd -c passwd <username>
    
  • Next, I created the direcories for the repositories. As root, I ran these commands:
    # cd /var/www
    # mkdir svn
    # cd svn
    # svnadmin create <name of repository>
    # chown -R apache:apache <name of repository>
    
  • I then activated the HTTP service so the Apache Web Server would be started at boot time. I did this using a GUI tool called system-config-services. I had to check the box next to the httpd entry. I didn’t want to reboot right aways, so I clicked Start in the toolbar. The tool told me that the service would now be running and I could verify this by going to http://localhost/. Testing the Subversion part was easy, too. Navigating to http://localhost/repos/example did the trick. Note that the actual repository name must be used instead of "example".
  • Oh, and I want at least minimal security. That is, I want the HTTP Server to serve pages only to the local machine. Therefore, I changed the Listen directive in the server config uration file to this:
    Listen 127.0.0.1:80
    

The second part was installing the SVN::Web CGI Script. Here’s what I did to do it.

  • First, I had to install the subversion-perl package with yum. As root, I ran
    # yum install subversion-perl
  • Second, I installed the actuall SVN::Web script through CPAN. Again as root, I did
    # cpan install SVN::Web
  • I then created a directory that would hold all the SVN::Web files:
    #cd /var/www
    #mkdir svnweb
    
  • In that directory, I let the Perl script set itself up using default values:
    # cd /var/www/svnweb
    # svnweb-install
    
  • The last step created a file called config.yaml. It must be edited so the CGI script finds the repositories. Near the end, I edited the reposparent value:
    reposparent: '/var/www/svn'
  • Now, as the final step, the script needs to be introduced to the Apache Server. I created a file svnweb.conf in /etc/httpd/conf.d with the following contents:
    Alias /svnweb /var/www/svnweb
    
    AddHandler cgi-script .cgi
    
    <Directory /var/www/svnweb>
            Options All ExecCGI
            DirectoryIndex index.cgi
    </Directory>
    

After restarting the Apache HTTP Server, I could access http://localhost/svnweb and see the repositories.

TianoCore on IBM’s OpenClient

I’ve started work at IBM yesterday. Today I found a room and got all my user accounts and passwords set up so I can actually start to work.

I’m trying to build the TianoCore EDKII (SVN Revision 4792) on IBM’s own Linux distribution, called OpenClient. The distribution sucks but it’s the only kind of Linux we’re allowed to use around here. It’s based on Red Hat (I think), but it feels "different".

To bootstrap the TianoCore Build Environment, I sort of followed my FreeBSD and Fedora Core notes. Here’s what I did:

  • Installed Java JDK 6 Update 4 (JDK 1.6.0_04) for Linux through the self-extracting binary file (not the RPM) that Sun provides. I placed the JDK in /opt.
  • Installed the binary distribution of Apache Ant 1.7.0, Saxon 8.1.1, XMLBeans 2.1.0 as well as ant-contrib 1.0b3 and placed all of them in /opt.
  • Created the symlinks:
    $ cd /opt/apache-ant-1.7.0/lib
    $ sudo ln -s /opt/ant-contrib/ant-contrib-1.0b3.jar ant-contrib.jar
    $ sudo ln -sf /opt/saxonb8.1.1/saxon8.jar /opt/xmlbeans-2.1.0/lib/saxon8.jar
    

I then created a small script that automates the build tool bootstrapping process:

export JAVA_HOME=/opt/jdk1.6.0_04
export XMLBEANS_HOME=/opt/xmlbeans-2.1.0
export ANT_HOME=/opt/apache-ant-1.7.0
export WORKSPACE=/home/phs/Sources/edk2
export PATH=$PATH:$XMLBEANS_HOME/bin:$ANT_HOME/bin
. edksetup.sh ForceRebuild

Sourcing the script successfully builds the build tools. Then, in the file $WORKSPACE/Tools/Conf/target.txt, two settings need to be adjusted:

ACTIVE_PLATFORM=EdkUnixPkg/Unix.fpd
TOOL_CHAIN_TAG=ELFGCC

Of course, the previously mentioned patch needs to be applied. After that, the EDKII Unix Emulation Package can be built and run as described in the tutorial:

$ cd $WORKSPACE/
$ build
$ cd Build/Unix
$ . run.cmd

I found that the IBM OpenClient distribution already includes the e2fsprogs-devel package as well as the relevant X11 development packages. Please also note that it is not neccessary to build an PE32+ cross compiler on Linux.

TianoCore on Fredora Core 8

My attempts building anything of the TianoCore EDK2 codebase on FreeBSD/amd64 have been extremely frustrating so far. I guess trying to build on an unsupported operating system and on an unsupported architecture may have been too much of a hurdle for the beginning. So I figured I’d try building the EDK Unix Simulator (Trunk Revision 4679) on Fredora Core 8. Here’s what I’ve done.

I pretty much followed this tutorial. It’s for Gentoo, so a little tweaking was needed.

First, I downloaded and installed the JDK 6 from SUN via the RPMs they provide. The JDK ends up in /usr/java/jdk1.6.0_04, so the JAVA_HOME environment variable needs to be set to that.

Second, I didn’t bother installing the needed (Java) tools through the Fedora Package Manager but instead downloaded the files manually and placed them under /opt. This is similar to what I did for my FreeBSD attempts. I needed two symlinks like the tutorial says:

$ cd /opt/apache-ant-1.7.0/lib
$ sudo ln -s /opt/ant-contrib/ant-contrib-1.0b3.jar ant-contrib.jar
$ sudo ln -sf /opt/saxonb8.1.1/saxon8.jar /opt/xmlbeans-2.1.0/lib/saxon8.jar

Third, I needed to install the e2fsprogs-devel package. The e2fsprogs package (without the “-devel” suffix) isn’t enough. Also, I had to install the X development packages. I don’t know what exact package was needed, but the Fedora Core 8 package manager has this option that lets you install some pre-selected packages related to X development.

Forth, I had to apply the following patch (the tutorial mentions this):

Index: EdkModulePkg/Bus/Pci/PciBus/Dxe/PciHotPlugSupport.c
===================================================================
--- EdkModulePkg/Bus/Pci/PciBus/Dxe/PciHotPlugSupport.c (Revision 4679)
+++ EdkModulePkg/Bus/Pci/PciBus/Dxe/PciHotPlugSupport.c (Arbeitskopie)
@@ -21,7 +21,7 @@

 --*/

-#include "Pcibus.h"
+#include "pcibus.h"
 #include "PciHotPlugSupport.h"

 EFI_PCI_HOT_PLUG_INIT_PROTOCOL  *gPciHotPlugInit;

Finally, contrary to what the tutorial says, I used the following script to set up the environment. Note that I didn’t include the TOOL_CHAIN line and that I didn’t build the PE/COFF capable GCC.

export JAVA_HOME=/usr/java/jdk1.6.0_04
export XMLBEANS_HOME=/opt/xmlbeans-2.1.0 
export ANT_HOME=/opt/apache-ant-1.7.0
export WORKSPACE=/home/phs/edk2/edk2
export PATH="$PATH:$XMLBEANS_HOME/bin:$ANT_HOME/bin" 

The build went smoothly after that and I was able to use the EDK Unix environment. Note that the thing should not be called “Unix” Package, since it heavily assumes that it runs on Linux in some areas.

TianoCore on FreeBSD/amd64, Take 2

Now that I have build a cross compiler, I’m trying to build the TianoCore EDK again.

In this post I list a few environment variables that need to be set in order to build the EDK. Because the build process needs to use the cross compiler, another environment variable is needed:

$ export CC=/opt/i386-tiano-pe/bin/gcc

Since I’m lazy and all, I put everything in a little script called env.sh:

export WORKSPACE=/home/phs/edk2
export JAVA_HOME=/usr/local/diablo-jdk1.5.0
export ANT_HOME=/opt/apache-ant-1.6.5
export XMLBEANS_HOME=/opt/xmlbeans-2.1.0
export PATH=$PATH:$ANT_HOME/bin:$XMLBEANS_HOME/bin
export CC=/opt/i386-tiano-pe/bin/gcc

Also, the EDK build notes mention that the default build target is the Windows NT Emulation environment. However, that target cannot be built using GCC, so I needed to edit the file Tools/Conf/target.txt and change the ACTIVE_PLATFORM to:

ACTIVE_PLATFORM       = EdkUnixPkg/Unix.fpd

Now, I can tip of the build process as follows:

$ cd ~/edk2
$ . env.sh
$ . edksetup.sh newbuild

Note that the build script must be “sourced”, not executed. Unfortunately, the build process still fails, but now with a different error:

       [cc] 1 total files to be compiled.
       [cc] In file included from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.h:3,
       [cc]                  from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:17:
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:27:20: error: jni_md.h: No such file or directory
       [cc] In file included from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.h:3,
       [cc]                  from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:17:
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:45: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jsize'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:104: error: expected specifier-qualifier-list before 'jbyte'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:193: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1834: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1842: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1851: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1888: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1927: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1930: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1933: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1937: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1940: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'void'
       [cc] In file included from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:17:
       [cc] /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jbyteArray'
       [cc] /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:29: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jbyteArray'

BUILD FAILED
/home/phs/edk2/Tools/build.xml:22: The following error occurred while executing this line:
/home/phs/edk2/Tools/CCode/Source/build.xml:254: The following error occurred while executing this line:
/home/phs/edk2/Tools/CCode/Source/CompressDll/build.xml:45: gcc failed with return code 1

The root cause of this is that the compiler can’t find the jni_md.h header which is located in $JAVA_HOME/include/freebsd (at least on my system). I worked around this problem by editing $JAVA_HOME/include/jni.h as follows:

--- jni.h.orig  2008-02-06 11:50:05.000000000 +0100
+++ jni.h       2008-02-06 11:50:16.000000000 +0100
@@ -24,7 +24,7 @@
 /* jni_md.h contains the machine-dependent typedefs for jbyte, jint
    and jlong */
 
-#include "jni_md.h"
+#include "freebsd/jni_md.h"
 
 #ifdef __cplusplus
 extern "C" {

Now I’m stuck with yet another compilation error:

init:
     [echo] Building the EDK Tool Library: CompressDll

Lib:
       [cc] 1 total files to be compiled.
       [cc] /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c: In function 'Java_org_tianocore_framework_tasks_Compress_CallCompress':
       [cc] /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:57: warning: overflow in implicit constant conversion
       [cc] Starting link
       [cc] /usr/bin/ld: /home/phs/edk2/Tools/CCode/Source/Library/libCommonTools.a(EfiCompress.o): relocation R_X86_64_32S can not be used when making a shared object; recompile with -fPIC
       [cc] /home/phs/edk2/Tools/CCode/Source/Library/libCommonTools.a: could not read symbols: Bad value

BUILD FAILED
/home/phs/edk2/Tools/build.xml:22: The following error occurred while executing this line:
/home/phs/edk2/Tools/CCode/Source/build.xml:254: The following error occurred while executing this line:
/home/phs/edk2/Tools/CCode/Source/CompressDll/build.xml:45: gcc failed with return code 1

Well, I guess we’ll see where this ends…

Building a Cross Compiler on FreeBSD

I’m currently trying to build a cross compiler (and other required tools) on FreeBSD. The compiler will run on FreeBSD/amd64 and should produce i386 binaries. This wouldn’t be too hard since that task can easily be accomplished by using the FreeBSD source tree. However, I need the toolchain to produce binaries in the PE/COFF format instead of the default ELF format.

Building the toolchain is somewhat tricky, at least I found it to be poorly documented. But maybe I just looked in the wrong places. Building the toolchain requires:

  • Building binutils.
  • Locating some header files.
  • Building the compiler.
  • Building a C Library.
  • Building the compiler again so it uses the C library that was just built.

For my needs, I found that the last two steps weren’t needed. I wrote a script that downloads the sources, extracts the archives and builds the toolchain. Here’s the script in full quote (I really wish there was a way to upload files to this thing):

#/bin/sh

#
# Copyright (c) 2008 Philip Schulz 
#

#
# This script downloads, builds and installs GCC and GNU Binutils that can
# produce x86 binaries in PE/COFF Format. The cross toolchain needs some headers
# that aren't usually present on the host system. However, those headers can be
# obtained from the cygwin sources, that's why a snapshot of the cygwin sources
# is downloaded.
#
# After the script finishes, the tools will be located at
# ${PREFIX}/${TARGET_ARCH}/bin. Some other binaries will be installed in
# ${PREFIX}/bin with their own prefix of ${TARGET_ARCH} but I don't know that
# they are for.
#

# Prefix where the Cross-Tools will live
PREFIX="${PREFIX:-/opt}"

# Target architecture.
TARGET_CPU="${TARGET_CPU:-i386}"
TARGET_ARCH=${TARGET_CPU}-tiano-pe

# Program that can fetch the files.
FETCH_COMMAND="/usr/bin/ftp -V"

# GNU Make
GNU_MAKE=`which gmake`

################################################################################
#
# GCC settings.
#
################################################################################
# What version of GCC will be fetched, built and installed 
GCC_VERSION=gcc-4.2.3
# What mirror to use.
GCC_MIRROR=ftp://ftp-stud.fht-esslingen.de/pub/Mirrors/ftp.gnu.org
# File name of the GCC sources. Should probably not be changed.
GCC_ARCHIVE=$GCC_VERSION.tar.bz2
# Where the GCC Sources can be fetched from. Should probably not be changed.
GCC_URL=$GCC_MIRROR/gcc/$GCC_VERSION/$GCC_ARCHIVE
# Arguments for the GCC configure script. Should probably not be changed.
GCC_CONFIGURE_ARGS="--prefix=${PREFIX} --target=${TARGET_ARCH} "
GCC_CONFIGURE_ARGS+="--with-gnu-as --with-gnu-ld --with-newlib "
GCC_CONFIGURE_ARGS+="--disable-libssp --disable-nls --enable-languages=c "
GCC_CONFIGURE_ARGS+="--program-prefix=${TARGET_ARCH}- "
GCC_CONFIGURE_ARGS+="--program-suffix=-4.2.3 "


################################################################################
#
# Binutils settings.
#
################################################################################
# What version of the GNU binutils will be fetched, build and installed
BINUTILS_VERSION=binutils-2.18
# What mirror to use.
BINUTILS_MIRROR=ftp://ftp-stud.fht-esslingen.de/pub/Mirrors/ftp.gnu.org
# File name of the binutils sources. Should probably not be changed.
BINUTILS_ARCHIVE=$BINUTILS_VERSION.tar.gz
# Where the GCC Sources can be fetched from. Should probably not be changed.
BINUTILS_URL=$BINUTILS_MIRROR/binutils/$BINUTILS_ARCHIVE
# Arguments for the GCC configure script. Should probably not be changed.
BINUTILS_CONFIGURE_ARGS="--prefix=${PREFIX} --target=${TARGET_ARCH} "
BINUTILS_CONFIGURE_ARGS+="--disable-nls "

################################################################################
#
# Cygwin settings.
#
################################################################################
CYGWIN_SNAPSHOT=20080129
CYGWIN_ARCHIVE=cygwin-src-${CYGWIN_SNAPSHOT}.tar.bz2
CYGWIN_MIRROR=http://cygwin.com/
CYGWIN_URL=${CYGWIN_MIRROR}snapshots/${CYGWIN_ARCHIVE}
CYGWIN_DIR=cygwin-snapshot-${CYGWIN_SNAPSHOT}-1

################################################################################
#
# Batch code.
#
################################################################################
#
# Fetches the files.
#
do_fetch() {
        if [ ! ( -f $GCC_ARCHIVE ) ] ; then
                echo "Fetching ${GCC_URL}"
                ${FETCH_COMMAND} ${GCC_URL}
        else
                echo $GCC_ARCHIVE already locally present.
        fi

        if [ ! ( -f $CYGWIN_ARCHIVE ) ] ; then
                echo "Fetching ${CYGWIN_URL}"
                ${FETCH_COMMAND} ${CYGWIN_URL}
        else
                echo $CYGWIN_ARCHIVE already locally present.
        fi

        if [ ! ( -f $BINUTILS_ARCHIVE ) ] ; then
                echo "Fetching ${BINUTILS_URL}"
                ${FETCH_COMMAND} ${BINUTILS_URL}
        else
                echo $BINUTILS_ARCHIVE already locally present.
        fi
}

#
# Extracts the archives.
#
do_extract() {
        # Remove already extracted files first.
        rm -rf ${GCC_VERSION}
        rm -rf ${CYGWIN_DIR}
        rm -rf ${BINUTILS_VERSION}

        # Extract the archives
        if [ -f $GCC_ARCHIVE ] ; then
                echo "Extracting ${GCC_ARCHIVE}"
                tar -jxf ${GCC_ARCHIVE}
        fi

        if [ -f $CYGWIN_ARCHIVE ] ; then
                echo "Extracting ${CYGWIN_ARCHIVE}"
                tar -jxf ${CYGWIN_ARCHIVE}
        fi

        if [ -f $BINUTILS_ARCHIVE ] ; then
                echo "Extracting ${BINUTILS_ARCHIVE}"
                tar -xzf ${BINUTILS_ARCHIVE}
        fi
}


BUILD_DIR_PREFIX=build-

#
# Builds Binutils.
#
do_binutils_build() {
        BUILD_DIR_BINUTILS=${BUILD_DIR_PREFIX}binutils-${TARGET_ARCH}

        # Remove dir if it exists.
        if [ -d $BUILD_DIR_BINUTILS ] ; then
                rm -rf $BUILD_DIR_BINUTILS
        fi

        echo "Building binutils..."

        # Changing directory, so use a sub-shell (?)
        (
                # Create a the build directory.
                mkdir ${BUILD_DIR_BINUTILS} && cd ${BUILD_DIR_BINUTILS};
                # Configure, build and install binutils
                ../${BINUTILS_VERSION}/configure ${BINUTILS_CONFIGURE_ARGS} &&
                ${GNU_MAKE} -j 12 -w all && ${GNU_MAKE} -w install
        )

        # Remove build dir
        rm -rf $BUILD_DIR_BINUTILS

        echo "Binutils Build done."
}

#
# "Builds" cygwin. Actually, it only copies some headers around.
#
do_cygwin_build() {
        HEADERS=${PREFIX}/${TARGET_ARCH}/sys-include

        mkdir -p $HEADERS  &&
        cp -rf ${CYGWIN_DIR}/newlib/libc/include/* $HEADERS &&
        cp -rf ${CYGWIN_DIR}/winsup/cygwin/include/* $HEADERS
}

#
# Builds GCC
#
do_gcc_build() {
        BUILD_DIR_GCC=${BUILD_DIR_PREFIX}gcc-${TARGET_ARCH}

        # Remove dir if it exists.
        if [ -d $BUILD_DIR_GCC ] ; then
                rm -rf $BUILD_DIR_GCC
        fi

        echo "Building GCC..."

        # Changing directory, so use a sub-shell (?)
        (
                # Create a the build directory.
                mkdir ${BUILD_DIR_GCC} && cd ${BUILD_DIR_GCC};
                # Configure, build and install GCC.
                ../${GCC_VERSION}/configure $GCC_CONFIGURE_ARGS &&
                ${GNU_MAKE} -j 12 -w all && ${GNU_MAKE} -w install
        )
        rm -rf $BUILD_DIR_BINUTILS

        echo "GCC Build done."
}

do_fetch
do_extract
do_binutils_build
do_cygwin_build
do_gcc_build

Unfortunately, the gcc binary built by the script, located in /opt/i386-tiano-pe/bin, can’t produce binaries. Invoking the compiler on a source file (“Hello, World!” program) dies with:

$ /opt/i386-tiano-pe/bin/gcc main.c -o main
/opt/lib/gcc/i386-tiano-pe/4.2.3/../../../../i386-tiano-pe/bin/ld: crt0.o: No such file: No such file or directory
collect2: ld returned 1 exit status

I assume this is because I skipped the last two steps in the list at the beginning of this post. However, using the compiler to generate an assembler file (parameter -S) and then running the assembler on that file to produce an object file does indeed produce a PE/COFF object file.

$ cd /opt/i386-tiano-pe/bin
$ ./gcc -S ~/main.c
$ ./as -o main.o main.s
$ file ./main.o
./main.o: MS Windows COFF Intel 80386 object file

TianoCore on FreeBSD/amd64

I’m attempting to build the TianoCore code base on FreeBSD/amd64. Here’s what I did so far.

  • In order to be able to check out the EDK2 sources, I installed the devel/subversion port. To check out the source tree, I did this in my home directory:

    $ svn co https://edk2.tianocore.org/svn/edk2/trunk/edk2 edk2
    
  • I installed the java/diablo-jdk15 port.
  • Downloaded Apache Ant 1.6.5. I didn’t install it through the port but instead downloaded the binary distribution and extracted the archive under /opt since the EDK2 build framework requires very specific versions of the tools.
  • Did the same thing with Ant-Contrib 1.0b3, XMLBeans 2.1.0 and Saxon 8.1.1.
  • I created a symbolic link at /opt/xmlbeans-2.1.0/lib to /opt/saxon-8.1.1/saxon8.jar. The build notes for the EDK said a copy was needed, but a symbolic link works just as good. I guess they were running Windows or didn’t know about links. Whatever.
  • Then I set up the environment for the build process as described in the build notes. This is what I did (note that my shell is bash):

    $ export WORKSPACE=/home/phs/edk2
    $ export JAVA_HOME=/usr/local/diablo-jdk1.5.0
    $ export ANT_HOME=/opt/apache-ant-1.6.5
    $ export XMLBEANS_HOME=/opt/xmlbeans-2.1.0
    $ export PATH=$PATH:$ANT_HOME/bin:$XMLBEANS_HOME/bin
    
  • I kicked off the build process with this command:

    $ bash edksetup.sh newbuild
    

    Unfortunately, the build fails with an error:

    BUILD FAILED
    /usr/home/phs/edk2/Tools/build.xml:22: The following error occurred while executing this line:
    /usr/home/phs/edk2/Tools/CCode/Source/build.xml:247: The following error occurred while executing this line:
    /usr/home/phs/edk2/Tools/CCode/Source/PeCoffLoader/build.xml:68: ar failed with return code 139
    /phs/edk2/Tools/CCode/Source/PeCoffLoader/build.xml:68: ar failed with return code 139
    

    This error can be solved by using a GCC that produces PE/COFF binaries instead of the default ELF images.

Qemu and Coreboot on FreeBSD/amd64

Today I tried to boot FreeBSD inside Qemu, using coreboot. I already wrote a short guide on how to do this, but I only tested it on a FreeBSD/i386 host. Now I’m trying to do the same thing, except this time the host is a FreeBSD/amd64 system.

I encountered several problems and I’m going to describe them along with the workarounds here.

First, the devel/dev86 port I created was marked as running on i386 only. I initially did this because I wasn’t sure if the software would run on amd64, but after testing it I can say that it does work. So the port needs a little tweaking, namely this line

ONLY_FOR_ARCHS= i386

needs to be extended to

ONLY_FOR_ARCHS= i386 amd64

or maybe even removed completely.

Second, the RomCC utility used in the coreboot build process crashes, apparently due to a change in GCC 4 as opposed to GCC 3.4 used in older versions on FreeBSD. It crashes in a function called “free_basic_block” and the workaround is this:

Index: util/romcc/romcc.c
===================================================================
--- util/romcc/romcc.c  (revision 3088)
+++ util/romcc/romcc.c  (working copy)
@@ -15083,6 +15083,8 @@

static void free_basic_block(struct compile_state *state, struct block *block)
{
+       return;
+
      struct block_set *edge, *entry;
      struct block *child;
      if (!block) {

Third, the linker on FreeBSD/amd64 does not understand the -m32 flag. So I needed a cross compiler for i386. Warner Losh has an entry in his blog that describes how to cross build FreeBSD. I don’t need all of FreeBSD but only the compiler, so I use this:

$ cd /usr/src
$ export TARGET=i386
$ export TARGET_ARCH=i386
$ make toolchain

This gives me an i386 cross compiler in /usr/obj/i386/usr/src/tmp/usr/bin that I need to use to compile coreboot. I did this to tell the GNU make utility to use the cross compiler:


gmake CC=/usr/obj/i386/usr/src/tmp/usr/bin/gcc


There is one outstanding problem: Qemu crashes when I try to use coreboot as a BIOS replacement. It may have to do with the version of Qemu I’m using right now. I’ll try to use an older version later this afternoon to see if it solves the problem.

Using Qemu 0.9.0 with the required patches (see the Qemu Build Tutorial) made the problem go away and I’m now able to boot FreeBSD/i386 8-CURRENT inside Qemu, using coreboot and ADLO. I’ve uploaded a complete archive of the port files to the coreboot wiki, see this link.

Vendor Branch Management with Subversion

So, I sometimes hack on FreeBSD and want version control for my local changes. Working with CVS and vendor branches seemed to much of a pain and everyone was talking about how nowadays you’d use Subversion anyways. So I decided to give Subversion a try.

What helped me a lot in getting started with Subversion was the Book “Version Control with Subversion“, which is available online.

So I set up a subversion server and import snapshots of the FreeBSD source tree on a more or less regular basis. Here’s what I do to update the vendor branch.

  • First, I fetch the latest and greatest of the FreeBSD sources, logged into svn.dmz.local.deadc0.de as user svn. I use csup, the CVSup replacement in the base system. I don’t use CVS since it would leave the CVS directories all over the place. It would also be slower and I don’t need its functionality anyways. This is the supfile I use:
    *default host=cvsup.dmz.local.deadc0.de
    *default base=/var/db
    *default prefix=/usr
    *default release=cvs tag=.
    *default delete use-rel-suffix
    
    *default compress
    
    src-all
    

    The actual command I run is:

    # csup -L 2 -g ~/supfile
    
  • Then, I use the Script svn_load_dirs, which can be installed through the devel/svn_load_dirs port. I invoke it as follows:
    $ svn_load_dirs -t tag http://.../repos/freebsd/vendor current /usr/src
    

    This causes the repository location /repos/freebsd/vendor/current to be updated with the contents of /usr/src. After that, a tag is created at /repos/freebsd/vendor/tag, caused by the parameter -t.

    This whole process takes a while, and every once in a while, something needs to be acknowledged. So unfortunately, it can’t be let run over night. One thing to remember: The command is run as the currently logged in user “svn”, but the repository username is “phs”. So when the script asks for the password, I hit enter. Then a blank line appears where I enter the repository user name. After hitting enter, the scripts asks for the password again, this time for the right user.

  • The next step is to SSH into knecht.dmz.local.deadc0.de and go to /usr/devel/src. There, I run

    $ svn up

    After that, I merge the new vendor code with:


    $ svn merge http://…/repos/vendor/freebsd/ http://…/repos/vendor/current .

  • The last step may have produced some conflicts, so I use svn status to detect conflicts.
  • After all conflicts have been resolved, I commit the changes with svn ci and voilĂ , that’s it.