Personal Homepage
It has been over two years since the last time I worked on TianoCore for coreboot. These days I had some free time to spend and I used it to continue the project. I updated my code to the latest versions of coreboot and TianoCore and got it to work again in QEMU.
Here’s a "screenshot" and a few words on what it’s all about.
I’ve re-written the GenFw tool part of the TianoCore BaseTools project. The source code can be found here. In order to use the tool, the file Source/C/GenFw/GenFw.c must be replaced with the re-written one. Then, the base tools must be re-built. After that, the EDK2 build process can be started. It will automatically pick up the new tool which will brand an ELF file with an UEFI file type.
Currently, the re-written tool will not compile on Linux. The reason is that Linux lacks implementations of err(3), errx(3), warn(3), etc. library functions which the BSDs have. It should be easy to add some compatibility macros using a combination of fprintf(3), strerror(3) and exit(3). I might add those should the need arise.
Update (Dec 3rd, 2008): I’ve added the compatibility macros for Linux. An updated version of the source code can be downloaded here.
It’s been a while since I last worked on combining TianoCore and coreboot. Tonight I had some spare time and tried to pursue the project.
The previously mentioned build failure does indeed stem from the fact that the build tools cannot cope with ELF binaries. Especially problematic is the GenFw tool which is supposed to convert the binary file into an UEFI firmware volume file. In order to do that, it parses the header information of the input binary executable file and encodes the type of file (in UEFI terms) in a spare header field. The tool expects to work on PE32 files but the TianoCore developers have added code which converts an ELF image into a PE32 image internally if the tool is pointed at an ELF file. However, this facility is only compiled in if #defined(Linux) is true. Of course, that won’t work on FreeBSD but changing the relevant pre-processor condition allowed me to produce an UEFI firmware volume without any further changes to the code.
However, this shortcut will only work on x86 and only if the target platform is x86, too. The real solution is to avoid the conversion and instead encode the UEFI file type directly into the ELF header. I’ve already done this for my thesis project (*) and back then it seemed that re-writing the GenFw tool was easier than fixing the existing implementation. Well, here’s the next item on the ToDo list…
(*) I used the Java-based tools for the thesis project which means that a different tool with essentially the same functionality was the culprit.
In order to create a UEFI payload for coreboot, I’ve started a coreboot platform as part of the TianoCore EDK II. The sources for the platform can be obtained here. Note that the CorebootPkg directory must be placed in the TianoCore $WORKSPACE directory.
To build the package on FreeBSD, a GNU toolchain from vendor sources must be used. This is because the TianoCore tools use some compiler/linker flags unknown to the toolchain included in the FreeBSD base system. The path as well as the names of the toolchain binaries must be adjusted in Conf/tools_def.txt. Because I built the toolchain according to these instructions, the preprocessor will not look in /usr/include for headers which causes errors in the ProcessorBind.h header when it attempts to include stdint.h. This patch can be applied to fix this.
Note that the build process still cannot complete as the tools producing the final Firmware Volume (FV) cannot cope with the ELF binaries produced by the GNU toolchain.
This is part III of my attempts to build the TianoCore EDK II with the Python-based tools. In order to circumvent the error that stopped me in part II, the build process needs to be taught to use GNU make, i.e. gmake, on FreeBSD instead of make, which is BSD make. This can be done by editing the *_ELFGCC_*_MAKE_PATH variable in Conf/tools_def.txt.
The tools_def.txt file is automatically copied from a template part of the BaseTools sources. This patch fixes the template so the changes described above do not have to be applied manually.
At this point, the build process starts and does actually build some modules. However, the UnixPkg cannot be built completely on FreeBSD. This is because the code makes some assumptions only true on Linux, e.g. the presence of the sys/vfs.h header.
So here’s the "sequel" to Part One. This time I’m trying to actually build a Firmware Volume with the Python-based tools.
The core of the tools is build.py, a Python script which invokes the tools in order to build a Firmware Volume (FV). On FreeBSD, build.py cannot be run until the following requirements are met:
Because I could not find a port for EasyInstall, I did this to install the script on FreeBSD:
$ fetch http://peak.telecommunity.com/dist/ez_setup.py $ chmod +x ez_setup.py $ ./ez_setup.py
Note that this isn’t the whole truth as the path to the interpreter in the script, i.e. the first line aka "shebang", must be adjusted to /usr/local/bin/python before the script can be executed.
After that, the easy_install command is available and the ANTLR module can be installed by running this:
$ eazy_install http://www.antlr.org/download/Python/antlr_python_runtime-3.0.1-py2.5.egg
In theory, running build.py and thus building a Firmware Volume should be as easy as this:
$ cd path/to/edk2 $ export PYTHONPATH=/path/to/basetools/Source/Python $ . edksetup.sh BaseTools $ python $PYTHONPATH/build/build.py
Unfortunately, the last step initially aborted with this error:
build... : error 5000: Please execute /home/phs/sandbox/basetools/Bin/FreeBSD-i386:/sbin: /bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin: /home/phs/bin/edksetup.bat to set /home/phs/sandbox/basetools/Bin/Freebsd7 in environment variable: PATH! - Failed -
After some try’n’error time, I think that the above error is caused by user error: I had previously copied the compiled C programs from Source/C/bin to bin/FreeBSD-i386 (paths relative to /path/to/basetools). After removing bin/FreeBSD-i386, I created a link to BinWrappers/PosixLike at the same location:
$ cd /path/to/basetools $ ln -s BinWrappers/PosixLike Bin/FreeBSD-i386
I then re-ran build.py (see above) and it produced some output that didn’t look like errors:
00:44:09, Sep.21 2008 [FreeBSD-7.1-PRERELEASE-i386-32bit-ELF] WORKSPACE = /usr/home/phs/sandbox/edk2 EDK_SOURCE = /usr/home/phs/sandbox/edk2/EdkCompatibilityPkg EFI_SOURCE = /usr/home/phs/sandbox/edk2/EdkCompatibilityPkg EDK_TOOLS_PATH = /home/phs/sandbox/basetools TARGET_ARCH = IA32 TARGET = DEBUG TOOL_CHAIN_TAG = ELFGCC Active Platform = UnixPkg/UnixPkg.dsc Flash Image Definition = UnixPkg/UnixPkg.fdf Processing meta-data . . . . . . .
Unfortunately, though, right after the dots, an error occured:
build...
UnixPkg/UnixPkg.dsc(...): error 4000: Instance of library class [NetLib] is not found
in [MdeModulePkg/Universal/Network/ArpDxe/ArpDxe.inf] [IA32]
consumed by module [MdeModulePkg/Universal/Network/ArpDxe/ArpDxe.inf]
- Failed -
00:44:17, Sep.21 2008 [00:08]
The UnixPkg part of the EDK II seems to be broken as the error above indicates a dependency error between modules which is caused by an incorrect platform definition file (*.dsc). Applying this patch fixes the problem.
The patch ensures that all dependencies are met, but the build process still fails with this error:
Processing meta-data . . . . . . . . done!
make: don't know how to make pbuild. Stop
build...
: error 7000: Failed to execute command
make pbuild [/usr/home/phs/sandbox/edk2/Build/Unix/DEBUG_ELFGCC/IA32/MdePkg/Library/BaseTimerLibNullTemplate/BaseTimerLibNullTemplate]
Waiting for all build threads exit...
make: don't know how to make pbuild. Stop
build...
: error 7000: Failed to execute command
make pbuild [/usr/home/phs/sandbox/edk2/Build/Unix/DEBUG_ELFGCC/IA32/MdePkg/Library/BaseLib/BaseLib]
build...
: error F002: Failed to build module
MdePkg/Library/BaseTimerLibNullTemplate/BaseTimerLibNullTemplate.inf [IA32, ELFGCC, DEBUG]
- Failed -
01:01:43, Sep.21 2008 [00:09]
Oh, well, to be continued…
Now that I can use coreboot and libpayload on FreeBSD, it’s time to try the new Python-based build process for the TianoCore EDK II on FreeBSD.
Prerequisites are:
Note that Subversion access requires a user account at the TianoCore project.
This is trivially easy, just do:
$ cd /usr/ports/misc/e2fs-libuuid $ sudo make install
That’s all. The headers and libraries are installed under /usr/local.
Compiling the Base Tools, i.e. the Python-based TianoCore build tools, isn’t complicated but doesn’t work out of the box, either. First, these two patches (patch 1, patch 2) must be applied:
$ cd /path/to/basetools $ patch -p0 < basetools_include.diff $ patch -p0 < basetools_make.diff
The first patch adjusts some include paths so that /usr/local/include is searched, too, which is required in order to find the uuid/uuid.h header. The second patch replaces invocations of make to use the $(MAKE) variable which holds the name of invoked the make binary. This is required as in FreeBSD (and other BSDs), make is not GNU make, however the latter is required to build the Base Tools. Consequently, when building the project, make sure that gmake is used:
$ gmake
So, I finally found some time to crawl through the TianoCore project’s Contributor’s Agreement. Here’s what I think it means.
Copyright (c) 2006-2008, Intel Corp. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of Intel Corp. nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
In addition to their own code which they release under the "BSD license", there is some code in the TianoCore tree that is released under other licenses. Specifically the FAT32 code, which is apparently covered by some patents. If other licenses apply, and that’s the key point here, the license is either in the source files themselves or packaged with the source files.
The next bullet point, c), is somewhat harder to understand. I interpret is as: If I hold a patent on something, and my contribution infringes that patent, I automatically grant a patent license. I grant it to everybody, who wants to exercise his rights I granted him with the copyrigth license mentioned above. However, here’s the catch: That patent license applies only to my unmodified contribution.
I’m not sure what to think about that. I think, Intel is trying to protect their own patents. So if they release some code to the TianoCore project which is covered by a patent they own, the only grant a minimum patent license. What remains unclear is whether the patent license is still valid, even if I modify their code as permitted by the copyright license they granted.
The last bullet point, d), is an easy one again. It’s simply the "provided ‘as is’" part in the copyright license cited above.
So what’s the conclusion here? I think Intel is pretty open about releasing their code. However, they are not so open about creating an open source project around their code. What I mean is that there are quite some legal hurdles one has to pass when contributing code to the TianoCore project. In effect, they force the BSD license on any code I contribute and I think that’s OK. On the other hand, however, they prevent me from forking the project by introducing that stupid patent clause since I have no easy way of checking whether a specific piece of code infringes one of their patents.
I really wonder if they only want to protect themselves from getting sued over some code contributed the project from non-Intel employees. Or are they really trying to create an impression of an Open Source project when it’s really not?
So it took me exactly one day to start the TianoCore DXE Core on a new platform. Of course, this doesn’t count the roughly 10 weeks it took me to understand how the TianoCore Codebase works 😉 Also, it took me a fair amount of work to fix one thing or the other.
Anyways, I wanted to take a note that the generic memory test module included in the TianoCore Codebase is nowhere near "quick", despite the fact that it has a mode called QuickMode, when you attempt to throw it at a 8 GByte memory range.
Just a quick note to remind myself that when porting the TianoCore stack to a new platform, the PEI needs an implementation of the PEI_LOAD_FILE_PPI before it can load any PEIMs from the FV.
For the platforms I have worked with so far, the PPI was implemented in the SEC.