F. Compiling the compiler yourself

F.1 Introduction

The Free Pascal team releases at intervals a completely prepared package, with compiler and units all ready to use, the so-called releases. After a release, work on the compiler continues, bugs are fixed and features are added. The Free Pascal team doesn't make a new release whenever they change something in the compiler, instead the sources are available for anyone to use and compile. Compiled versions of RTL and compiler are also made daily, and put on the web.

There are, nevertheless, circumstances when you'll want to compile the compiler yourself. For instance if you made changes to compiler code, or when you download the compiler via CVS.

There are essentially 2 ways of recompiling the compiler: by hand, or using the makefiles. Each of these methods will be discussed.

F.2 Before you begin

To compile the compiler easily, it is best to keep the following directory structure (a base directory of /pp/src is supposed, but that may be different):

/pp/src/Makefile
       /makefile.fpc
       /rtl/linux
           /inc
           /i386
           /...
       /compiler

If you want to use the makefiles, you must use the above directory tree.

The compiler and rtl source are zipped in such a way that if you unzip both files in the same directory (/pp/src in the above) the above directory tree results.

The makefile.fpc and Makefile come from the base.zip file on the ftp site. If you compile manually, you don't need them.

There are 2 ways to start compiling the compiler and RTL. Both ways must be used, depending on the situation. Usually, the RTL must be compiled first, before compiling the compiler, after which the compiler is compiled using the current compiler. In some special cases the compiler must be compiled first, with a previously compiled RTL.

How to decide which should be compiled first? In general, the answer is that you should compile the RTL first. There are 2 exceptions to this rule:

1. The first case is when some of the internal routines in the RTL have changed, or if new internal routines appeared. Since the OLD compiler doesn't know about these changed internal routines, it will emit function calls that are based on the old compiled RTL, and hence are not correct. Either the result will not link, or the binary will give errors.
2. The second case is when something is added to the RTL that the compiler needs to know about (a new default assembler mechanism, for example).

How to know if one of these things has occurred ? There is no way to know, except by mailing the Free Pascal team. If you cannot recompile the compiler when you first compile the RTL, then try the other way.

F.3 Compiling using make

When compiling with make it is necessary to have the above directory structure. Compiling the compiler is achieved with the target cycle.

Under normal circumstances, recompiling the compiler is limited to the following instructions (assuming you start in directory /pp/src):

cd compiler
make cycle

This will work only if the makefile.fpc is installed correctly and if the needed tools are present in the PATH. Which tools must be installed can be found in appendix .

The above instructions will do the following:

1. Using the current compiler, the RTL is compiled in the correct directory, which is determined by the OS you are under. e.g. under LINUX, the RTL is compiled in directory rtl/linux.
2. The compiler is compiled using the newly compiled RTL. If successful, the newly compiled compiler executable is copied to a temporary executable.
3. Using the temporary executable from the previous step, the RTL is re-compiled.
4. Using the temporary executable and the newly compiled RTL from the last step, the compiler is compiled again.

The last two steps are repeated 3 times, until three passes have been made or until the generated compiler binary is equal to the binary it was compiled with. This process ensures that the compiler binary is correct.

Compiling for another target: When you want to compile the compiler for another target, you must specify the OS_TARGET makefile variable. It can be set to the following values: win32, go32v2, os2 and linux. As an example, cross-compilation for the go32v2 target from the win32 target is chosen:

cd compiler
make cycle OS_TARGET=go32v2

This will compile the go32v2 RTL, and compile a go32v2 compiler.

If you want to compile a new compiler, but you want the compiler to be compiled first using an existing compiled RTL, you should specify the all target, and specify another RTL directory than the default (which is the ../rtl/$(OS_TARGET) directory). For instance, assuming that the compiled RTL units are in /pp/rtl, you could type

cd compiler
make clean
make all UNITDIR=/pp/rtl

This will then compile the compiler using the RTL units in /pp/rtl. After this has been done, you can do the 'make cycle', starting with this compiler:

make cycle PP=./ppc386

This will do the make cycle from above, but will start with the compiler that was generated by the make all instruction.

In all cases, many options can be passed to make to influence the compile process. In general, the makefiles add any needed compiler options to the command-line, so that the RTL and compiler can be compiled. You can specify additional options (e.g. optimization options) by passing them in OPT.

F.4 Compiling by hand

Compiling by hand is difficult and tedious, but can be done. We'll treat the compilation of RTL and compiler separately.

F.4.1 Compiling the RTL

To recompile the RTL, so a new compiler can be built, at least the following units must be built, in the order specified:

loaders

the program stubs, that are the startup code for each pascal program. These files have the .as extension, because they are written in assembler. They must be assembled with the GNU as assembler. These stubs are in the OS-dependent directory, except for LINUX, where they are in a processor dependent subdirectory of the linux directory (i386 or m68k).

system

the system unit. This unit is named differently on different systems:

• Only on GO32v2, it's called system.
• For LINUX it's called syslinux.
• For WINDOWS NT it's called syswin32.
• For OS/2 it's called sysos2

This unit resides in the OS-dependent subdirectories of the RTL.

strings

The strings unit. This unit resides in the inc subdirectory of the RTL.

dos

The dos unit. It resides in the OS-dependent subdirectory of the RTL. Possibly other units will be compiled as a consequence of trying to compile this unit (e.g. on LINUX, the linux unit will be compiled, on go32, the go32 unit will be compiled).

objects

the objects unit. It resides in the inc subdirectory of the RTL.

To compile these units on a i386, the following statements will do:

ppc386 -Tlinux -b- -Fi../inc -Fi../i386 -FE. -di386 -Us -Sg syslinux.pp
ppc386 -Tlinux -b- -Fi../inc -Fi../i386 -FE. -di386 ../inc/strings.pp
ppc386 -Tlinux -b- -Fi../inc -Fi../i386 -FE. -di386 dos.pp
ppc386 -Tlinux -b- -Fi../inc -Fi../i386 -FE. -di386 ../inc/objects.pp

These are the minimum command-line options, needed to compile the RTL.

For another processor, you should change the i386 into the appropriate processor. For another operating system (target) you should change the syslinux in the appropriate system unit file, and you should change the target OS setting (-T).

Depending on the target OS there are other units that you may wish to compile, but which are not strictly needed to recompile the compiler. The following units are available for all plaforms:

objpas

Needed for Delphi mode. Needs -S2 as an option. Resides in the objpas subdirectory.

sysutils

many utility functions, like in Delphi. Resides in the objpas directory, and needs -S2 to compile.

typinfo

functions to access RTTI information, like Delphi. Resides in the objpas directory.

math

math functions like in Delphi. Resides in the objpas directory.

mmx

extensions for MMX class Intel processors. Resides in in the i386 directory.

getopts

a GNU compatible getopts unit. resides in the inc directory.

heaptrc

to debug the heap. resides in the inc directory.

F.4.2 Compiling the compiler

Compiling the compiler can be done with one statement. It's always best to remove all units from the compiler directory first, so something like

rm *.ppu *.o

on LINUX, and on DOS

del *.ppu
del *.o

After this, the compiler can be compiled with the following command-line:

ppc386 -Tlinux  -Fu../rtl/linux -di386 -dGDB pp.pas

So, the minimum options are:

1. The target OS. Can be skipped if you're compiling for the same target as the compiler you're using.
2. A path to an RTL. Can be skipped if a correct ppc386.cfg configuration is on your system. If you want to compile with the RTL you compiled first, this should be ../rtl/OS (replace the OS with the appropriate operating system subdirectory of the RTL).
3. A define with the processor you're compiling for. Required.
4. -dGDB is not strictly needed, but is better to add since otherwise you won't be able to compile with debug information.
5. -Sg is needed, some parts of the compiler use goto statements (to be specific: the scanner).

So the absolute minimal command line is

ppc386 -di386 -Sg pp.pas

You can define some other command-line options, but the above are the minimum. A list of recognised options can be found in table (FPCdefines) .

Table: Possible defines when compiling FPC

Define	does what
USE_RHIDE	Generates errors and warnings in a format recognized
	by RHIDE.
TP	Needed to compile the compiler with Turbo or Borland Pascal.
Delphi	Needed to compile the compiler with Delphi from Borland.
GDB	Support of the GNU Debugger.
I386	Generate a compiler for the Intel i386+ processor family.
M68K	Generate a compiler for the M68000 processor family.
USEOVERLAY	Compiles a TP version which uses overlays.
EXTDEBUG	Some extra debug code is executed.
SUPPORT_MMX	only i386: enables the compiler switch MMX which
	allows the compiler to generate MMX instructions.
EXTERN_MSG	Don't compile the msgfiles in the compiler, always use
	external messagefiles (default for TP).
NOAG386INT	no Intel Assembler output.
NOAG386NSM	no NASM output.
NOAG386BIN	leaves out the binary writer.

This list may be subject to change, the source file pp.pas always contains an up-to-date list.