Class 1: kernel compilation

Date: 28.02.2019

Reading

  • Kernel-HOWTO (unfortunately, obsolete)
  • Kernel-Build-HOWTO
  • /usr/src/linux/README
  • /usr/src/linux/Documentation/*
  • info grub

Preparation of kernel sources

  • Clone the git repository from git.kernel.org (repository linux/kernel/git/linux-stable) or download and unpack tarball in version 4.20.8 from https://www.kernel.org/. If using git, check out tag v4.20.8.

Information about kernel versions – X.Y.ZZ.WW(-abc)

Formerly (kernel versions before 2.6)

Back then the numbers were in the form X.Y.ZZ(-abc).

The first number (X) – the main version number, changed on radical changes in the structure of the kernel. The first versions of Linux had (shortly) the number 0, then for a long time 1; the current main version is 4.

The second number (Y) – line or series of kernels, even numbers indicate ‘stable’ versions; odd – ‘development’ versions. In development versions, new concepts are being tested, which are later possibly transferred to stable versions. Stable versions are also constantly updated, so in a sense they are also developing.

The third number (ZZ) is the version of the kernel in the given line, the whole number is designated ‘release’.

In addition, releases can be marked with various suffixes, mostly specifying the types of patches applied to them or the version number of the test version. The “pre” suffix is for beta versions, e.g. 2.4.20pre7 means beta version 7 of kernel 2.4.20. After testing is complete, this kernel will receive the number 2.4.20, or it will turn into 2.4.20pre8 if Linus does not approve the version as an official release.

There are also suffixes designating kernels created in parallel to the “official” (validated by Linus Torvalds) kernels, e.g.:

  • arca – created by Andree Arcangeli,
  • ac – created by Alan Cox, e.g. 2.0.36-ac12.

Version numbers for kernels 2.6 and 3+

Starting from the 2.6 series, the “big” development series with odd numbers have been abandoned in favor of a more continuous development. In the 2.6 series, version numbers they are form 2.6.XX.YY. The third number (XX) is the version of the kernel in the given line – it is changed when new drivers are added or new functionality is implemented. A new kernel version that differs from the previous one only in bug fixes has a number that differs only in the fourth part (YY), a so-called “stable” branch. Of course there are also kernels released by people other than Linus Torvalds – usually their name ends with a suffix.

In July 2011, version 3.0 was released, starting a new series. In that series, version numbers are of the form 3.XX.YY, where XX and YY have the same meaning as in the 2.6 series. This change does not reflect any new functionality – version 3.0 does not differ more from 2.6.40 than 2.6.40 from 2.6.39. The transition to the new numbering system has been carried out due to the inadequacy of the old versioning system to the new kernel development model – version numbers were simply too long. From this time, the main version number is simply increased when the secondary number becomes “big enough” – it is not related to functionalities added or the number of changes made. Likewise, in April 2015 version 4.0 has been released (after version 3.19).

Downloading kernel sources

The easiest way to get kernel sources is to download a compressed .tar file from https://kernel.org/, which is the official kernel release archive. In addition to complete packages with a source, .patch files are also published, allowing you to update the previously downloaded package to a newer version.

A bit more complicated, but a much more flexible way of getting the sources is using git. This allows you to work on the latest code (not yet included in any official release), and to move almost immediately between all previously released versions included in history (from 2.6.12). Using git is also required if you send your own changes to be included in the official kernel version.

There are many git repositories with Linux sources – each subsystem is developed in its own repository, which is then merged into the main repository when it’s time to release a new version. The main reposityry, maintained by Linus Torvalds and used as a basis for new kernel releases, is located at the following address:

git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

All “large” releases (2.6.XX, 3.XX, 4.XX) and the release candidates are available in this repository as tags (v2.6.27, v3.1-rc3, etc.). Stable releases containing bugfixes (2.6.XX.YY, 3.XX.YY) are developed in a separate repository, available at the following address:

git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git

If we want to work only on external kernel modules without modifying existing code, it is not necessary to have complete sources. Sufficient kernel headers, installed mostly in w /usr/src/linux and symlinked to /lib/modules/<version>/build. However, they must be true kernel headers, not those intended for use by libc. The kernel version and configuration from which these headers come must also exactly match the kernel version under which the compiled modules will be used. Some distributions ship such headers in a separate package, named for example linux-headers.

Distributions also often ship kernel sources as packages in the standard repository – such sources usually contain patches applied by the distribution and are not identical to the official release.

Applying patches to the kernel source

Instead of downloading an entire new source tarball, you can apply patches (files named e.g. patch-xx.xx.xx.gz) on the old source through the patch command, e.g.

cd /usr/src/linux
gzip -cd patch.xx.xx.xx.gz | patch -p1

You can also use the patch-kernel script in the scripts directory (it automatically applies patches found in the directory from which it was launched).

The ‘official’ patches with the name patch-2.6.17.14.gz will work with the previous release (relative to the release named in the patch name), i.e. the mentioned patch will work with the 2.6.17.13 kernel. Unofficial patches (for example patch-2.6.11-ac4.gz) usually refer to the same release as in the name.

Before installing a new kernel, read the file Documentation/Changes that contains compiler version requirements, installed packages, library versions, etc. and make sure that the relevant versions are installed.

The structure of the kernel sources

The contents of the main directory

Documentation
a directory containing the documentation – in particular, please read the CodingStyle file
arch
source code dependent on the processor architecture
block
kernel block layer functions
crypto
cryptographic functions (as well as compression and decompression)
drivers
device drivers
firmware
the auxiliary code loaded by some device drivers to the controlled devices
fs
file systems
include
header files
init
platform independent part of the system initialization
ipc
IPC (System V inter-process communication)
kernel
kernel core – process management, interrupts, DMA, time
lib
auxiliary procedures (eg writing to the screen, unpacking compressed kernels)
mm
memory management
net
network protocols
samples
examples of the use of some internal kernel interfaces
scripts
scripts (e.g. for configuration)
security
security code (LSM – Linux Security Modules)
sound
sound card drivers and sound system code (ALSA)
usr
support programs; currently gen_init_cpio used to create a ramdisk loaded with the system kernel.
virt
virtualization-related code (KVM)

arch – code dependent on the hardware platform

The code for Intel processors is in the arch/x86 directory. It is possible to compile the kernel for a different processor than the one we work on. However, it requires (besides Linux sources) a compiler for the given platform, operating on our system (a so-called cross-compiler).

Kernel headers

The kernel contains two sets of headers: internal headers and headers for user space. The internal headers are intended for use only by kernel code and modules, and are installed in /usr/src/linux (if at all). The user space headers are in subdirectories named uapi and are intended for use by both kernel code and user programs. These headers are installed in /usr/include. Thanks to compatibility guarantees of kernel interfaces, the installed headers version may be different from the version of the kernel used.

Inside the kernel sources, headers are scattered across multiple directories:

  • include: main kernel headers
  • include/generated: main kernel headers, generated at compile time
  • arch/<processor>/include: kernel headers specific to the architecture
  • arch/<processor>/include/generated

The most important header subdirectories are asm, asm-generic and linux.

Kernel configuration

Before compilation, the kernel has to be configured with one of the following commands:

  • make config (text version, will ask one question for every option – not recommended),
  • make menuconfig (ncurses),
  • make xconfig (X11),
  • make oldconfig (like make config, but updates a configuration from an older kernel version – only asks about new options).

The simplest to use version is menuconfig or xconfig, though in the latter there are more errors.

The following points describe the essential elements of the configuration (the titles correspond to elements of the main menu in menuconfig).

Warning

Kernel options have quite complex dependencies and configuration programs do not show options that are excluded by other choices (e.g. if support for virtio devices is not selected in virtualization options, we will not see the virtual network card virtio-net option in the network drivers options at all). If we are unable to find an option where we expect it, it is worth using the search function (just press / in make menuconfig) – the results will show us where the option is in the selection tree and what options it depends on.

64-bit kernel

Selects whether the compiled kernel will run in 64-bit mode and support running 64-bit programs (running 32-bit programs is always possible, unless we explicitly disable this option in later configuration).

General setup

This part of the configuration controls the key components of the Linux kernel. The most important options are:

  • ‘Support for paging of anonymous memory (swap)’ – support for virtual memory on the disk
  • ‘System V IPC’ – handling inter-process communication
  • ‘Initial RAM filesystem and RAM disk (initramfs/initrd) support’ – allows booting Linux from ramdisk loaded before running (e.g. by GRUB), which allows you to load drivers for disks or file systems available only as modules or start the system from software-RAID devices.
  • ‘Initramfs source file (s)’ – list of files to be included in the ramdisk
  • ‘Embedded system’ – it does not change anything in itself, but it causes the appearance of options allowing you to disable the functionality usually considered necessary.

Enable loadable module support

Note

Module – a part of the kernel code that can be loaded or removed from the kernel on demand. All parts of the kernel that are not needed at startup system and are not constantly used while the system is running should be compiled as modules. Even most parts needed at boot time can be modules, as long as an initial ramdisk is used.

Configures support for kernel modules. Depending on your needs, you can enable or disable module loading (Enable loadable module support), enable removing modules (Module unloading, Forced module unloading), enable automatic loading of modules by the kernel (Automatic kernel module loading) and enable loading modules compiled for other kernel versions by adding additional information about required functions (Module versioning support). We can also put a checksum in all modules (Source checksum for all modules).

Processor type and features

Allows you to configure support and optimization of the kernel for a given processor (Processor family) – when in doubt, the safe settings for i386 architecture are 386 (always) and 586 (for pentium processors and up) or Pentium (for Intel processors from pentium up). Note – if you select an incorrect value, the kernel may not work at all, or work erroneously. Other important options available in this menu are:

  • ‘High Memory Support’ – support for systems with> 1GB of memory (32-bit only),
  • ‘Math emulation’ – emulation of the 387 unit for old processors without FPU,
  • ‘MTRR (Memory Type Range Register) support’ – support for memory access control registers, allowing the PCI/AGP bus to be set to “write-combining” mode, which can significantly speed up graphics applications.
  • ‘Symetric multi-processing support’ – support for multiple processors
  • ‘SMT (Hyperthreading) scheduler support’ – improving scheduler properties for systems with a processor with HT support.
  • ‘Preemption Model’ – allows you to choose whether the kernel can be preempted.
  • ‘Local APIC support on uniprocessors’ – support for advanced interrupt controllers available on newer motherboards. This option is only available if SMP is off (for SMP systems, APIC is always on).

Power management and ACPI options

Selection of supported energy saving methods – including ACPI support and changes in processor speed during system operation.

Bus options (PCI etc.)

Selection of supported system buses and their parameters. For modern computers, it’s a good idea to enable PCI bus support.

Executable file formats / Emulations

Support for executable file formats. Without ELF format support, there’s not much that can be done with traditional Linux distributions. In the case of 64-bit kernels, we can enable or disable support for 32-bit programs here.

Networking support

Typically, it is not possible to disable network support (Networking support), because not much would work without it. In addition to the ‘Networking options’ menu described below, various communication methods can be configured here – infrared, Bluetooth, Wi-Fi.

Networking options

This menu contains the configuration of network components and protocols. The most important of them are:

‘Packet socket’
direct access to network devices.
‘Unix domain sockets’
Unix sockets, enabling inter-process communication in a similar way to network communication. Such sockets are used e.g. by X-Windows, PostgreSQL.
‘TCP/IP Networking’
TCP/IP protocol support – very important. But difficult to not choose.
‘The IPv6 protocol’
support for the new version of TCP/IP. Currently not yet necessary, but in a while you probably will not be able to do without it.
‘Network packet filtering framework (Netfilter)’
filtering and modifying packets (firewall, NAT).

Device Drivers

Various driver settings grouped into multiple menus.

If you are building a kernel for classes, it is best to choose only necessary drivers – this will significantly speed up the process of building the kernel.

Block devices

The most important options are:

‘Loopback device support’
pseudo-device to create a block ‘device’, the contents of which are stored in a regular file.
‘RAM block device support’
support for RAM disks.
‘Packet writing on CD / DVD media’
allows you to write CDs / DVDs
‘Virtio block driver’
virtualized block device with a low overhead

NVM Express block device

Support for SSD drives mounted directly on the motherboard (M.2 connector).

SCSI device support

It mainly enables SCSI bus support, but also allows support for many other kinds of block devices using SCSI emulation (including SATA, ATA and USB). To use these devices, enable ‘SCSI disk support’, ‘SCSI CD-ROM support’, ‘SCSI generic support’. Additionally, in the ‘SCSI low-level drivers’ menu you can enable support for a hardware SCSI controller (if you have one).

Serial ATA and Parallel ATA drivers (libata)

Support for ATA and SATA disk devices. To use a disk, you should also enable support for your ATA or SATA controller here. ‘AHCI SATA Support’ and ‘Generic ATA support’ options support most devices, but may have less functionality than a specialized driver. This driver is made on the basis of the SCSI layer – to use the disk or optical drive, you should also enable support for the right type of device in the SCSI menu.

Multiple devices driver support (RAID and LVM)

‘RAID support’
enables software RAID support, enabling using multiple disks as one, which can improve performance and safety of disk operations.
‘Device mapper support’
support for low-level volume manager, which is used by programs that allow to define colume groups and logical disks (volumes) on them to simplify disk management in large systems.

The ‘RAID support’ option is also useful in home systems, assuming they have at least two hard drives – in this situation RAID-0 mode allows doubling the performance of disk operations.

Network device support

Allows you to compile the drivers for wred network adapters (‘Ethernet (10 or 100Mbit)’ , ‘Ethernet (1000 Mbit)’), wireless network cards (‘Wireless LAN’) and PPP support (‘PPP (point-to-point protocol) support’), as well as many other types of network cards and protocols. These four options, however, will be used most often.

Input device support

Support (general) for input devices. If we want to use mouse, keyboard, joystick or similar devices, turn on this option (luckily it is difficult to disable it) and the appropriate module. Support for USB input devices is enabled in the ‘HID Devices’ menu (see below).

Character devices

The most important option is ‘Virtual Terminal’, allowing you to use the Linux console. By default, it is invisible and enabled (it can only be disabled for embedded systems). You can also enable serial port support here.

Graphics support

Selection of supported graphics devices and console support (in the ‘Console display driver support’ menu). ‘VGA text console’ enables console support on a VGA device. This menu also contains hardware-accelerated graphics drivers (Direct Rendering Manager).

Sourd card support

Selection of supported sound systems and sound card drivers. It is recommended to use the ALSA system (Advanced Linux Sound Architecture).

HID support

Support for HID devices – mainly keyboards and mice connected via USB.

USB support

Here you can select supported chipsets and devices connected by the USB bus. ‘USB Mass Storage support’ allows the use of mass storage devices, including disks and flash memory. Many USB devices are located in other categories – for example, network cards are in ‘Network device support’.

Virtio drivers

Enables support for virtio devices – virtual devices with low overhead, provided by QEMU. It is worth to enable this option when compiling the kernel for a virtual machine. Some of the virtio drivers can be found in other places (eg virtio network device is among other network cards).

Other sections with drivers

In the remaining menus, you can configure various devices located in the system. Usually, they can be easily compiled as modules, because they are not needed to boot the system.

File systems

Allows you to enable support for various file systems. The most important thing is the system that is used on the system boot partition (usually ext4 or btrfs) – it must be compiled into the kernel or stored on the initial ramdisk. Other file systems can be compiled as modules. It is also important to select support for ‘Tmpfs virtual memory file system support’, ‘/proc’ and sysfs (all from ‘Pseudo filesystems’). ‘Filesystem in Userspace support’ is FUSE, which allows using file system drivers running in the user space. Other file systems can be compiled depending on your needs.

Compiling the kernel – Kbuild

Kbuild is the Linux build system. It is built on top of specially prepared Makefiles.

As always with Makefiles, it is used as follows:

make <options> <target> <optional variables for Kbuild>

A useful option to make is -j <N> – it will invoke parallel compilation with up to N processes at the same time (please do not overuse it on students).

make clean
Removes compiled files.
make bzImage
Compiles the kernel and places it in the arch/<architecture>/boot directory under as bzImage. This kernel is compressed (it unpacks itself at system startup).
make modules
Compiles parts of the kernel that have been configured as modules. They should then be installed with the command make modules_install.
make all
Works like make bzImage with make modules.
make modules_install
Installs modules into the /lib/modules/<version>/ directory and calls depmod to create dependency information. If the variable INSTALL_MOD_PATH is given, it installs in $INSTALL_MOD_PATH/lib/modules/<version>/.
make help
Shows available make commands.
make mrproper
Cleans the source directory exactly (including the configuration!), Removes dependencies, modules, etc.
make prepare
Prepares the target build directory. Especially useful when building in a directory other than the source.
make install
Installs the kernel and adds it to the bootloader configuration. Does not always work as expected.
make htmldocs
Compiles documentation in the DocBook format to HTML format.
make pdfdocs
Compiles documentation in the DocBook format to PDF format.
make rpm
Creates a RPM package with the kernel (useful in the RedHat system).

Variables

Some kernel compilation parameters can be specified by the make program variables (adding VARIABLE=value at the end of the make command). The most important:

V=1
Verbose, Kbuild will write exactly what it does.
ARCH=<arch>
Forces <arch> architecture, i386 for example.
EXTRA_CFLAGS=<flags>
When compiling, <flags> will be added to gcc calls (it may be useful to provide -g to have symbols).
INSTALL_MOD_PATH=<path>
Installs modules in the specified location.
O=<path>
Places the resulting files in a separate directory; after the first make call with this option, you can use make in the specified directory without additional options (an appropriate Makefile is placed there).

Exercise

Configure and compile the kernel and modules, then place the kernel in the /boot directory.

Remember to include all the options necessary to start the system, including:

  • selecting 64-bit kernel
  • PCI bus support (Bus options -> PCI support)
  • ELF format support (Exectable file formats -> Kernel support for ELF binaries)
  • virtio support (Device drivers -> virtio -> PCI driver for virtio devices)
  • virtio balloon support (Device drivers -> virtio -> Virtio balloon driver)
  • UNIX sockets (Network support -> Networking options -> Unix domain sockets)
  • IPv4 protocol (Network support -> Networking options -> TCP/IP networking)
  • virtio block driver (Device drivers -> Block devices -> Virtio block driver)
  • virtio net driver (Device drivers -> Network device support -> Virtio network driver)
  • virtconsole driver (Device drivers -> Character devices -> Virtio console)
  • ext4 file system (File systems -> The Extended 4 (ext4) filesystem)
  • proc file system (File systems -> Pseudo filesystems -> /proc file system support)
  • FUSE file system (File systems -> FUSE (Filesystem in Userspace) support)

Bootloaders

A bootloader is a program that loads the operating system. When the computer starts, the BIOS loads the master boot record (MBR) and gives it control. The MBR has a partition table and code that can be the code that loads the system (eg DOS loading code). Its task is to load the boot sector from the appropriate partition and give it control. If, however, we want to organize the coexistence of several systems on one disk, we put a loading program code in the MBR, which allows you to select at bootup which boot sector is to be loaded. Examples of such loading programs (for Linux on an x86 architecture) are:

  • grub: allows you to boot from the hard disk, directly supports a wide range of file systems and partition tables. In addition to the Linux kernel, it also supports the multiboot standard, FreeBSD / OpenBSD / NetBSD, and loading other bootloaders (so-called chain-loading). Has a fairly extensive command line, allowing (among others) for browsing the file system and modifying the kernel command line.
  • grub2: a more modular successor to grub with more functionality, already displacing the original.
  • syslinux: actually a bundle of several lightweight bootloaders:
    • syslinux: boots a Linux system from a disk partition, rarely used.
    • isolinux: boots the system from a CD or a DVD using ISO 9660 file system.
    • pxelinux: boots the system from the network, using PXE environment.
  • lilo: once a very popular bootloader for Linux, obsolete today.

GRUB

GRUB can be used through two interfaces: a more powerful command line interface or a simple menu. Usually the latter is enough. Once started, GRUB searches for a configuration file. If it’s found, GRUB displays entries from this file (corresponding to Linux images or other systems) in the form of menu items. The menu items can be edited (but the changes only affect the current boot, they are not remembered) – switching to the editing mode is done by pressing ‘e’ (and in this way it can be possible to fix errors from the configuration file). From the menu, you can also go to the command line by pressing ‘c’ (return via ESC). If your GRUB is protected by a password, it is only possible to enter the editing mode or the command line after pressing ‘p’ and entering the password.

The GRUB configuration file is /etc/grub.conf. The file consists of entries for kernel images or other operating systems. The entries are numbered starting from 0.

The most important commands included in an entry are:

  • TITLE image_name – the name that will be visible in the menu. Starts the entry; the entry ends at the end of the file or with the next title command (starting next entry).
  • ROOT root_device – where we will look for the image.
  • ROOTNOVERIFY root_device – as above, but the device will not be mounted, and the command is usually used to specify the location of another boot loader, which allows the so-called chain-loading and loading eg Windows.
  • KERNEL image_file [kernel_options]

The important commands that are also part of grub.conf are:

  • DEFAULT=entry_index – which entry will be selected by default (if not specified, entry 0 will be selected).
  • TIMEOUT=time_in_seconds – GRUB will wait time_in_seconds for system selection. If this does not happen, it will load the default.

Line comments are started with the # sign.

The grub.conf file can look eg. like this:

# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/hda2

default=1
timeout=3

title dos
    rootnoverify (hd0,0) # sets the GRUB root device, without mounting
    makeactive
    chainloader +1       # loads another boot loader
title linux
        root (hd0,4)
        kernel /vmlinuz ro root=/dev/hda5 hdc=ide-scsi
title new
    root (hd0,4)
    kernel /vmlinuz-2.2.17 ro root=/dev/hda5 hdc=ide-scsi

GRUB 2

GRUB 2 is the successor to GRUB, with greater modularity and functionality. The menu interface is quite similar to GRUB, but the configuration has changed significantly.

The GRUB 2 configuration file is /boot/grub/grub.cfg, but it should not be modified directly – it is generated by grub-mkconfig program based on scripts in the /etc/grub.d directory and configuration in the file /etc/default/grub.

Thanks to the configuration scripts, you do not have to manually create the configuration entries for each kernel - just put the kernel as /boot/vmlinuz-<version>, and possibly initramfs as /boot/initrd.img-<version>. Options passed to the installed kernel can be set in the /etc/default/grub file.

After changing the configuration, issue the following command:

grub-mkconfig -o /boot/grub/grub.cfg

To install GRUB 2 for the first time, you must issue the following command:

grub-install /dev/<disk>

Exercise

  1. Modify the /etc/grub.conf file so that it can load the newly created image.
  2. Restart the computer, ask the gods for support and launch the newly created kernel.