When developing the Linux kernel, it is quite troublesome to build, install, reboot and test it every time you rewrite it. [QEMU](https://www.qemu.org/) saves me some trouble. In this article, I will share my kernel development environment. ## Environment I usually do all my development on EC2. I'm not particular about Linux distributions, so I choose Amazon Linux 2 somehow. OS: Amazon Linux 2 ## Steps I will work in my home directory (~/). Please rewrite each command as necessary. ``` $ WORKDIR=~ ``` ### 1. Get the Linux kernel source code First, let's clone the Linux kernel repository with git. You can download the latest source code of each subsystem from [git.kernel.org](https://git.kernel.org/). I chose the [net-next](https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/) tree of netdev for developing the networking subsystem. Unless you have a specific one, you should select the [linux-next](https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/) tree. ``` $ cd ${WORKDIR} $ sudo yum install -y git $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git ``` ### 2. Build the Linux kernel I will emulate the x86_64 architecture with QEMU, so I generated the default configuration for x86_64 in `.config` and built the Linux kernel. You can change the configuration with `make menuconfig`, but in that case, you need to install `ncurses-devel`. ``` $ sudo yum install -y gcc flex bison elfutils-libelf-devel openssl-devel $ cd net-next $ make x86_64_defconfig $ make -j $(nproc) ``` ### 3. Get the Buildroot source code The root file system is required to boot the Linux kernel. I use the [Buildroot](https://buildroot.org/) to create a root file system easily that contains a set of essential commands. ``` $ cd ${WORKDIR} $ git clone git://git.buildroot.net/buildroot ``` ### 4. Build the Buildroot You can build a root file system as follows. At that time, configure the architecture and necessary commands with `make menuconfig`. It will take much time to build, so you may want to spin the washing machine and wash the dishes while streaming your [favourite artist's live](https://www.youtube.com/watch?v=YU-RGLuC-Ho) on YouTube. ``` $ sudo yum install -y ncurses-devel gcc-c++ patch perl-Data-Dumper perl-ExtUtils-MakeMaker perl-Thread-Queue $ cd buildroot $ make menuconfig $ make -j $(nproc) ``` I chose x86_64 architecture to run the Linux kernel on the x86_64 virtualized system. Also, I chose ext4 for a file system because I have been using Ubuntu for a long time, but any file system is OK if it is a read-write file system. If you want to develop Btrfs, choose btrfs. - Target options - Target Architecture - select "x86_64" - Filesystem images - check "ext2/3/4 root filesystem" - ext2/3/4 variant - select "ext4" You may want to run commands in multiple sessions, so I recommend openssh. I also selected curl, iptables, nginx, tcpdump, and python3. - Toolchain - C library - select "glibc" - check "Install glibc utilities" - Target packages - Interpreter languages and scripting - check "python3" - core python3 modules - check "ssl" - Libraries - Crypto - openssl support - ssl library - select "openssl" - openssl - check "openssl binary" - check "openssl additional engines" - Networking - check "libcurl" - checl "curl binary" - SSL/TLS library to use - select "OpenSSL" - Networking applications - check "iptables" - check "nginx" - check "ngx_http_ssl_module" - check "openssh" - check "tcpdump" ### 5. Configure DHCP You must assign an IP address to a virtual machine to connect to the external network. If you do not specify the `-netdev` argument, you can use a built-in DHCP server. (See also: [QEMU - Networking - User-mode networking](https://wiki.archlinux.org/index.php/QEMU#User-mode_networking)) The DHCP configurations are in `/etc/network/interfaces` in the file system (`${WORKDIR}/buildroot/output/images/rootfs.ext4`) created by Buildroot. ``` # interface file auto-generated by buildroot auto lo iface lo inet loopback ``` By default, only is the loopback network interface (lo) assigned an IP address, so you need to put the following in the file to assign an IP address to eth0. ``` auto eth0 iface eth0 inet dhcp ``` Type these commands to mount the root file system and add settings. ``` $ sudo mkdir /mnt/buildroot $ sudo mount -o loop ${WORKDIR}/buildroot/output/images/rootfs.ext4 /mnt/buildroot $ echo -e '\nauto eth0\niface eth0 inet dhcp' | sudo tee -a /mnt/buildroot/etc/network/interfaces ``` ### 6. Configure SSH The default user is the root and no password is set. This is a test environment in QEMU, so you need not set up public-key authentication for another user. I modified `/etc/ssh/sshd_config` as follows to perform SSH login as root without a password. ``` PermitRootLogin yes PermitEmptyPasswords yes ``` Configure `/etc/ssh/sshd_config` in the root file system and umount it. ``` $ sudo sed -i -e 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' \ -e 's/#PermitEmptyPasswords no/PermitEmptyPasswords yes/' \ /mnt/buildroot/etc/ssh/sshd_config $ sudo umount /mnt/buildroot ``` ### 7. Run the Linux kernel on QEMU Once you install QEMU, you can run the Linux kernel with a bit long command. The command changes according to git trees and file systems. This command binds port 10022 on the host with port 22 (SSH) on the guest. ``` $ sudo yum install -y qemu $ qemu-system-x86_64 -boot c -m 2048M \ -kernel ${WORKDIR}/net-next/arch/x86/boot/bzImage \ -hda ${WORKDIR}/buildroot/output/images/rootfs.ext4 \ -append "root=/dev/sda rw console=ttyS0,115200 nokaslr" \ -serial stdio -display none \ -nic user,hostfwd=tcp::10022-:22 ``` You can log in as root without a password. ``` Welcome to Buildroot buildroot login: root # uname -r 5.8.0-rc1 ``` From another terminal, you can perform SSH login. ``` $ ssh root@localhost -p 10022 ... # curl -kI https://kuniyu.jp HTTP/1.1 200 OK ... ``` To terminate QEMU, press Ctrl+C in the terminal where you launched QEMU. ``` # (press Ctrl+C) qemu-system-x86_64: terminating on signal 2 ``` ### 8. Add an alias I cannot remember the command of course, and it is also troublesome to copy and paste it every time to launch QEMU. So, I created an alias to run QEMU with the `net-next` command. ``` $ echo "alias net-next='qemu-system-x86_64 -boot c -m 2048M -kernel ${WORKDIR}/net-next/arch/x86/boot/bzImage -hda ${WORKDIR}/buildroot/output/images/rootfs.ext4 -append \"root=/dev/sda rw console=ttyS0,115200 nokaslr\" -serial stdio -display none -nic user,hostfwd=tcp::10022-:22'" >> ~/.bashrc $ source ~/.bashrc ``` ## Postface When I started kernel development half a year ago, I was rewriting the code and building it again and again. I often misconfigured GRUB or wrote wrong code, resulting in that EC2 has gone away... I hope newbies like me will not have such a bitter experience.