Introduction

Introduction

Welcome to our brief tutorial articles concerning how to customize your distribution based on the Yocto Project.

This is a written companion to our video tutorial series entitled "Yocto Project Customization". To play the entire series, you can go here:

Play the tutorial series on YouTube.

We also link in the specific episodes that correspond to each section here as well, to make it easier for you to view the parts of the tutorial that most important or relevant to the problem you are trying to solve.

Motivation

Why customize your distribution? First, what we provide are recipes for images that are meant to run on our development boards. After this, developers usually want to make changes to those images so that the distribution works well for their end application. The logical step for developers is customize them for their purposes, which with the Yocto Project is not very intuitive and it takes some time to learn.

In these tutorials, we present the process to customize a distribution based on our Yocto BSP. You'll learn how (and why) to:

• Create your own metadata layer, which is important to be able to maintain the customizations you'll need for your distribution.
• Create your own MACHINE, which allows you to control to a much greater extent the code that supports your hardware.
• Create a custom bootloader recipe. At first, this will just be modifying the TechNexion bootloader recipe using a special file called a .bbappend file, which may make some sense for some projects, and change the splash screen in u-boot (something that almost every developer with a display in their embedded product wants to do).
• Create a custom kernel recipe - so that you can build from your own kernel source, and configuration, and make changes to device trees and so on.
• Finally - Create your own image recipes which you can use to add packages and fully customize your image.

Creating a Metadata Layer

In this first section, we lay the foundation. We make a custom layer that will contain all of the customized recipes for our product. The first file we'll add to this layer is a custom machine file.

Why use your own layer?

The entire BSP is more easily maintained and integrated with the rest of Yocto Project metadata if you utilize the Open Embedded layering scheme properly. While it is possible to modify other meta layers directly in order to customize them for your purposes, when those layers are updated, incorporating the upstream updates can be much more difficult. You'll either need to rebase your changes, or re-port your changes.

If you are maintaining them in your own layer, it is much easier to marge updates to other layers.

Creating your custom metadata layer

The following steps are to set up the BSP for the first time. If you've already done this, you can skip ahead.

Download, Configure, and Build for the first time

Get the repo tool

mkdir ~/bin
curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
chmod a+x ~/bin/repo
PATH=${PATH}:~/bin

Download the BSP sources

By default, we will put them into a directory in the home directory named edm_yocto.

PATH=${PATH}:~/bin
mkdir edm_yocto
cd edm_yocto

Get the sources (based on Yocto Zeus, or Yocto version 3.0):

repo init -u https://github.com/TechNexion/tn-imx-yocto-manifest.git -b zeus_5.4.y-next -m imx-5.4.70-2.3.0.xml
repo sync -j8

Install required host packages

This assumes you are running Ubuntu 18.04, but has also been shown to work on Ubuntu 20.04:

sudo apt-get install gawk wget git git-core diffstat unzip texinfo gcc-multilib build-essential \
chrpath socat cpio python python3 python3-pip python3-pexpect \
python3-git python3-jinja2 libegl1-mesa pylint3 rsync bc bison \
xz-utils debianutils iputils-ping libsdl1.2-dev xterm \
language-pack-en coreutils texi2html file docbook-utils \
python-pysqlite2 help2man desktop-file-utils \
libgl1-mesa-dev libglu1-mesa-dev mercurial autoconf automake \
groff curl lzop asciidoc u-boot-tools libreoffice-writer \
sshpass ssh-askpass zip xz-utils kpartx vim screen

Configure the BSP for the first time

For this tutorial, we are assuming that we are building a custom machine based on the PICO-IMX8M-MINI module, so our configuration MACHINE will be pico-imx8mm. In addition, we'll be building an distribution that supports Xwayland.

MACHINE=pico-imx8mm DISTRO=fsl-imx-xwayland WIFI_FIRMWARE=y source tn-setup-release.sh -b build-xwayland

This process sets up the build directory, ensures that you agree with the NXP EULA, initializes the configuration files for the MACHINE and DISTRO, and initializes the environment to build for the first time.

Build an image

Build an image for the first time. In this case, we'll build an image that is more full-featured for testing, but doesn't include a lot of extra packages.

bitbake imx-image-core

When this is done for the first time, it can take quite a while to complete, as all packages have to be downloaded, all of the binary utilities needed to build those packages have to be built, and then all of the packages are built.

Subsequent builds do not take as long because of something called the Shared State Cache. As the build progresses, all of the final products and intermediate products of the build are stored in the shared state cache, which removes the need to rebuild packages from scratch every time.

cd edm_yocto
. ./setup-environment.sh <build directory>

What is needed now is to create a custom layer for our project. For this, we are going to need a project name. We selected the name "scorpion" because, well, no reason really. We just thought it sounded cool.

Creating the meta-scorpion layer

Once you have an image built, and the sources already set up correctly, you're ready to make your own layer. This is where all of the customization will be contained for your project.

First, let's view all of the layers in our project. Make sure that you have set up the environment properly or the bitbake-layerscommand will not be in your path.

Show the layers using the bitbake-layers command with the show-layers argument:

~/edm_yocto/build-xwayland$ bitbake-layers show-layers
NOTE: Starting bitbake server...
layer                 path                                      priority
==========================================================================
meta                  /home/john/projects/tn-yocto/zeus-next/sources/poky/meta  5
meta-poky             /home/john/projects/tn-yocto/zeus-next/sources/poky/meta-poky  5
meta-oe               /home/john/projects/tn-yocto/zeus-next/sources/meta-openembedded/meta-oe  6
meta-multimedia       /home/john/projects/tn-yocto/zeus-next/sources/meta-openembedded/meta-multimedia  6
meta-python           /home/john/projects/tn-yocto/zeus-next/sources/meta-openembedded/meta-python  7
meta-freescale        /home/john/projects/tn-yocto/zeus-next/sources/meta-freescale  5
meta-freescale-3rdparty  /home/john/projects/tn-yocto/zeus-next/sources/meta-freescale-3rdparty  4
meta-freescale-distro  /home/john/projects/tn-yocto/zeus-next/sources/meta-freescale-distro  4
meta-bsp              /home/john/projects/tn-yocto/zeus-next/sources/meta-imx/meta-bsp  8
meta-sdk              /home/john/projects/tn-yocto/zeus-next/sources/meta-imx/meta-sdk  8
meta-ml               /home/john/projects/tn-yocto/zeus-next/sources/meta-imx/meta-ml  8
meta-nxp-demo-experience  /home/john/projects/tn-yocto/zeus-next/sources/meta-nxp-demo-experience  7
meta-gnome            /home/john/projects/tn-yocto/zeus-next/sources/meta-openembedded/meta-gnome  7
meta-networking       /home/john/projects/tn-yocto/zeus-next/sources/meta-openembedded/meta-networking  5
meta-filesystems      /home/john/projects/tn-yocto/zeus-next/sources/meta-openembedded/meta-filesystems  6
meta-python2          /home/john/projects/tn-yocto/zeus-next/sources/meta-python2  7
meta-browser          /home/john/projects/tn-yocto/zeus-next/sources/meta-browser  7
meta-clang            /home/john/projects/tn-yocto/zeus-next/sources/meta-clang  7
meta-rust             /home/john/projects/tn-yocto/zeus-next/sources/meta-rust  7
meta-qt5              /home/john/projects/tn-yocto/zeus-next/sources/meta-qt5  7
meta-tn-imx-bsp       /home/john/projects/tn-yocto/zeus-next/sources/meta-tn-imx-bsp  9
meta-nxp-nfc          /home/john/projects/tn-yocto/zeus-next/sources/meta-nxp-nfc  6

What this shows is the layer name, the path to the layer, and the priority.

Next, create the meta-scorpion layer. This can be in the sources subdirectory, or you can create it in a different directory. In this tutorial, we will put it with the rest of the metadata in the sources subdirectory.

~/edm_yocto/build-xwayland$ cd ../sources
~/edm_yocto/sources$ bitbake-layers create-layer meta-scorpion
NOTE: Starting bitbake server...
Add your new layer with 'bitbake-layers add-layer meta-scorpion'

Next, add the layer to your project using the add-layer argument. To add a layer, the current directory must be the build directory.

~/edm_yocto/sources$ cd ../build-xwayland

Now, add the layer, giving the path to the layer source:

~/edm_yocto/build-xwayland$ bitbake-layers add-layer ../sources/meta-scorpion

Finally, check to make sure that the layer was added by showing the layers again:

~/edm_yocto/build-xwayland$ bitbake-layers show-layers
NOTE: Starting bitbake server...
layer                 path                                      priority
==========================================================================
meta                  /home/john/projects/tn-yocto/zeus-next/sources/poky/meta  5
meta-poky             /home/john/projects/tn-yocto/zeus-next/sources/poky/meta-poky  5
meta-oe               /home/john/projects/tn-yocto/zeus-next/sources/meta-openembedded/meta-oe  6
meta-multimedia       /home/john/projects/tn-yocto/zeus-next/sources/meta-openembedded/meta-multimedia  6
meta-python           /home/john/projects/tn-yocto/zeus-next/sources/meta-openembedded/meta-python  7
meta-freescale        /home/john/projects/tn-yocto/zeus-next/sources/meta-freescale  5
meta-freescale-3rdparty  /home/john/projects/tn-yocto/zeus-next/sources/meta-freescale-3rdparty  4
meta-freescale-distro  /home/john/projects/tn-yocto/zeus-next/sources/meta-freescale-distro  4
meta-bsp              /home/john/projects/tn-yocto/zeus-next/sources/meta-imx/meta-bsp  8
meta-sdk              /home/john/projects/tn-yocto/zeus-next/sources/meta-imx/meta-sdk  8
meta-ml               /home/john/projects/tn-yocto/zeus-next/sources/meta-imx/meta-ml  8
meta-nxp-demo-experience  /home/john/projects/tn-yocto/zeus-next/sources/meta-nxp-demo-experience  7
meta-gnome            /home/john/projects/tn-yocto/zeus-next/sources/meta-openembedded/meta-gnome  7
meta-networking       /home/john/projects/tn-yocto/zeus-next/sources/meta-openembedded/meta-networking  5
meta-filesystems      /home/john/projects/tn-yocto/zeus-next/sources/meta-openembedded/meta-filesystems  6
meta-python2          /home/john/projects/tn-yocto/zeus-next/sources/meta-python2  7
meta-browser          /home/john/projects/tn-yocto/zeus-next/sources/meta-browser  7
meta-clang            /home/john/projects/tn-yocto/zeus-next/sources/meta-clang  7
meta-rust             /home/john/projects/tn-yocto/zeus-next/sources/meta-rust  7
meta-qt5              /home/john/projects/tn-yocto/zeus-next/sources/meta-qt5  7
meta-tn-imx-bsp       /home/john/projects/tn-yocto/zeus-next/sources/meta-tn-imx-bsp  9
meta-nxp-nfc          /home/john/projects/tn-yocto/zeus-next/sources/meta-nxp-nfc  6
meta-scorpion         /home/john/projects/tn-yocto/zeus-next/sources/meta-scorpion  6

Note that the meta-scorpion layer has been added and the default priority is 6. You can easily change the priority by modifying the <build dir>/conf/bblayers.conf file.

Creating a Custom Machine

What is a MACHINE?

A MACHINE in OpenEmbedded/Yocto is a piece of embedded hardware. However, in practice what this means is an specific embedded processor and all of the platform-specific packages that are necessary for all of the hardware features of the platform to work. Think kernel, bootloader, any firmware needed for boot, any special device drivers that are needed, the locations of the firmware in storage, and so on.

Why would you need a custom machine recipe? You might, or you might not, depending on your specific requirements. With respect to TechNexion SoMs, a MACHINE is the SoM platform, which in most cases is not baseboard-specific. However, you may want to override some elements of our predefined machines in order to customize some things such as the kernel recipe used, bootloader recipe, device tree names, and so on.

The point of all of this is to provide you with ways to create your own customized Linux distribution for your project, using your own project layer.

Creating a machine configuration for our project

Since we are basing the project on the PICO-IMX8M-MINI module, the easiest way to create a custom machine configuration file is to copy the file for the PICO-IMX8M-MINI module.

First, create a conf/machine subdirectory in our new layer (meta-scorpion):

~/edm_yocto/build-xwayland$ cd ../sources/meta-scorpion
~/edm_yocto/sources/meta-scorpion$ mkdir -p conf/machine

Now, copy the machine configuration file for module to a new file for our custom machine. We'll call this scorpion.conf:

~/edm_yocto/sources/meta-scorpion$ cp ../meta-tn-imx-bsp/conf/machine/pico-imx8mm.conf scorpion.conf

Next, we need to change some things within the new machine file. Since this is a copy of the pico-imx8mm.conf file, there are some settings that are specific to the PICO-IMX8M-MINI machine that we need to remove.

Basically, we need to remove the suffixes of variables that end in _pico-imx8mm. For example:

KERNEL_DEVICETREE_pico-imx8mm = "freescale/imx8mm-pico-pi.dtb \
                                 freescale/imx8mm-pico-pi-ili9881c.dtb \
                                 freescale/imx8mm-pico-pi-voicehat.dtb"

The KERNEL_DEVICETREE variable informs other recipes in the distribution about the device tree files that should be packaged up for the MACHINE. If we append it with the machine name (in this case _pico-imx8mm it will only pertain to images built for that specific MACHINE. If we set MACHINE=scorpion, and we use this machine configuration file as-is, there will be no device tree blobs present, which is not great because then U-boot will not proceed because then it won't be able to locate the device trees. We need to remove this.

In the scorpion.conf file, remove _pico-imx8mm from the KERNEL_DEVICETREE and UBOOT_ENTRYPOINT variable names, so they should appear like this:

KERNEL_DEVICETREE = "freescale/imx8mm-pico-pi.dtb \
                                 freescale/imx8mm-pico-pi-ili9881c.dtb \
                                 freescale/imx8mm-pico-pi-voicehat.dtb"

and:

UBOOT_ENTRYPOINT = "0x40480000"

Make sure to save the file. We are now ready to build for our custom machine.

Build the image for the scorpion machine

The easiest way to change the MACHINE name is to edit the local.conf file, where it is set by default.

~/edm_yocto/sources/meta-scorpion$ cd ../../build-xwayland
~/edm_yocto/build-xwayland$ 

Edit conf/local.conf and change the line setting the MACHINE variable from this:

MACHINE ??= 'pico-imx8mm'

To this:

MACHINE ??= 'scorpion'

Make sure to save the file.

Build the image

~/edm_yocto/build-xwayland$ bitbake imx-image-core

Once the build completes, you will have an image in tmp/deploy/images/{MACHINE}/ directory. Because we built for the scorpion MACHINE, we should now have an image in tmp/deploy/images/scorpion/.

The image files are created using the Open Embedded wictool (done for you automatically when you build an image) and have .wic.in the file name:

~/edm_yocto/build-xwayland$ ls tmp/deploy/images/scorpion/*.wic.*

You should see some files listed there.

Incidentally, if you want to know more about the wictool you can follow this link to the Yocto Project mega manual:
Creating Partitioned Images Using Wic

Feel free to write this file to the board using uuu or using U-boot's ums (USB Mass Storage) command to try it out.