Archive:

Build Orange Pi 5 Image with Docker

jwnhy Dec 28, 2023

Orange Pi 5 GPU

The reason I bought Orange Pi 5 is that it comes with the G610 Mali GPU, which is needed by my research project. I need to mess with its kernel space driver, which exposes a devices named /dev/mali0. So, I wish to accomplish two things at the same time.

  1. Find a cheap way to build images with GPU support (/dev/mali0)
  2. Find a way of efficiently replace the kernel (or the GPU kernel modules)

1st failed attempt: kernel module

Arm has generously open sourced their GPU drivers. So, I tried to install it (how naive I am).

After spending whole day compiling and installing it. Here is what I have found.

  1. The driver is NOT usable, as it is expected to be integrated by the vendor to their out-of-tree kernels.
  2. The driver is confusingly named. The Valhalla (Arm’s Gen 4 GPU) driver has a folder named Midgard (Arm’s Gen 3 GPU).

This basically sentenced death to this path.

2nd failed attempt: Armbian image

I went to the well-supported Armbian community, trying to see if there is support for Orange Pi 5. Given it’s a new board, the community support is not that complete. At first I tried to see if the main-stream kernel had supported this board — sadly not (no HDMI, no USB, no GPU).

After some more digging, I found that I have to use the legacy Rockchip-supported kernel, which is 5.10.160. Not a big deal as I am only interested in the GPU support. So I downloaded the Armbian Build, which by the way, really easy to use, and compiled the Armbian image.

The image turns out fine, with only one flaw. There is NO /dev/mali0. I have tried the following to get mali0.

  1. Use the .config from Orange Pi 5 images (which supports /dev/mali0)
  2. Compile individual GPU kernel modules and install

The former did nothing, while the latter hang the system. It seems that the GPU driver cannot be configured as a kernel module.

Last resort: official image

The reason why I didn’t try the official image builder in the first place are the following.

  1. It only supports Ubuntu 22.04, whilst I use Gentoo. The builder says annoying shits like “It seems you ignore documentation…”.
  2. The script is poorly maintained. The vendor just took some ancient Armbian script and modified it without any updates.
  3. It’s barely usable; it just assumes some stuff presents in your system.

I began to construct a Ubuntu 22.04 for this builder and wished it would work. To construct a usable docker image, I need to find out what packages needed for the Orange Pi build script. The image provided in docker hub is a minimal image without many useful package.

Luckily, I was able to find the needed package list in general.sh:1422 from the official builder script. With some basic setup, this is what I got.

FROM ubuntu:22.04
RUN apt-get update && apt-get install ca-certificates --reinstall -y 
RUN apt-get update
RUN DEBIAN_FRONTEND="noninteractive" apt-get install -y build-essential whiptail dialog psmisc acl uuid uuid-runtime curl gnupg gawk git tzdata sudo locales acl aptly aria2 bc binfmt-support bison btrfs-progs build-essential  ca-certificates ccache cpio cryptsetup curl debian-archive-keyring debian-keyring debootstrap device-tree-compiler dialog dirmngr dosfstools dwarves f2fs-tools fakeroot flex gawk gcc-arm-linux-gnueabihf gdisk gpg imagemagick jq kmod libbison-dev libc6-dev-armhf-cross libelf-dev libfdt-dev libfile-fcntllock-perl libfl-dev liblz4-tool libncurses-dev libpython2.7-dev libssl-dev libusb-1.0-0-dev linux-base locales lzop ncurses-base ncurses-term nfs-kernel-server ntpdate p7zip-full parted patchutils pigz pixz pkg-config pv python3-dev python3-distutils qemu-user-static rsync swig systemd-container u-boot-tools udev unzip uuid-dev wget whiptail zip zlib1g-dev fdisk
RUN locale-gen en_US.UTF-8
ENV TZ="Asia/Shanghai"
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

Issue I: docker --privilege , /sys and chroot

The first issue I met is docker-related. Since the script is not designed for docker, it requires access to system filesystems like /sys, /proc and /dev. These filesystems are not accessible inside docker due to security reasons. This leads to two symptoms.

  1. The qemu-binfmt becomes extremely slow. As we are building aarch64 images on x86 machines, we need binary-translation like qemu-binfmt to run aarch64 executables.
  2. Warning are popping out, saying “Unable to mount /sys” before chroot.

I am not sure if these symptoms are connected or not. Anyways, after referencing some materials. This issue is resolved by using docker with --privilege option.

Issue II: Handling OR in the dependencies of .deb packages

Some packages of Orange Pi 5 require patching to have certain functionalities (e.g., GPU acceleration). Thus, they are distributed in .deb formats and need manual dependency management.

The script scans such a package via dpkg-deb -I to obtain its dependencies. Then, with a sequence of sed’s, the script extracts the dependencies in a nice format. The only problem with this pipeline is that it does not handles cases like LIB_A | LIB_B. The script just irresponsibly ignore such cases, which leads to errors. So I added the following to handle the OR cases, scripts/image_helper.sh:297.

It’s not a perfect solution as it only keeps the former option between the OR, but it’s sufficient.

for element in "${dep_array[@]}"; do
    if [[ $element == *"|"* ]]; then
        # added by me to handle '|'
        package_dependencies+=($(echo $element | sed 's/|.*//g'))
	else
		if ! find_in_array "$element" "${package_dependencies[@]}"; then
			package_dependencies+=("${element}")
		fi
	fi
done

Issue III: Weird dependencies

Getting rid of the OR issue is not enough. Chromium has some weird dependencies that prevents installation. Anyways, I decided just installing them manually.

chroot $SDCARD /bin/bash -c "apt-get update; apt-get -y -qq install libxtst6 xdg-utils libpipewire-0.3-0" >> "${DEST}"/${LOG_SUBPATH}/install.log 2>&1

Issue IV: Proxy

When installing packages inside chroot, proxy should be disabled, otherwise apt often complains about 502 Bad Gateway.

Summary

The customized script I wrote is in the repo.