will/InfiniTime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Unify the Dockerfiles by fleshing out build.sh

Browse Source 
Script is written to handle it's own dependencies so it can be used within Docker or on the host system
This commit is contained in:
Joe Eaves 2020-12-17 13:12:06 +00:00
parent 276c8aa308
commit a7df0a0279
7 changed files with 171 additions and 139 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

97
doc/buildWithDocker.md
View File

@ -1,69 +1,58 @@
# Build the project using Docker
There are [Docker images (Dockerfile)](../docker) containing the build environment for AMD64 (x86_64) and ARM64 architectures. These images make the build of the firmware and the generation of the DFU file for OTA quite easy, as well as preventing clashes with any other toolchains or development environments you may have installed.
## Using the image from Docker Hub
The image is avaiable via Docker Hub for both the amd64 and arm64v8 architectures at [pfeerick/infinitime-build](https://hub.docker.com/repository/docker/pfeerick/infinitime-build).
It can be pulled (downloaded) using the following command:
```
docker pull pfeerick/infinitime-build
```
The default `latest` tag *should* automatically identify the correct image architecture, but if for some reason Docker does not, you can specify it manually:
* For AMD64 (x86_64) systems: `docker pull pfeerick/infinitime-build:amd64`
* For ARM64v8 (ARM64/aarch64) systems: `docker pull pfeerick/infinitime-build:arm64v8`
The Docker Hub images are built using 1000:1000 for the user id and group id. If this is different to your user or group ids (run `id -u` and `id -g` to find out what your id values are if you are unsure), you will need to override them via the `--user` parameter in order to prevent permission errors during and after compilation.
The below example will run the container, setting the user and group ids automatically:
```
docker run --rm -v <project_root>:/sources --user $(id -u):$(id -g) pfeerick/infinitime-build
```
Or you can specify your user id and group id (by number, not by name) directly:
```
docker run --rm -v <project_root>:/sources --user uid_num:gid_num pfeerick/infinitime-build
```
## Build the image yourself
Building the docker images yourself is quite easy. The following commands must be run from the root of the project.
The `PUID` and `PGID` build arguments are used to set the user and group ids used in the container, meaning you will not need to specify it later unless they change for some reason. Specifying them is not mandatory, as this can be over-ridden at build time via the `--user` flag, but doing so will make the command you need to run later a bit shorter. In the below examples, they are set to your current user id and group id automatically. You can specify them manually, but they must be specified by number, not by name.
If you are running on a AMD64 (x86_64) computer:
```
docker image build -t infinitime-build --build-arg PUID=$(id -u) --build-arg PGID=$(id -g) docker/amd64/
```
If you are running on an ARM64 computer (tested on Raspberry Pi 4 and Pine64 Pinebook Pro):
```
docker image build -t infinitime-build --build-arg PUID=$(id -u) --build-arg PGID=$(id -g) docker/arm64v8/
```
This operation will take some time, as it builds a Docker image based on Ubuntu, installs some required packages, downloads the ARM toolchain, the NRF SDK, MCUBoot and adafruit-nrfutil.
Based on Ubuntu 18.04 with the following build dependencies:
When this is done, a new image named *infinitime-build* is available.
* ARM GCC Toolchain
* nRF SDK
* MCUBoot
* adafruit-nrfutil
## Run a container to build the project:
The `infinitime-build` image contains all the dependencies you need. The default `CMD` will compile sources found in `/sources`, so you need only mount your code.
The command to run the container is essentially the same, regardless of whether you built it yourself from the dockerfiles, or are using the Docker Hub images (use `pfeerick/infinitime-build` instead of `infinitime-build` for the later):
This example will build the firmware, generate the MCUBoot image and generate the DFU file. Outputs will be written to **<project_root>/build/output**:
```
docker run --rm -v <project_root>:/sources infinitime-build
```bash
$ cd <project_root> # e.g. cd ./work/Pinetime
$ docker run --rm -it -v $(pwd):/sources infinitime-build
```
This will start a container (removing it when finished), build the firmware and generate the MCUBoot image and DFU file. The output of the build is stored in `<project_root>/built/output`.
* `--rm` to delete the container after we're done. Build cache will be written out to host disk, so no worries.
* `-it` for better interactivity with the running container.
* `-i` Interactive - required if you want to send CTRL+C
* `-t` TTY - This lets colours work nicely etc?
* `-v` for mounting a volume to the container
Replace *<project_root>* by the path of the root of the project on your computer. For example:
Output files (and the cmake build cache) will have `uid`/`gid` not matching your host user. To override them, use `docker run` like this:
```
docker run --rm -v /home/jf/git/PineTime:/sources infinitime-build
```bash
$ docker run --rm -it -v $(pwd):/sources \
-e USER_ID=$(id -u) -e GROUP_ID=$(id -g) \
infinitime-build
```
If you encounter permission errors (due to being logged in as a different user, changed user id, running the docker hub image, etc.), see the `--user` parameter mentioned above in the Docker Hub image section to see if this resolves the issue for you.
If you only want to build a single CMake target, you can pass it in as the first parameter to the build script. This means calling the script explicitly as it will override the `CMD`. Here's an example For `pinetime-app`:
```bash
$ docker run --rm -it -v $(pwd):/sources infinitime-build /opt/build.sh pinetime-app
```
## Build the image
The image is not (yet) available on DockerHub, you need to build it yourself, but that is quite easy. The following commands must be run from the root of the project. This operation will take some time but, when done, a new image named *infinitime-build* is available.
```bash
$ docker image build -t infinitime-build ./docker
```
The resulting
You can bake your custom USER_ID and GROUP_ID numbers in to the image, then you don't have to pass them to `docker run` every time.
```bash
docker image build -t infinitime-build \
--build-arg USER_ID=$(id -u) \
--build-arg GROUP_ID=$(id -g) \
./docker
```

37
docker/Dockerfile Normal file
View File

@ -0,0 +1,37 @@
FROM ubuntu:18.04
RUN \
apt-get update -qq && \
apt-get install -y \
# x86_64 / generic packages
bash git gosu \
cmake make build-essential \
wget unzip \
python3 python3-pip \
# aarch64 packages
libffi-dev libssl-dev python3-dev \
&& rm -rf /var/lib/apt/lists/*;
RUN pip3 install adafruit-nrfutil
# build.sh knows how to compile
COPY build.sh /opt/
# Lets get each in a separate docker layer for better downloads
# GCC
RUN bash -c "source /opt/build.sh; GetGcc;"
# NrfSdk
RUN bash -c "source /opt/build.sh; GetNrfSdk;"
# McuBoot
RUN bash -c "source /opt/build.sh; GetMcuBoot;"
# Set and arg and use it in the env for power to override at build AND runtime
ARG USER_ID=33333
ARG GROUP_ID=33333
ENV USER_ID $USER_ID
ENV GROUP_ID $GROUP_ID
ENV SOURCES_DIR /sources
COPY entrypoint.sh /opt/
ENTRYPOINT ["/opt/entrypoint.sh"]
CMD ["/opt/build.sh"]

34
docker/amd64/Dockerfile
View File

@ -1,34 +0,0 @@
FROM amd64/ubuntu:18.04
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update -qq \
&& apt-get install -y \
build-essential \
cmake \
git \
make \
python3 \
python3-pip \
tar \
unzip \
wget \
&& rm -rf /var/cache/apt/* /var/lib/apt/lists/*
RUN wget -q https://developer.arm.com/-/media/Files/downloads/gnu-rm/9-2020q2/gcc-arm-none-eabi-9-2020-q2-update-x86_64-linux.tar.bz \
&& tar -xjf gcc-arm-none-eabi-9-2020-q2-update-x86_64-linux.tar.bz -C /opt \
&& rm gcc-arm-none-eabi-9-2020-q2-update-x86_64-linux.tar.bz
RUN wget -q https://developer.nordicsemi.com/nRF5_SDK/nRF5_SDK_v15.x.x/nRF5_SDK_15.3.0_59ac345.zip \
&& unzip -q nRF5_SDK_15.3.0_59ac345.zip -d /opt/ && rm nRF5_SDK_15.3.0_59ac345.zip
RUN git clone https://github.com/JuulLabs-OSS/mcuboot.git /opt/mcuboot \
&& pip3 install -r /opt/mcuboot/scripts/requirements.txt
RUN pip3 install adafruit-nrfutil
ARG PUID=1000
ARG PGID=1000
RUN groupadd --system --gid $PGID infinitime && useradd --system --uid $PUID --gid $PGID infinitime
USER infinitime:infinitime
CMD ["/sources/docker/build.sh"]

37
docker/arm64v8/Dockerfile
View File

@ -1,37 +0,0 @@
FROM arm64v8/ubuntu:18.04
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update -qq \
&& apt-get install -y \
build-essential \
cmake \
git \
libffi-dev \
libssl-dev \
make \
python3 \
python3-dev \
python3-pip \
tar \
unzip \
wget \
&& rm -rf /var/cache/apt/* /var/lib/apt/lists/*
RUN wget -q https://developer.arm.com/-/media/Files/downloads/gnu-rm/9-2020q2/gcc-arm-none-eabi-9-2020-q2-update-aarch64-linux.tar.bz2 \
&& tar -xjf gcc-arm-none-eabi-9-2020-q2-update-aarch64-linux.tar.bz2 -C /opt \
&& rm gcc-arm-none-eabi-9-2020-q2-update-aarch64-linux.tar.bz2
RUN wget -q https://developer.nordicsemi.com/nRF5_SDK/nRF5_SDK_v15.x.x/nRF5_SDK_15.3.0_59ac345.zip \
&& unzip -q nRF5_SDK_15.3.0_59ac345.zip -d /opt/ \
&& rm nRF5_SDK_15.3.0_59ac345.zip
RUN git clone https://github.com/JuulLabs-OSS/mcuboot.git /opt/mcuboot && pip3 install -r /opt/mcuboot/scripts/requirements.txt
RUN pip3 install adafruit-nrfutil
ARG PUID=1000
ARG PGID=1000
RUN groupadd --system --gid $PGID infinitime && useradd --system --uid $PUID --gid $PGID infinitime
USER infinitime:infinitime
CMD ["/sources/docker/build.sh"]

73
docker/build.sh
View File

@ -1,12 +1,73 @@
#!/bin/sh
#!/bin/bash
(return 0 2>/dev/null) && SOURCED="true" || SOURCED="false"
export LC_ALL=C.UTF-8
export LANG=C.UTF-8
set -x
set -e
mkdir /sources/build
cd /sources/build
# Default locations if the var isn't already set
export TOOLS_DIR="${TOOLS_DIR:=/opt}"
export SOURCES_DIR="${SOURCES_DIR:=/sources}"
export BUILD_DIR="${BUILD_DIR:=$SOURCES_DIR/build}"
export OUTPUT_DIR="${OUTPUT_DIR:=$BUILD_DIR/output}"
cmake -DARM_NONE_EABI_TOOLCHAIN_PATH=/opt/gcc-arm-none-eabi-9-2020-q2-update -DNRF5_SDK_PATH=/opt/nRF5_SDK_15.3.0_59ac345 -DUSE_OPENOCD=1 ../
make -j$(nproc)
export BUILD_TYPE=${BUILD_TYPE:=Release}
export GCC_ARM_VER=${GCC_ARM_VER:="gcc-arm-none-eabi-9-2020-q2-update"}
export NRF_SDK_VER=${NRF_SDK_VER:="nRF5_SDK_15.3.0_59ac345"}
sh /sources/docker/post_build.sh
MACHINE="$(uname -m)"
[[ "$MACHINE" == "arm64" ]] && MACHINE="aarch64"
main() {
local target="$1"
[[ ! -d "$TOOLS_DIR/$GCC_ARM_VER" ]] && GetGcc
[[ ! -d "$TOOLS_DIR/$NRF_SDK_VER" ]] && GetNrfSdk
[[ ! -d "$TOOLS_DIR/mcuboot" ]] && GetMcuBoot
mkdir -p "$BUILD_DIR"
CmakeGenerate
CmakeBuild "$target"
if [[ "$DISABLE_POSTBUILD" != "true" ]]; then
source "$BUILD_DIR/post_build.sh"
fi
}
GetGcc() {
GCC_SRC="$GCC_ARM_VER-$MACHINE-linux.tar.bz"
wget -q https://developer.arm.com/-/media/Files/downloads/gnu-rm/9-2020q2/$GCC_SRC -O - | tar -xj -C $TOOLS_DIR/
}
GetMcuBoot() {
git clone https://github.com/JuulLabs-OSS/mcuboot.git "$TOOLS_DIR/mcuboot"
pip3 install -r "$TOOLS_DIR/mcuboot/scripts/requirements.txt"
}
GetNrfSdk() {
wget -q "https://developer.nordicsemi.com/nRF5_SDK/nRF5_SDK_v15.x.x/$NRF_SDK_VER.zip" -O /tmp/$NRF_SDK_VER
unzip -q /tmp/$NRF_SDK_VER -d "$TOOLS_DIR/"
rm /tmp/$NRF_SDK_VER
}
CmakeGenerate() {
# We can swap the CD and trailing SOURCES_DIR for -B and -S respectively
# once we go to newer CMake (Ubuntu 18.10 gives us CMake 3.10)
cd "$BUILD_DIR"
cmake -G "Unix Makefiles" \
-DCMAKE_BUILD_TYPE=$BUILD_TYPE \
-DUSE_OPENOCD=1 \
-DARM_NONE_EABI_TOOLCHAIN_PATH="$TOOLS_DIR/$GCC_ARM_VER" \
-DNRF5_SDK_PATH="$TOOLS_DIR/$NRF_SDK_VER" \
"$SOURCES_DIR"
cmake -L -N .
}
CmakeBuild() {
local target="$1"
[[ -n "$target" ]] && target="--target $target"
cmake --build "$BUILD_DIR" --config $BUILD_TYPE "$target" -- -j$(nproc)
}
[[ $SOURCED == "false" ]] && main "$@" || echo "Sourced!"

7
docker/entrypoint.sh Executable file
View File

@ -0,0 +1,7 @@
#!/bin/bash
set -e
## Create a user on-the-fly before running CMD
## This allows us to override at runtime, allowing use of a pre-built docker image
addgroup --gid $GROUP_ID user
adduser --disabled-password --gecos '' --uid $USER_ID --gid $GROUP_ID user
exec gosu user:user /bin/bash -c "$@"

25
docker/post_build.sh.in
View File

@ -2,15 +2,24 @@
export LC_ALL=C.UTF-8
export LANG=C.UTF-8
set -x
set -e
mkdir -p /sources/build/output
/opt/mcuboot/scripts/imgtool.py create --align 4 --version 1.0.0 --header-size 32 --slot-size 475136 --pad-header /sources/build/src/pinetime-mcuboot-app-@PROJECT_VERSION_MAJOR@.@PROJECT_VERSION_MINOR@.@PROJECT_VERSION_PATCH@.bin /sources/build/output/image-@PROJECT_VERSION_MAJOR@.@PROJECT_VERSION_MINOR@.@PROJECT_VERSION_PATCH@.bin
adafruit-nrfutil dfu genpkg --dev-type 0x0052 --application /sources/build/output/image-@PROJECT_VERSION_MAJOR@.@PROJECT_VERSION_MINOR@.@PROJECT_VERSION_PATCH@.bin /sources/build/output/dfu-@PROJECT_VERSION_MAJOR@.@PROJECT_VERSION_MINOR@.@PROJECT_VERSION_PATCH@.zip
export PROJECT_VERSION="@PROJECT_VERSION_MAJOR@.@PROJECT_VERSION_MINOR@.@PROJECT_VERSION_PATCH@"
cp /sources/build/src/*.bin /sources/build/output/
cp /sources/build/src/*.hex /sources/build/output/
cp /sources/build/src/*.out /sources/build/output/
cp /sources/build/src/*.map /sources/build/output/
cp /sources/bootloader/bootloader-5.0.4.bin /sources/build/output/bootloader.bin
mkdir -p "$OUTPUT_DIR"
"$TOOLS_DIR"/mcuboot/scripts/imgtool.py create --version 1.0.0 \
--align 4 --header-size 32 --slot-size 475136 --pad-header \
"$BUILD_DIR/src/pinetime-mcuboot-app-$PROJECT_VERSION.bin" \
"$OUTPUT_DIR/image-$PROJECT_VERSION.bin"
adafruit-nrfutil dfu genpkg --dev-type 0x0052 \
--application "$OUTPUT_DIR/image-$PROJECT_VERSION.bin" \
"$OUTPUT_DIR/dfu-$PROJECT_VERSION.zip"
cp "$BUILD_DIR"/src/*.bin \
"$BUILD_DIR"/src/*.hex \
"$BUILD_DIR"/src/*.out \
"$BUILD_DIR"/src/*.map \
$OUTPUT_DIR
cp "$SOURCES_DIR"/bootloader/bootloader-5.0.4.bin $OUTPUT_DIR/bootloader.bin