diff --git a/doc/buildWithDocker.md b/doc/buildWithDocker.md
index 1db5e7a7..771dd774 100644
--- a/doc/buildWithDocker.md
+++ b/doc/buildWithDocker.md
@@ -1,69 +1,58 @@
 # Build the project using Docker
 There are [Docker images (Dockerfile)](../docker) containing the build environment for AMD64 (x86_64) and ARM64 architectures. These images make the build of the firmware and the generation of the DFU file for OTA quite easy, as well as preventing clashes with any other toolchains or development environments you may have installed.
 
-## Using the image from Docker Hub
-
-The image is avaiable via Docker Hub for both the amd64 and arm64v8 architectures at [pfeerick/infinitime-build](https://hub.docker.com/repository/docker/pfeerick/infinitime-build). 
-
-It can be pulled (downloaded) using the following command:
-
-```
-docker pull pfeerick/infinitime-build
-```
-
-The default `latest` tag *should* automatically identify the correct image architecture, but if for some reason Docker does not, you can specify it manually:
-
-* For AMD64 (x86_64) systems: `docker pull pfeerick/infinitime-build:amd64`
-
-* For ARM64v8 (ARM64/aarch64) systems: `docker pull pfeerick/infinitime-build:arm64v8`
-
-The Docker Hub images are built using 1000:1000 for the user id and group id. If this is different to your user or group ids (run `id -u` and `id -g` to find out what your id values are if you are unsure), you will need to override them via the `--user` parameter in order to prevent permission errors during and after compilation.
-
-The below example will run the container, setting the user and group ids automatically:
-
-```
-docker run --rm -v <project_root>:/sources --user $(id -u):$(id -g) pfeerick/infinitime-build
-```
-
-Or you can specify your user id and group id (by number, not by name) directly:
-
-```
-docker run --rm -v <project_root>:/sources --user uid_num:gid_num pfeerick/infinitime-build
-```
-
-## Build the image yourself
-Building the docker images yourself is quite easy. The following commands must be run from the root of the project.
-
-The `PUID` and `PGID` build arguments are used to set the user and group ids used in the container, meaning you will not need to specify it later unless they change for some reason. Specifying them is not mandatory, as this can be over-ridden at build time via the `--user` flag, but doing so will make the command you need to run later a bit shorter. In the below examples, they are set to your current user id and group id automatically. You can specify them manually, but they must be specified by number, not by name.
-
-If you are running on a AMD64 (x86_64) computer: 
-```
-docker image build -t infinitime-build --build-arg PUID=$(id -u) --build-arg PGID=$(id -g) docker/amd64/
-```
-
-If you are running on an ARM64 computer (tested on Raspberry Pi 4 and Pine64 Pinebook Pro):
-```
-docker image build -t infinitime-build --build-arg PUID=$(id -u) --build-arg PGID=$(id -g) docker/arm64v8/
-```
-
-This operation will take some time, as it builds a Docker image based on Ubuntu, installs some required packages, downloads the ARM toolchain, the NRF SDK, MCUBoot and adafruit-nrfutil.
+Based on Ubuntu 18.04 with the following build dependencies:
 
 When this is done, a new image named *infinitime-build* is available.
+* ARM GCC Toolchain
+* nRF SDK
+* MCUBoot
+* adafruit-nrfutil
 
 ## Run a container to build the project:
+The `infinitime-build` image contains all the dependencies you need. The default `CMD` will compile sources found in `/sources`, so you need only mount your code. 
 
-The command to run the container is essentially the same, regardless of whether you built it yourself from the dockerfiles, or are using the Docker Hub images (use `pfeerick/infinitime-build` instead of `infinitime-build` for the later):
+This example  will build the firmware, generate the MCUBoot image and generate the DFU file. Outputs will be written to **<project_root>/build/output**:
 
-```
-docker run --rm -v <project_root>:/sources infinitime-build
+```bash
+$ cd <project_root> # e.g. cd ./work/Pinetime
+$ docker run --rm -it -v $(pwd):/sources infinitime-build
 ```
 
-This will start a container (removing it when finished), build the firmware and generate the MCUBoot image and DFU file. The output of the build is stored in `<project_root>/built/output`.
+* `--rm` to delete the container after we're done. Build cache will be written out to host disk, so no worries.
+* `-it` for better interactivity with the running container. 
+    * `-i` Interactive - required if you want to send CTRL+C
+    * `-t` TTY - This lets colours work nicely etc?
+* `-v` for mounting a volume to the container
 
-Replace *<project_root>* by the path of the root of the project on your computer. For example:
+Output files (and the cmake build cache) will have `uid`/`gid` not matching your host user. To override them, use `docker run` like this:
 
-```
-docker run --rm -v /home/jf/git/PineTime:/sources infinitime-build
+```bash
+$ docker run --rm -it -v $(pwd):/sources \
+    -e USER_ID=$(id -u) -e GROUP_ID=$(id -g) \
+    infinitime-build
 ```
 
-If you encounter permission errors (due to being logged in as a different user, changed user id, running the docker hub image, etc.), see the `--user` parameter mentioned above in the Docker Hub image section to see if this resolves the issue for you.
+If you only want to build a single CMake target, you can pass it in as the first parameter to the build script. This means calling the script explicitly as it will override the `CMD`. Here's an example For `pinetime-app`:
+
+```bash
+$ docker run --rm -it -v $(pwd):/sources infinitime-build /opt/build.sh pinetime-app
+```
+
+## Build the image
+The image is not (yet) available on DockerHub, you need to build it yourself, but that is quite easy. The following commands must be run from the root of the project. This operation will take some time but, when done, a new image named *infinitime-build* is available.
+
+```bash
+$ docker image build -t infinitime-build ./docker
+```
+
+The resulting
+
+You can bake your custom USER_ID and GROUP_ID numbers in to the image, then you don't have to pass them to `docker run` every time.
+
+```bash
+docker image build -t infinitime-build \
+    --build-arg USER_ID=$(id -u) \
+    --build-arg GROUP_ID=$(id -g) \
+    ./docker
+```
diff --git a/docker/Dockerfile b/docker/Dockerfile
new file mode 100644
index 00000000..8f56356c
--- /dev/null
+++ b/docker/Dockerfile
@@ -0,0 +1,37 @@
+FROM ubuntu:18.04
+
+RUN \
+    apt-get update -qq && \
+    apt-get install -y \
+# x86_64 / generic packages
+      bash git gosu \
+      cmake make build-essential \
+      wget unzip \
+      python3 python3-pip \ 
+# aarch64 packages
+      libffi-dev libssl-dev python3-dev \ 
+    && rm -rf /var/lib/apt/lists/*;
+
+RUN pip3 install adafruit-nrfutil
+
+# build.sh knows how to compile
+COPY build.sh /opt/
+
+# Lets get each in a separate docker layer for better downloads
+# GCC
+RUN bash -c "source /opt/build.sh; GetGcc;"
+# NrfSdk
+RUN bash -c "source /opt/build.sh; GetNrfSdk;"
+# McuBoot
+RUN bash -c "source /opt/build.sh; GetMcuBoot;"
+
+# Set and arg and use it in the env for power to override at build AND runtime
+ARG USER_ID=33333
+ARG GROUP_ID=33333
+ENV USER_ID $USER_ID
+ENV GROUP_ID $GROUP_ID
+
+ENV SOURCES_DIR /sources
+COPY entrypoint.sh /opt/
+ENTRYPOINT ["/opt/entrypoint.sh"]
+CMD ["/opt/build.sh"]
diff --git a/docker/amd64/Dockerfile b/docker/amd64/Dockerfile
deleted file mode 100644
index 5f3c77ec..00000000
--- a/docker/amd64/Dockerfile
+++ /dev/null
@@ -1,34 +0,0 @@
-FROM amd64/ubuntu:18.04
-
-ARG DEBIAN_FRONTEND=noninteractive
-RUN apt-get update -qq \
-    && apt-get install -y \ 
-        build-essential \
-        cmake \
-        git \
-        make \
-        python3 \
-        python3-pip \
-        tar \
-        unzip \
-        wget \
-    && rm -rf /var/cache/apt/* /var/lib/apt/lists/*
-
-RUN wget -q https://developer.arm.com/-/media/Files/downloads/gnu-rm/9-2020q2/gcc-arm-none-eabi-9-2020-q2-update-x86_64-linux.tar.bz \
-  && tar -xjf gcc-arm-none-eabi-9-2020-q2-update-x86_64-linux.tar.bz -C /opt \
-  && rm gcc-arm-none-eabi-9-2020-q2-update-x86_64-linux.tar.bz
-
-RUN wget -q https://developer.nordicsemi.com/nRF5_SDK/nRF5_SDK_v15.x.x/nRF5_SDK_15.3.0_59ac345.zip  \
-  && unzip -q nRF5_SDK_15.3.0_59ac345.zip -d /opt/ && rm nRF5_SDK_15.3.0_59ac345.zip
-
-RUN git clone https://github.com/JuulLabs-OSS/mcuboot.git /opt/mcuboot \
-  && pip3 install -r /opt/mcuboot/scripts/requirements.txt
-
-RUN pip3 install adafruit-nrfutil
-
-ARG PUID=1000
-ARG PGID=1000
-RUN groupadd --system --gid $PGID infinitime && useradd --system --uid $PUID --gid $PGID infinitime
-
-USER infinitime:infinitime
-CMD ["/sources/docker/build.sh"]
diff --git a/docker/arm64v8/Dockerfile b/docker/arm64v8/Dockerfile
deleted file mode 100644
index cea2b837..00000000
--- a/docker/arm64v8/Dockerfile
+++ /dev/null
@@ -1,37 +0,0 @@
-FROM arm64v8/ubuntu:18.04
-
-ENV DEBIAN_FRONTEND=noninteractive
-RUN apt-get update -qq \
-    && apt-get install -y \ 
-        build-essential \
-        cmake \
-        git \
-        libffi-dev \
-        libssl-dev \
-        make \
-        python3 \
-        python3-dev \
-        python3-pip \
-        tar \
-        unzip \
-        wget \
-    && rm -rf /var/cache/apt/* /var/lib/apt/lists/*
-
-RUN wget -q https://developer.arm.com/-/media/Files/downloads/gnu-rm/9-2020q2/gcc-arm-none-eabi-9-2020-q2-update-aarch64-linux.tar.bz2 \
-  && tar -xjf gcc-arm-none-eabi-9-2020-q2-update-aarch64-linux.tar.bz2 -C /opt \
-  && rm gcc-arm-none-eabi-9-2020-q2-update-aarch64-linux.tar.bz2
-
-RUN wget -q https://developer.nordicsemi.com/nRF5_SDK/nRF5_SDK_v15.x.x/nRF5_SDK_15.3.0_59ac345.zip \
-  && unzip -q nRF5_SDK_15.3.0_59ac345.zip -d /opt/ \
-  && rm nRF5_SDK_15.3.0_59ac345.zip
-
-RUN git clone https://github.com/JuulLabs-OSS/mcuboot.git /opt/mcuboot && pip3 install -r /opt/mcuboot/scripts/requirements.txt
-
-RUN pip3 install adafruit-nrfutil
-
-ARG PUID=1000
-ARG PGID=1000
-RUN groupadd --system --gid $PGID infinitime && useradd --system --uid $PUID --gid $PGID infinitime
-
-USER infinitime:infinitime
-CMD ["/sources/docker/build.sh"]
diff --git a/docker/build.sh b/docker/build.sh
index fcb819a6..1c697d40 100755
--- a/docker/build.sh
+++ b/docker/build.sh
@@ -1,12 +1,73 @@
-#!/bin/sh
+#!/bin/bash
+(return 0 2>/dev/null) && SOURCED="true" || SOURCED="false"
 export LC_ALL=C.UTF-8
 export LANG=C.UTF-8
 set -x
+set -e
 
-mkdir /sources/build
-cd /sources/build
+# Default locations if the var isn't already set
+export TOOLS_DIR="${TOOLS_DIR:=/opt}"
+export SOURCES_DIR="${SOURCES_DIR:=/sources}"
+export BUILD_DIR="${BUILD_DIR:=$SOURCES_DIR/build}"
+export OUTPUT_DIR="${OUTPUT_DIR:=$BUILD_DIR/output}"
 
-cmake -DARM_NONE_EABI_TOOLCHAIN_PATH=/opt/gcc-arm-none-eabi-9-2020-q2-update -DNRF5_SDK_PATH=/opt/nRF5_SDK_15.3.0_59ac345 -DUSE_OPENOCD=1 ../
-make -j$(nproc)
+export BUILD_TYPE=${BUILD_TYPE:=Release}
+export GCC_ARM_VER=${GCC_ARM_VER:="gcc-arm-none-eabi-9-2020-q2-update"}
+export NRF_SDK_VER=${NRF_SDK_VER:="nRF5_SDK_15.3.0_59ac345"}
 
-sh /sources/docker/post_build.sh
+MACHINE="$(uname -m)"
+[[ "$MACHINE" == "arm64" ]] && MACHINE="aarch64"
+
+main() {
+  local target="$1"
+  [[ ! -d "$TOOLS_DIR/$GCC_ARM_VER" ]] && GetGcc
+  [[ ! -d "$TOOLS_DIR/$NRF_SDK_VER" ]] && GetNrfSdk
+  [[ ! -d "$TOOLS_DIR/mcuboot" ]] && GetMcuBoot
+
+  mkdir -p "$BUILD_DIR"
+
+  CmakeGenerate
+  CmakeBuild "$target"
+
+  if [[ "$DISABLE_POSTBUILD" != "true" ]]; then
+    source "$BUILD_DIR/post_build.sh"
+  fi
+}
+
+GetGcc() {
+  GCC_SRC="$GCC_ARM_VER-$MACHINE-linux.tar.bz"
+  wget -q https://developer.arm.com/-/media/Files/downloads/gnu-rm/9-2020q2/$GCC_SRC -O - | tar -xj -C $TOOLS_DIR/
+}
+
+GetMcuBoot() {
+  git clone https://github.com/JuulLabs-OSS/mcuboot.git "$TOOLS_DIR/mcuboot"
+  pip3 install -r "$TOOLS_DIR/mcuboot/scripts/requirements.txt"
+}
+
+GetNrfSdk() {
+  wget -q "https://developer.nordicsemi.com/nRF5_SDK/nRF5_SDK_v15.x.x/$NRF_SDK_VER.zip" -O /tmp/$NRF_SDK_VER
+  unzip -q /tmp/$NRF_SDK_VER -d "$TOOLS_DIR/"
+  rm /tmp/$NRF_SDK_VER
+}
+
+CmakeGenerate() {
+  # We can swap the CD and trailing SOURCES_DIR for -B and -S respectively
+  # once we go to newer CMake (Ubuntu 18.10 gives us CMake 3.10)
+  cd "$BUILD_DIR"
+
+  cmake -G "Unix Makefiles" \
+    -DCMAKE_BUILD_TYPE=$BUILD_TYPE \
+    -DUSE_OPENOCD=1 \
+    -DARM_NONE_EABI_TOOLCHAIN_PATH="$TOOLS_DIR/$GCC_ARM_VER" \
+    -DNRF5_SDK_PATH="$TOOLS_DIR/$NRF_SDK_VER" \
+    "$SOURCES_DIR"
+  cmake -L -N .
+}
+
+CmakeBuild() {
+  local target="$1"
+  [[ -n "$target" ]] && target="--target $target"
+  cmake --build "$BUILD_DIR" --config $BUILD_TYPE "$target" -- -j$(nproc)
+}
+
+[[ $SOURCED == "false" ]] && main "$@" || echo "Sourced!"
\ No newline at end of file
diff --git a/docker/entrypoint.sh b/docker/entrypoint.sh
new file mode 100755
index 00000000..5adb88f5
--- /dev/null
+++ b/docker/entrypoint.sh
@@ -0,0 +1,7 @@
+#!/bin/bash
+set -e
+## Create a user on-the-fly before running CMD
+## This allows us to override at runtime, allowing use of a pre-built docker image
+addgroup --gid $GROUP_ID user
+adduser --disabled-password --gecos '' --uid $USER_ID --gid $GROUP_ID user
+exec gosu user:user /bin/bash -c "$@"
\ No newline at end of file
diff --git a/docker/post_build.sh.in b/docker/post_build.sh.in
index 0665100a..414fdb40 100755
--- a/docker/post_build.sh.in
+++ b/docker/post_build.sh.in
@@ -2,15 +2,24 @@
 export LC_ALL=C.UTF-8
 export LANG=C.UTF-8
 set -x
+set -e
 
-mkdir -p /sources/build/output
-/opt/mcuboot/scripts/imgtool.py create --align 4 --version 1.0.0 --header-size 32 --slot-size 475136 --pad-header /sources/build/src/pinetime-mcuboot-app-@PROJECT_VERSION_MAJOR@.@PROJECT_VERSION_MINOR@.@PROJECT_VERSION_PATCH@.bin /sources/build/output/image-@PROJECT_VERSION_MAJOR@.@PROJECT_VERSION_MINOR@.@PROJECT_VERSION_PATCH@.bin
-adafruit-nrfutil dfu genpkg --dev-type 0x0052 --application /sources/build/output/image-@PROJECT_VERSION_MAJOR@.@PROJECT_VERSION_MINOR@.@PROJECT_VERSION_PATCH@.bin /sources/build/output/dfu-@PROJECT_VERSION_MAJOR@.@PROJECT_VERSION_MINOR@.@PROJECT_VERSION_PATCH@.zip
+export PROJECT_VERSION="@PROJECT_VERSION_MAJOR@.@PROJECT_VERSION_MINOR@.@PROJECT_VERSION_PATCH@"
 
-cp /sources/build/src/*.bin /sources/build/output/
-cp /sources/build/src/*.hex /sources/build/output/
-cp /sources/build/src/*.out /sources/build/output/
-cp /sources/build/src/*.map /sources/build/output/
-cp /sources/bootloader/bootloader-5.0.4.bin /sources/build/output/bootloader.bin
+mkdir -p "$OUTPUT_DIR"
 
+"$TOOLS_DIR"/mcuboot/scripts/imgtool.py create --version 1.0.0 \
+    --align 4 --header-size 32 --slot-size 475136 --pad-header \
+    "$BUILD_DIR/src/pinetime-mcuboot-app-$PROJECT_VERSION.bin" \
+    "$OUTPUT_DIR/image-$PROJECT_VERSION.bin"
 
+adafruit-nrfutil dfu genpkg --dev-type 0x0052 \
+    --application "$OUTPUT_DIR/image-$PROJECT_VERSION.bin" \
+    "$OUTPUT_DIR/dfu-$PROJECT_VERSION.zip"
+
+cp  "$BUILD_DIR"/src/*.bin \
+    "$BUILD_DIR"/src/*.hex \
+    "$BUILD_DIR"/src/*.out \
+    "$BUILD_DIR"/src/*.map \
+    $OUTPUT_DIR
+cp "$SOURCES_DIR"/bootloader/bootloader-5.0.4.bin $OUTPUT_DIR/bootloader.bin
\ No newline at end of file