How to build your OCI images with Buildpacks

Docker has become the new standard for building your application. In a Docker image, we place our source code, its dependencies, some configurations, and our application is almost ready to be deployed on our workstation or on our production, either in the cloud or on premises. For several years, Docker has been overshadowed by the open source standard OCI (Open Container Initiative). Today, it is not even necessary to use a Docker file to build our applications! Let's take a look at what Buildpacks offers in this regard, but first we need to understand what an OCI image actually is.

OCI stock

Let's take a basic Node.js application as an example:

myapp
├── package.json
└── src
    └── index.js

To maintain our application, we usually write one Docker file. It contains the necessary instructions to build an environment that will be used to run our application.

FROM node:16

WORKDIR /app

COPY package.json /app/package.json
COPY src/ /app/src


RUN npm install
CMD 'npm start'

Once built, we can inspect our image with docker inspect: (selected bits)

  (...)
  "Config": {
      "Hostname": "",
      "Domainname": "",
      "User": "",
      "AttachStdin": false,
      "AttachStdout": false,
      "AttachStderr": false,
      "Tty": false,
      "OpenStdin": false,
      "StdinOnce": false,
      "Env": [
        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
        "NODE_VERSION=16.17.0",
        "YARN_VERSION=1.22.19"
      ],
      "Cmd": [
        "/bin/sh",
        "-c",
        "npm start"
      ],
      "Image": "sha256:ca5108589bcee5007319db215f0f22048fb7b75d4d6c16e6310ef044c58218c0",
      "Volumes": null,
      "WorkingDir": "/app",
      "Entrypoint": [
          "docker-entrypoint.sh"
      ],
      "OnBuild": null,
      "Labels": null
  },
  (...)
      "Type": "layers",
      "Layers": [
          "sha256:20833a96725ec17c9ab15711400e43781a7b7fe31d23fc6c78a7cca478935d33",
          "sha256:07b905e91599cd0251bd575fb77d0918cd4f40b64fa7ea50ff82806c60e2ba61",
          "sha256:5cbe2d191b8f91d974fbba38b5ddcbd5d4882e715a9e008061ab6d2ec432ef7b",
          "sha256:47b6660a2b9bb2091a980c361a1c15b2a391c530877961877138fffc50d2f6f7",
          "sha256:1e69483976e43c657172c073712c3d741245f9141fb560db5af7704aee95114c",
          "sha256:a51886a0928017dc307b85b85c6fb3f9d727b5204ea944117ce59c4b4acc5e05",
          "sha256:ba9804f7abedf8ddea3228539ae12b90966272d6eb02fd2c57446e44d32f1b70",
          "sha256:c77311ff502e318598cc7b6c03a5bd25182f7f6f0352d9b68fad82ed7b4e8c26",
          "sha256:93a6676fffe0813e4ca03cae4769300b539b66234e95be66d86c3ac7748c1321",
          "sha256:3cf3c6f03984f8a31c0181feb75ac056fc2bd56ef8282af9a72dafd1b6bb0c41",
          "sha256:02dacaf7071cc5792c28a4cf54141b7058ee492c93f04176f8f3f090c42735eb",
          "sha256:85152f012a08f63dfaf306c01ac382c1962871bf1864b357549899ec2fa7385d",
          "sha256:8ceb0bd5afef8a5fa451f89273974732cd0c89dac2c80ff8b7855531710fbc49"
      ]
  (...)

We can see a configuration block with:

• The environmental variables
• The entrypoint command, the default command
• The work directory
• The user of the image

And a second block “Store” with a list of checksums. Each checksum corresponds to a compressed archive file (.tar.gz). All these layers applied on top of each other build a complete file system. To learn more about this topic, I invite you to read David's article.

How Docker builds an image

To understand how Docker builds an image, it is necessary to know docker commit command. This command runs on a running container. It creates an image from the state of this container.

This mechanism is used, for example, in our spring RUN npm install instruction:

• An intermediate container is started by Docker.
• In this container we start the command npm install.
• When the command is ready, Docker commits the container, the difference between the previous image makes it possible to get an extra layer and thus a new intermediate image.

Though, docker build has an important drawback: by default the build is not reproducible. Two consecutive harbor constructions, with the same Docker file, will not necessarily produce the same layers, and therefore the same checksums. What will cause this phenomenon is mostly timestamps. Every file in a standard Linux file system will have a creation date, a last modified date, and a last accessed date. The image also has a timestamp that is embedded in the image and changes the checksum. This makes it very difficult to isolate our different layers, and it is difficult to logically associate an operation in ours Docker file with a layer in our final image.

For modification of the base image (the FROM instruction, for security reasons for example), Docker necessarily launches a complete version. Uploading this update to our registry also sends all layers. Depending on the size of our image, this can lead to a lot of traffic on each of our machines that host our image and want to update.

An image is simply a stack of layers on top of each other, along with configuration files. However, Docker (and its Dockerfile) is not the only way to build an image. Buildpacks offer a different principle for building images. Buildpacks is a project incubated at Cloud Native Computing Foundation.

What is a build kit?

A buildpack is a set of executable scripts that allow you to build and launch your application.

A construction kit consists of 3 components:

• buildpack.toml: Metadata for your buildpack
• bin/detect: script that determines if the build package applies to your application
• bin/build: script that starts the build sequence for the program

Building your application means “running” buildpacks one after the other.

To do this we use a builder. A builder is an image that includes a full set of build packages, a lifecycle, and a reference to a very light run image where we will integrate our application. The pair build image/driving image is called one stack.

When we come back to our Node.js application, Buildpack will only use the information in the package.json.

{
  "name": "myapp",
  "version": "0.1",
  "main": "index.js",
  "license": "MIT",
  "dependencies": {
    "express": "^4.18.1"
  },
  "scripts": {
    "start": "node src/index.js"
  },
  "engines": {
    "node": "16"
  }
}

This is all we need to build an image containing our application with buildpacks:

• The base image (the FROM of the Dockerfile) will be the one specified in the stack
• The build package lifecycle discovers a package.json and thus starts the installation process of Node.js and dependencies.
• The version of the node will be the version specified in package.json.
• The default command will be node src/index.js because it is start command over package.json.

The only command you need to know to use buildpacks is the following:

pack build myapp --builder gcr.io/buildpacks/builder:v1

Here we use the builder provided by Google ‘gcr.io/buildpacks/builder:v1'. Other builders are available (see pack stack suggest) or you can build your own!

It is then immediately possible to start our application.

$ docker run myapp

> myapp@0.0.1 start
> node src/index.js

Example app listening on port 3000

The benefits of using build kits include:

• The absence of a Dockerfile. The developer can only concentrate on his code.
• The respect for good image building practices, in terms of security, limiting the number and size of stock. This is the responsibility of the buildpack designer, not the developer.
• Caching is built in, whether it's the binaries to be installed or the software libraries.
• Using your organization's best practices if you create your own build package (use of internal mirrors, code analysis…)
• Each layer in the final image is logically linked to the dependencies it entails.
• Each operation of a buildpack affects only a limited area (a folder with the name of the buildpack in the runtime image)
• Buildpacks make it possible to get reproducible builds with less effort.
  • The files copied to the image all have a timestamp of 0.
  • However, it is necessary that the intermediate build steps (eg compiling Java code) must be idempotent.
  • This makes it possible to start an operation that builds up the “rebase” calls.

Rebasing images with Buildpacks

Image rebasing consists in being able to replace several layers of an image, without having to modify the upper layers. This is especially relevant when you change base image of our driving picture. Let's take an example of a Node.js application (which I simplify roughly)

At build time, Buildpacks rely on the lightest possible runtime image and add layer after layer:

• A repository where the node's binaries are located
• A layer there node_modules (dependencies) of our application exist

Note that the npm binary is not needed in our runtime. It will not be included, but it is used in our build image to install node_modules which will be integrated into our driving picture.

In the event that our container is deployed in Kubernetes and an error is detected in its base image, it is necessary to update it. With Docker, this would require us to completely rebuild our image, upload our entire new image to our registry, and every Kube worker would need to download this new image.

This is not necessary with Buildpacks, only the layers related to the base image need to be uploaded. Here we can compare two OCI images, one with an ubuntu:18.04 version, the other with ubuntu:20.04. The first 3 layers (in blue) are the Ubuntu related layers. The next layers (in red) are the layers added by Buildpacks: these are identical. This behavior is possible in Docker (but complicated to set), it is by default with Buildpacks.

Restrictions

Buildpacks come with an important limitation: a buildpack can only act on a very limited area of ​​the filesystem. Libraries and binaries must be installed in the folder in the buildpack that installs them. This allows for a clear separation of the perimeter of each build package but requires more rigor. It is therefore no longer possible to just start installations via the package manager (apt, yum or apk). If it is not possible to work around this limitation, it is necessary to modify the stack image.

Conclusion

If you want to easily implement imaging best practices for your containerized applications, Buildpacks are a great choice to consider. It easily integrates with your CI/CD via projects like kpack. It may even already be integrated into your DevOps infrastructure since Buildpacks is behind it Gitlab's Auto DevOps feature.