Advanced Docker: how to use secrets the right way

Published 19 Oct 2020 - 5 min read

Secrets are one of the sneakiest vulnerability issues you can have in a Docker image if you don’t know how to handle them.

If you need to clone a private repository or to download a private package you must use sensitive data during your docker build, there’s no easy way around that.

In this tutorial on the advanced usage of Docker series, I’ll explain how to use a build secret in a safe way.

Tutorial on the advanced usage of Docker build secrets in a safe way

What is Buildkit

I explained last week what is the Buildkit build engine, how to set it up, and how you can use Buildkit to speed up docker build.

Never use COPY and rm

If you are dealing with secrets during your development, I’m sure the first thing you’ve tried is to COPY a file with credentials from your Dockerfile and then remove it with rm when you don’t need it anymore…

This is so wrong

because you are just deleting the file from that layer but the credentials are still in the layer above!

Unsafe code example

Let’s use this Dockerfile.

FROM ubuntu:bionic
COPY .netrc /
RUN rm .netrc

And let’s build it.

$ docker build -t unsafe . -f Dockerfile.not-safe
Sending build context to Docker daemon  4.096kB
Step 1/3 : FROM ubuntu:bionic
 ---> c14bccfdea1c
Step 2/3 : COPY .netrc /
 ---> 18d1eb74c6da
Step 3/3 : RUN rm .netrc
 ---> Running in fafd31acf728
Removing intermediate container fafd31acf728
 ---> d7d4315738a6
Successfully built d7d4315738a6
Successfully tagged unsafe:latest

Now we want to use the command docker history to list all the layers of the image.

$ docker history d7d4315738a6
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
d7d4315738a6        10 seconds ago      /bin/sh -c rm .netrc                            0B
18d1eb74c6da        14 seconds ago      /bin/sh -c #(nop) COPY file:a0fa732884be950b…   19B
c14bccfdea1c        4 weeks ago         /bin/sh -c #(nop)  CMD ["/bin/bash"]            0B
<missing>           4 weeks ago         /bin/sh -c mkdir -p /run/systemd && echo 'do…   7B
<missing>           4 weeks ago         /bin/sh -c [ -z "$(apt-get indextargets)" ]     0B
<missing>           4 weeks ago         /bin/sh -c set -xe   && echo '#!/bin/sh' > /…   745B
<missing>           4 weeks ago         /bin/sh -c #(nop) ADD file:84f8ddc4d76e1e867…   63.2MB

Here we can see the latest layer created, d7d4315738a6, but you don’t care about it. The best part is the previous layer, 18d1eb74c6da, which we can analyze deeper.

$ docker run -it 18d1eb74c6da
root@09fb719ec3dc:/# cat .netrc
my secret password

This is the deal: every layer of your image is available, including the ones with your secrets!

Think about it next time you do COPY and rm.

Buildkit to the rescue with --secret

Buildkit adds a new flag called --secret for the docker build command. You can use it to provide safely a secret to your Dockerfile at build time! Buildkit mounts the secret using tmpfs in a temporary file located in /run/secrets that we can use to access a secret in the Dockerfile.

Using this feature we are sure that no secrets will remain in the image!

Safe code example

Dockerfile

Let’s use the following Dockerfile

# syntax = docker/dockerfile:1.0-experimental

FROM ubuntu:bionic

RUN --mount=type=secret,id=mynetrc,dst=/.netrc cat /.netrc
RUN cat /.netrc

The first thing to notice is # syntax = docker/dockerfile:1.0-experimental, we tell Docker to use the new syntax to exploit the new Buildkit functionality. Then, with the first RUN command, the magic happens. We tell Docker to mount a secret with the id mynetrc to the destination /.netrc and in the same line we execute the cat command just for the sake of the example. Then we RUN again the cat command on the same file.

Build command

To build our Dockerfile this is the command:

$ DOCKER_BUILDKIT=1 docker build --secret id=mynetrc,src=.netrc --progress=plain --no-cache -f Dockerfile.safe -t safe .

You can notice here the flag --secret which tells Docker the secret name and location. We also need to set DOCKER_BUILDKIT=1 to use the Buildkit build engine.

OK, let’s build it.

$ DOCKER_BUILDKIT=1 docker build --secret id=mynetrc,src=.netrc --progress=plain --no-cache -f Dockerfile.safe -t safe .

#...

#8 [1/3] FROM docker.io/library/ubuntu:bionic
#8 CACHED

#9 [2/3] RUN --mount=type=secret,id=mynetrc,dst=/.netrc cat /.netrc
#9 0.808 my secret password
#9 DONE 1.7s

#10 [3/3] RUN cat /.netrc
#10 DONE 2.0s

#11 exporting to image
#11 exporting layers
#11 exporting layers 0.7s done
#11 writing image sha256:b86ed6e0585c2f2e5cb14796b896dae0004f75004ccece0949a3de0ca600b113 0.0s done
#11 naming to docker.io/library/safe 0.0s done
#11 DONE 1.0s

As you can see, in the RUN --mount=type=secret,id=mynetrc,dst=/.netrc cat /.netrc we can access the content of the file, instead on the following RUN there’s no output.

The .netrc file, in fact, is present in the final layer of the image but it’s empty.

Let’s use the command docker history to list all the layers of this new image.

$ docker history b86ed6e0585c
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
b86ed6e0585c        5 hours ago         RUN /bin/sh -c cat /.netrc # buildkit           0B                  buildkit.dockerfile.v0
<missing>           5 hours ago         RUN /bin/sh -c cat /.netrc # buildkit           0B                  buildkit.dockerfile.v0
<missing>           4 weeks ago         /bin/sh -c #(nop)  CMD ["/bin/bash"]            0B                  
<missing>           4 weeks ago         /bin/sh -c mkdir -p /run/systemd && echo 'do…   7B                  
<missing>           4 weeks ago         /bin/sh -c [ -z "$(apt-get indextargets)" ]     0B                  
<missing>           4 weeks ago         /bin/sh -c set -xe   && echo '#!/bin/sh' > /…   745B                
<missing>           4 weeks ago         /bin/sh -c #(nop) ADD file:84f8ddc4d76e1e867…   63.2MB

As you can see, it’s not possible now to load an older layer to read the secret.

This is it!

I hope this was useful for you, now go and refactor your old Dockerfile!

Reach me on Twitter @gasparevitta and let me know how you use manage secrets.

You can find the code snippets on Github.

Get emails about new articles!

I write about Continuous Integration, Continuous Deployment, testing, and other cool stuff.
Gaspare Vitta on Twitter