This is a follow up to my previous post of handy Docker commands that I always find myself having to Google. The full list of commands discussed in this and the previous post are shown below. Hope you find them useful!

• Don't require sudo to execute Docker commands
• Examining the file system of a failed Docker build
• Examining the file system of an image with an ENTRYPOINT
• Copying files from a Docker container to the host
• Copying files from a Docker image to the host
• View the space used by Docker
• Remove old docker images and containers
• Speeding up builds by minimising the Docker context
• Viewing (and minimising) the Docker context
• Bonus: Making a file executable in git
• Bonus 2: Forcing script files to keep Unix Line endings in git

View the space used by Docker

One of the slightly insidious things about Docker is the way it can silently chew up your drive space. Even more than that, it's not obvious exactly how much space it's actually using!

Luckily Docker includes a handy command, which lets you know how much space you're using, in terms of images, containers, and local volumes (essentially virtual hard drives attached to containers):

$ docker system df
TYPE                TOTAL               ACTIVE              SIZE                RECLAIMABLE
Images              47                  23                  9.919GB             6.812GB (68%)
Containers          48                  18                  98.35MB             89.8MB (91%)
Local Volumes       6                   6                   316.1MB             0B (0%)

As well as the actual space used up, this table also shows how much you could reclaim by deleting old containers, images, and volumes. In the next section, I'll show you how.

Remove old docker images and containers.

Until recently, I was manually deleting my old images and containers using the scripts in this gist, but it turns out there's a native command in Docker to cleanup - docker system prune -a.

This command removes all unused containers, volumes (and networks), as well as any unused or dangling images. What's the difference between an unused and dangling image? I think it's described well in this stack overflow answer:

An unused image means that it has not been assigned or used in a container. For example, when running docker ps -a - it will list all of your exited and currently running containers. Any images shown being used inside any of containers are a "used image".

On the other hand, a dangling image just means that you've created the new build of the image, but it wasn't given a new name. So the old images you have becomes the "dangling image". Those old image are the ones that are untagged and displays "" on its name when you run docker images.

When running docker system prune -a, it will remove both unused and dangling images. Therefore any images being used in a container, whether they have been exited or currently running, will NOT be affected. Dangling images are layers that aren't used by any tagged images. They take up space.

When you run the prune command, Docker double checks that you really mean it, and then proceeds to clean up your space. It lists out all the IDs of removed objects, and gives a little summary of everything it reclaimed (truncated for brevity):

$ docker system prune -a
WARNING! This will remove:
        - all stopped containers
        - all networks not used by at least one container
        - all images without at least one container associated to them
        - all build cache
Are you sure you want to continue? [y/N] y
Deleted Containers:
c4b642d3cdb67035278e3529e07d94574d62bce36a9330655c7b752695a54c2d
91de184f79942877c334b20eb67d661ec569224aacf65071e52527230b92932b
...
93d4a795a635ba0e614c0e0ba9855252d682e4e3290bed49a5825ca02e0b6e64
4d7f75ec610cbb1fcd1070edb05b7864b3f4b4079eb01b77e9dde63d89319e43

Deleted Images:
deleted: sha256:5bac995a88af19e91077af5221a991b901d42c1e26d58b2575e2eeb4a7d0150b
deleted: sha256:82fd2b23a0665bd64d536e74607d9319107d86e67e36a044c19d77f98fc2dfa1
...
untagged: microsoft/dotnet:2.0.3-runtime
deleted: sha256:a75caa09eb1f7d732568c5d54de42819973958589702d415202469a550ffd0ea

Total reclaimed space: 6.679GB

Be aware, if you are working on a new build using a Dockerfile, you may have dangling or unused images that you want to keep around. It's best to leave the pruning until you're at a sensible point.

Speeding up builds by minimising the Docker context

Docker is designed with two components: a client and a deamon/service. When you write docker commands, you're sending commands using the client to the Docker deamon which does all the work. The client and deamon can even be on two separate machines.

In order for the Docker deamon to build an image from a Dockerfile using docker build ., the client needs to send it the "context" in which the command should be executed. The context is essentially all the files in the directory passed to the docker build command (e.g., the current directory when you call docker build .). You can see the client sending this context when you build using a Dockerfile:

Sending build context to Docker daemon   38.5MB

For big projects, the context can get very large. This slows down the building of your Dockerfiles as you have to wait for the client to send all the files to the deamon. In an ASP.NET Core app for example, the top level directory includes a whole bunch of files that just aren't required for most Dockerfile builds - Git files, Visual Studio / Visual Studio Code files, previous bin and obj folders. All these additional files slow down the build when they are sent as part of the context.

Luckily, you can exclude files by creating a .dockerignore file in your root directory. This works like a .gitignore file, listing the directories and files that Docker should ignore when creating the context, for example:

.git
.vs
.vscode
artifacts
dist
docs
tools
**/bin/*
**/obj/*

The syntax isn't quite the same as for Git, but it's the same general idea. Depending on the size of your project, and how many extra files you have, adding a .dockerignore file can make a big difference. For this very small project, it reduced the context from 38.5MB to 2.476MB, and instead of taking 3 seconds to send the context, it's practially instantaneous. Not bad!

Viewing (and minimising) the Docker context

As shown in the last section, reducing the context is well worth the effort to speed up your builds. Unfortunately, there's no easy way to actually view the files that are part of the context.

The easiest approach I've found is described in this Stack Overflow question. Essentially, you build a basic image, and just copy all the files from the context. You can then run the container and browse the file system, to see what you've got.

The following Dockerfile builds a simple image using the common BusyBox base image, copies all the context files into the /tmp directory, and runs find as the command when run as a container.

FROM busybox
WORKDIR /tmp
COPY . .
ENTRYPOINT ["find"]

If you create a new Dockerfile in the root directory callled InspectContext.Dockerfile containing these layers, you can create an image from it using docker build and passing the -f argument. If you don't pass the -f, Docker will use the default Dockerfile file

$ docker build -f InspectContext.Dockerfile --tag inspect-context .
Sending build context to Docker daemon  2.462MB
Step 1/4 : FROM busybox
 ---> 6ad733544a63
Step 2/4 : WORKDIR /tmp
 ---> 6b1d4fad3942
Step 3/4 : COPY . .
 ---> c48db59b30d9
Step 4/4 : ENTRYPOINT find
 ---> bffa3718c9f6
Successfully built bffa3718c9f6
Successfully tagged inspect-context:latest

Once the image is built (which only takes a second or two), you can run the container. The find entrypoint will then spit out a list of all the files and folders in the /tmp directory, i.e. all the files that were part of the context.

$ docker run --rm inspect-context
.
./LICENSE
./.dockerignore
./aspnetcore-in-docker.sln
./README.md
./build.sh
./build.cake
./.gitattributes
./docker_build.sh
./Dockerfile
./.gitignore
./src
./src/AspNetCoreInDocker.Lib
./src/AspNetCoreInDocker.Lib/AspNetCoreInDocker.Lib.csproj
./src/AspNetCoreInDocker.Lib/Class1.cs
./src/AspNetCoreInDocker.Web
...

With this list of files you can tweak your .dockerignore file to keep your context as lean as possible. Alternatively, if you want to browse around the container a bit further you can override the entrypoint, for example: docker run --entrypoint sh -it --rm inspect-context

That's pretty much it for the Docker commands for this article. I'm going to finish off with a couple of commands that are somewhat related, in that they're Git commands I always find myself reaching for when working with Docker!

Bonus: Making a file executable in git

This has nothing to do with Docker specifically, but it's something I always forget when my Dockerfile uses an external build script (for example when using Cake with Docker/). Even if the file itself has executable permissions, you have to tell Git to store it as executable too:

git update-index --chmod=+x build.sh

Bonus 2: Forcing script files to keep Unix Line endings in git

If you're working on Windows, but also have scripts that will be run in Linux (for example via a mapped folder in the Linux VM), you need to be sure that the files are checked out with Linux line endings (LF instead of CRLF).

You can set the line endings Git uses when checking out files with a .gitattributes file. Typically, my file just contains * text=auto so that line endings are auto-normalised for all text files. That means my .sh files end up with CRLF line endings when I check out on Windows. You can add an extra line to the file that forces all .sh files to use LF endings, no matter which platform they're checked out on:

* text=auto
*.sh eol=lf

Summary

These are the commands I find myself using most often, but if you have any useful additions, please leave them in the comments! :)