Docker in Docker vs Docker Outside of Docker

I wrote a tutorial back in April 2021 titled Jenkins: Docker-in-Docker. Back then I had some idea that the demonstration should not contain the term "Docker in Docker" as the entire deed was a demonstration of Docker daemon being outside (on the host) and only the Docker client being inside the container.

I couldn't give it the proper term DooD until I encountered the term while working on a crazy Jenkins project.

Docker in Docker

In DinD, you run Docker daemon inside a container using Privileged mode.

This can be quickly demonstrated as follows:

docker pull docker:20.10.12-dind
docker run --privileged --name docker-daemon -d docker:20.10.12-dind

Lets get a shell inside the Docker container docker-daemon:

docker exec -it docker-daemon /bin/sh

Inside the container you can see using ps command that dockerd and containerd are running:

PID   USER     TIME  COMMAND
    1 root      0:00 docker-init -- dockerd --host=unix:///var/run/docker.sock --host=tcp://0.0.0.0:2376 --tlsverify --tlscacert /certs/server/ca.pem --tlscert /certs/server/cert.pem --tlskey /certs/server/key.pem
   62 root      0:00 dockerd --host=unix:///var/run/docker.sock --host=tcp://0.0.0.0:2376 --tlsverify --tlscacert /certs/server/ca.pem --tlscert /certs/server/cert.pem --tlskey /certs/server/key.pem
   75 root      0:00 containerd --config /var/run/docker/containerd/containerd.toml --log-level info
  182 root      0:00 /bin/sh
  188 root      0:00 ps

You can pull docker image(s) inside the container use the image(s) to run commands:

docker pull amazon/aws-cli
docker run --rm -it amazon/aws-cli:latest --version

With the above docker run command you can see the AWS CLI version:

aws-cli/2.4.12 Python/3.8.8 Linux/5.10.0-10-amd64 docker/x86_64.amzn.2 prompt/off

The AWS CLI image pulled inside the container is not available to the host Docker daemon. This, along with the fact that the AWS CLI container ran inside another container, is why this is called Docker in Docker. You are running Docker container(s) using Docker inside a Docker container. Its basically Inception.

However in practice, this is not how DinD container should be used. We must attach a non-privileged container so that we can run our docker commands in there safely. Lets use the docker-compose.yml below:

version: "3.9"
services:
  docker-daemon:
    container_name: "docker-daemon"
    image: "docker:20.10.12-dind"
    volumes:
      - "socket:/var/run"
    privileged: true
  docker:
    container_name: "docker-client"
    image: "docker:20.10.12"
    command: "sleep 99d"
    volumes:
      - "socket:/var/run"
    depends_on:
      - "docker-daemon"

volumes:
  socket:

docker-compose up -d

I wrote this docker-compose snippet file from Kubernetes Pod manifest in the git repository for Kubernetes Jenkins plugin.

In this formation, Docker daemon and Docker CLI are run in 2 separate containers. They share the same volume where Docker daemon places its unix socket file. By default, Docker CLI uses the unix socket to communicate with the daemon and perform user wanted actions. Lets get a shell inside the client container:

docker exec -it docker-client /bin/sh

You can see that the client container is up due to the sleep command here using ps.

PID   USER     TIME  COMMAND
    1 root      0:00 sleep 99d
   19 root      0:00 /bin/sh
   37 root      0:00 ps

And you can run docker command here:

docker pull amazon/aws-cli
docker run --rm -it amazon/aws-cli:latest --version

With the above docker run command you can see the AWS CLI version:

aws-cli/2.4.12 Python/3.8.8 Linux/5.10.0-10-amd64 docker/x86_64.amzn.2 prompt/off

After you are done, you can remove the containers and it will also remove any container images downloaded under docker-daemon container:

docker-compose down

This formation can be used in multiple scenarios including but not limited to:

• Building Docker container images (although Kaniko is a good way to do it without requiring the Docker daemon).
• Running docker-compose based testing workflows under your CI/CD pipelines.

You might want to know that the Docker images (docker:20.10.12-dind and docker:20.10.12) referenced in the above mentioned commands and the docker-compose snippet are built atop Alpine Linux.

Docker Outside of Docker

Since the mechanics of this are already discussed in my other blog post, I don't think I need to discuss it any further.

For me, DooD has one big disadvantage. All the images are download on the host and are stuck there. And if you use DooD to build images then those built images are stuck there and they keep consuming the host storage, which is not ideal. Hence in DooD there will be a lot of images left on the host that you would not actually be using. You'll have to create a separate script to remove them all.

Tagged as: Linux Docker Docker-Compose Containers DinD DooD Kubernetes K8s