Docker compose by default creates a network that is unreasonably big. Most compose projects have one or two containers. Five is already pretty unusual. Docker reserves over 65 thousand IP addresses for a single network. That is a lot of wasted address space. It turns out that with the docker defaults, you run out of available subnets by the time you try to run over 30 projects.

Let’s do some experiments to see this limit in action. Skip to the bottom if you just want to see the config changes. We’ll start by creating a simple docker compose project with a single container. We will use nginx for this example, but any container would do. Create a folder p1 for the project and inside create the docker-compose.yaml file with the following content.

services:
  nginx:
    image: nginx:latest

We now start it up.

$ cd p1
$ docker compose up -d
$ docker compose ps
NAME         IMAGE          COMMAND                  SERVICE  ...   PORTS
p1-nginx-1   nginx:latest   "/docker-entrypoint.…"   nginx    ...   80/tcp

We now have a running container. Let’s find out its network.

$ docker network ls
NETWORK ID     NAME         DRIVER    SCOPE
7b047718c8e0   bridge       bridge    local
47210f63e1c2   host         host      local
5180e5eccca9   none         null      local
69eb0bc90fac   p1_default   bridge    local

The p1_default is the network we are interested in. Let’s inspect it to see what the subnet is.

$ docker network inspect p1_default

[
    {
        "Name": "p1_default",
        ...
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1"
                }
            ]
        },
        ...
    }
]

This shows the subnet is 172.18.0.0/16, and we can also see this subnet in ip route.

$ ip route
...
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
172.18.0.0/16 dev br-69eb0bc90fac proto kernel scope link src 172.18.0.1

Ok, so this confirms that the network subnet is /16. The 16 refers to the number of bits out of the 32 bits of the IP address that remain fixed in this network. The remaining 16 bits can be used to assign addresses inside the subnet. So 2^16 = 65536 addresses. That is where the 65 thousand number comes from.

Now let’s go ahead and create new copies of this project until we run out of subnets.

cd .. # go back to parent
for n in $(seq 2 100); do
  # copy the p1 project into a new folder p$n
  cp -r p1 "p$n"

  # start up docker compose and bail out of the loop if it fails
  (cd "p$n" && docker compose up -d) || { echo "failed at p$n"; break; }
done

Running this you should eventually get an error like this:

 ✘ Network p31_default   Error Error response from daemon: all predefined address pools have been fully subnetted
 ✘ network:default       failed to create network p31_default: Error response from daemon: all predefined address pools have been fully subnetted
failed to create network p31_default: Error response from daemon: all predefined address pools have been fully subnetted
failed at p31

We have failed successfully at project 31! The actual number you reach might be different depending on what else is reserving network addresses on your system.

Let’s clean up those containers before we forget.

for d in p*/; do
    (cd "$d" && docker compose down);
done

Configuring smaller default network sizes

In docker’s defence, everything we covered up to this point is well documented by the docker networking docs. They even warn about this very issue and tell you how to configure smaller default network sizes.

The default pools use large subnets, which limits the number of networks you can create. You can divide base subnets into smaller pools to support more networks.

To configure the default network size to be /28 (so 16 IPs) we modify (or create) the file at /etc/docker/daemon.json so that it looks something like this.

{
  "default-address-pools": [
    { "base": "172.18.0.0/16", "size": 28 }
  ]
}

Then we need to restart the docker daemon.

$ sudo systemctl restart docker

This setup will allow us to create 4096 networks, and each can hold up to 13 containers (3 addresses are reserved for the network, gateway and broadcast addresses). Now we could re-run the brute force approach to find the new limit. But at this point, we would probably run out of memory before we reach the limit, and the cleanup would be pretty painful.