Last Updated on March 2, 2024 by KnownSense
Containers are the business when it comes to a stateless or non-persistent, stuff that’s here today, but gone tomorrow. And you know what? They do immutable as well where we roll one out and it never changes. Then when we do need to change things, we never touch the live container. We always create a totally new one and roll that out and replace the old one. That’s all great and it changed the world. But let’s face it. Most apps have at least some persistent data, like a database or a key value store for a changeable set of important data. Well this is where volumes come into play. They bring persistent data to the ephemeral immutable world of containers. The very high level is that individual containers can continue to be ephemeral, but the data that they create and use, that can be persistent. Now, we’ll see how to create volumes, list them, and delete them, and then we’ll see how to hook them into containers and actually use them.
Persistence: Big Picture
From a big picture perspective, containers are the go‑to solution for non‑persistent ephemeral stateless stuff. You spin up a container and it does a job. Maybe it performs an authentication or checks a stock level or tracks a shipment or something. The point is it does its job. Then it can go away and a lot of the time when doing this, they don’t create any lasting data. Magic. Well, you know what? They’re also great for immutable design patterns, where you deploy something, and then it is hands off and never to be touched again. If and when you do need to change it, instead of log on and change the live in instance, you build a new one and you deploy that.
Back to our topic though. When talking about data, there’s broadly two types, persistent and nonpersistent. Persistence is the stuff that we want to keep, like customer records, orders, audit stuff. The nonpersistent is the stuff that we don’t really care about. Since the very beginning, containers have always been an epic fit for the nonpersistent stuff. For the persistent stuff we need volumes.
Data Persistence with Docker
In the Docker world, every container gets its own nonpersistent storage. This is usually block storage that’s managed by the storage driver. Every container gets thin writeable layer that it slaps on top of the read‑only image. Well, this thin layer is an area of local storage on the Docker host that the container is running on. But the problem is, it’s tightly coupled to the life of the container. So if the container dies or maybe you delete it, then the data goes with it. Well, fortunately, persistent storage is different. Usually we call this volume storage. We have to specifically create it and then specifically copy it to a container. Now, in its most basic form, a volume also is a directory on the Docker host which gets mounted into the container at a specific mount point. But behind the scenes, this can be sitting on your fancy‑dan, high‑speed, uber‑resilient storage appliance that’s got all the bells and whistles. So as long as your external storage system’s got a Docker volume driver, it’ll just work.
That’s all fine, but it’s kind of a bit behind the scenes. What’s important to us is that volumes are decoupled from containers so they have entirely separate lifecycles. So every time we spin up a new container, docker run or whatever, each one gets its own nonpersistent graph driver storage. This is that thin writeable layer for nonpersistent stuff. Well, on Linux, it’s carved out of var/lib/docker and then the name of the storage driver. On Windows, it’s C:\ProgramData\Docker\WindowsFilter. Anyway, this lives and dies with the container. It’s ephemeral. Volumes though are different. They’re decoupled from containers so they can actually be created before a container, attached to it at runtime, and then even continue to exist after the container is deleted. They’ve even got their own docker volume subcommand. We can start, stop, update, and even delete containers and any data they write to volumes, not impacted. So, it’s possible to create a volume, mount it into a container, have the container write data to it. That container will then fail or be deleted, and the volume still exists. Then mount that volume into another or a surviving container. You can even mount a volume into multiple containers if you want to, though you do need to be careful to avoid data corruption.
Creating and Managing Volumes
To reinforce this idea that volumes are fully independent from containers, there’s a full on docker volume subcommand and it follows the common syntax
- docker volume create: This command is used to create a new volume. You specify the name of the volume as an argument.
docker volume create my_volume - docker volume ls: Lists all the volumes on the Docker host.
docker volume ls - docker volume inspect: Provides detailed information about a specific volume, including its configuration and mount point. You specify the volume name or ID as an argument.
docker volume inspect my_volume - docker volume rm: Deletes one or more volumes. You specify the volume name(s) or ID(s) as arguments.
docker volume rm my_volume - docker volume prune: Deletes all volumes that are not being used by any container. This is useful for cleaning up unused volumes.
docker volume prune - docker volume cp: Copies files or directories from a container’s filesystem to a volume. You specify the source path in the container and the destination path in the volume.
docker volume cp container_name:/path/to/source /path/to/destination - docker run -v: When running a container, you can use the
-vflag to mount a volume into the container.docker run -v my_volume:/container/path container_image
Working with Volumes
The process of utilizing Docker volumes for persistent storage within containers begins with creating a container and attaching a volume using the docker run command, which automatically creates the volume if it doesn’t exist. Data written to the volume by the container is stored on the host’s filesystem, enabling persistence across container lifecycles. Even after stopping or removing containers, the data remains intact within the volume, making volumes independent of container lifecycles. Multiple containers can share the same volume, ensuring data consistency. Management of volumes is facilitated through Docker volume subcommands, allowing for creation, inspection, and removal. Volumes can also be utilized with Docker services and in Dockerfiles. Additionally, Docker’s volume model is adaptable, permitting integration with external storage systems through plugins or drivers.
Conclusion
The page emphasizes the significance of persistent storage in containerized environments and introduces Docker volumes as the solution for maintaining data across container lifecycles. It underscores the decoupling of volumes from containers, highlighting their independent lifecycle management. Through Docker volume subcommands, users can create, manage, and utilize volumes effectively, ensuring data consistency and durability. Overall, Docker volumes provide a robust mechanism for incorporating persistent storage into containerized applications, enhancing their versatility and reliability.