Where is docker volumes in windows

Your volume directory is /var/lib/docker/volumes/blog_postgres-data/_data, and /var/lib/docker usually mounted in C:\Users\Public\Documents\Hyper-V\Virtual hard disks. Anyway you can check it out by looking in Docker settings.

You can refer to these docs for info on how to share drives with Docker on Windows.

BTW, Source is the location on the host and Destination is the location inside the container in the following output:

"Mounts": [
{
    "Name": "fac362...80535",
    "Source": "/var/lib/docker/volumes/fac362...80535/_data",
    "Destination": "/webapp",
    "Driver": "local",
    "Mode": "",
    "RW": true,
    "Propagation": ""
}
]

Updated to answer questions in the comment:

My main curiosity here is that sharing images etc is great but how do I share my data?

Actually volume is designed for this purpose (manage data in Docker container). The data in a volume is persisted on the host FS and isolated from the life-cycle of a Docker container/image. You can share your data in a volume by:

Mount Docker volume to host and reuse it

docker run -v /path/on/host:/path/inside/container image

Then all your data will persist in /path/on/host; you could back it up, copy it to another machine, and re-run your container with the same volume.
Create and mount a data container.

Create a data container: docker create -v /dbdata --name dbstore training/postgres /bin/true

Run other containers based on this container using --volumes-from: docker run -d --volumes-from dbstore --name db1 training/postgres, then all data generated by db1 will persist in the volume of container dbstore.

For more information you could refer to the official Docker volumes docs.

Simply speaking, volumes is just a directory on your host with all your container data, so you could use any method you used before to backup/share your data.

can I push a volume to docker-hub like I do with images?

No. A Docker image is something you can push to a Docker hub (a.k.a. ‘registry’); but data is not. You could backup/persist/share your data with any method you like, but pushing data to a Docker registry to share it does not make any sense.

can I make backups etc?

Yes, as posted above

Источник

You can refer to these docs for info on how to share drives with Docker on Windows.

BTW, Source is the location on the host and Destination is the location inside the container in the following output:

"Mounts": [
{
    "Name": "fac362...80535",
    "Source": "/var/lib/docker/volumes/fac362...80535/_data",
    "Destination": "/webapp",
    "Driver": "local",
    "Mode": "",
    "RW": true,
    "Propagation": ""
}
]

Updated to answer questions in the comment:

My main curiosity here is that sharing images etc is great but how do I share my data?

Mount Docker volume to host and reuse it

docker run -v /path/on/host:/path/inside/container image

Then all your data will persist in /path/on/host; you could back it up, copy it to another machine, and re-run your container with the same volume.
Create and mount a data container.

Create a data container: docker create -v /dbdata --name dbstore training/postgres /bin/true

Run other containers based on this container using --volumes-from: docker run -d --volumes-from dbstore --name db1 training/postgres, then all data generated by db1 will persist in the volume of container dbstore.

For more information you could refer to the official Docker volumes docs.

Simply speaking, volumes is just a directory on your host with all your container data, so you could use any method you used before to backup/share your data.

can I push a volume to docker-hub like I do with images?

can I make backups etc?

Yes, as posted above

Источник

Problem Description:

I’m trying to learn docker at the moment and I’m getting confused about where data volumes actually exist.

I’m using Docker Desktop for Windows. (Windows 10)

In the docs they say that running docker inspect on the object will give you the source:https://docs.docker.com/engine/tutorials/dockervolumes/#locating-a-volume

$ docker inspect web

"Mounts": [
    {
        "Name": "fac362...80535",
        "Source": "/var/lib/docker/volumes/fac362...80535/_data",
        "Destination": "/webapp",
        "Driver": "local",
        "Mode": "",
        "RW": true,
        "Propagation": ""
    }
]

however I don’t see this, I get the following:

$ docker inspect blog_postgres-data
[
    {
        "Driver": "local",
        "Labels": null,
        "Mountpoint": "/var/lib/docker/volumes/blog_postgres-data/_data",
        "Name": "blog_postgres-data",
        "Options": {},
        "Scope": "local"
    }
]

Can anyone help me? I just want to know where my data volume actually exists is it on my host machine? If so how can i get the path to it?

Solution – 1

Each container has its own filesystem which is independent from the host filesystem. If you run your container with the -v flag you can mount volumes so that the host and container see the same data (as in docker run -v hostFolder:containerFolder).

The first output you printed describes such a mounted volume (hence mounts) where /var/lib/docker/volumes/fac362...80535/_data (host) is mounted to /webapp (container).

I assume you did not use -v hence the folder is not mounted and only accessible in the container filesystem which you can find in /var/lib/docker/volumes/blog_postgres-data/_data. This data will be deleted if you remove the container (docker -rm) so it might be a good idea to mount the folder.

As to the question where you can access this data from windows. As far as I know, docker for windows uses the bash subsystem in Windows 10. I would try to run bash for windows10 and go to that folder or find out how to access the linux folders from windows 10. Check this page for a FAQ on the linux subsystem in windows 10.

Update: You can also use docker cp to copy files between host and container.

Solution – 2

Your volume directory is /var/lib/docker/volumes/blog_postgres-data/_data, and /var/lib/docker usually mounted in C:UsersPublicDocumentsHyper-VVirtual hard disks. Anyway you can check it out by looking in Docker settings.

You can refer to these docs for info on how to share drives with Docker on Windows.

BTW, Source is the location on the host and Destination is the location inside the container in the following output:

"Mounts": [
{
    "Name": "fac362...80535",
    "Source": "/var/lib/docker/volumes/fac362...80535/_data",
    "Destination": "/webapp",
    "Driver": "local",
    "Mode": "",
    "RW": true,
    "Propagation": ""
}
]

Updated to answer questions in the comment:

My main curiosity here is that sharing images etc is great but how do I share my data?

Mount Docker volume to host and reuse it

docker run -v /path/on/host:/path/inside/container image

Then all your data will persist in /path/on/host; you could back it up, copy it to another machine, and re-run your container with the same volume.
Create and mount a data container.

Create a data container: docker create -v /dbdata --name dbstore training/postgres /bin/true

Run other containers based on this container using --volumes-from: docker run -d --volumes-from dbstore --name db1 training/postgres, then all data generated by db1 will persist in the volume of container dbstore.

For more information you could refer to the official Docker volumes docs.

Simply speaking, volumes is just a directory on your host with all your container data, so you could use any method you used before to backup/share your data.

can I push a volume to docker-hub like I do with images?

can I make backups etc?

Yes, as posted above 🙂

Solution – 3

Mounting any NTFS based directories did not work for my purpose (MongoDB – as far as I’m aware it is also the case for Redis and CouchDB at least): NTFS permissions did not allow necessary access for such DBs running in containers. The following is a setup with named volumes on HyperV.

The following approach starts an ssh server within a service, setup with docker-compse such that it automatically starts up and uses public key encryption between host and container for authorization. This way, data can be uploaded/downloaded via scp or sftp.

The full docker-compose.yml for a webapp + mongodb is below, together with some documentation on how to use ssh service:

version: '3'
services:
  foo:
    build: .
    image: localhost.localdomain/${repository_name}:${tag}
    container_name: ${container_name}
    ports:
      - "3333:3333"
    links:
      - mongodb-foo
    depends_on:
      - mongodb-foo
      - sshd
    volumes:
      - "${host_log_directory}:/var/log/app"

  mongodb-foo:
    container_name: mongodb-${repository_name}
    image: "mongo:3.4-jessie"
    volumes:
      - mongodata-foo:/data/db
    expose:
      - '27017'

  #since mongo data on Windows only works within HyperV virtual disk (as of 2019-4-3), the following allows upload/download of mongo data
  #setup: you need to copy your ~/.ssh/id_rsa.pub into $DOCKER_DATA_DIR/.ssh/id_rsa.pub, then run this service again
  #download (all mongo data): scp -r -P 2222 user@localhost:/data/mongodb [target-dir within /c/]
  #upload (all mongo data): scp -r -P 2222 [source-dir within /c/] user@localhost:/data/mongodb
  sshd:
    image: maltyxx/sshd
    volumes:
        - mongodata-foo:/data/mongodb
        - $DOCKER_DATA_DIR/.ssh/id_rsa.pub:/home/user/.ssh/keys/id_rsa.pub:ro
    ports:
        - "2222:22"
    command: user::1001

#please note: using a named volume like this for mongo is necessary on Windows rather than mounting an NTFS directory.
#mongodb (and probably most other databases) are not compatible with windows native data directories due ot permissions issues.
#this means that there is no direct access to this data, it needs to be dumped elsewhere if you want to reimport something.
#it will however be persisted as long as you don't delete the HyperV virtual drive that docker host is using.
#on Linux and Docker for Mac it is not an issue, named volumes are directly accessible from host.
volumes:
  mongodata-foo:

this is unrelated, but for a fully working example, before any docker-compose call the following script needs to be run:

#!/usr/bin/env bash
set -o errexit
set -o pipefail
set -o nounset

working_directory="$(pwd)"
host_repo_dir="${working_directory}"
repository_name="$(basename ${working_directory})"
branch_name="$(git rev-parse --abbrev-ref HEAD)"
container_name="${repository_name}-${branch_name}"
host_log_directory="${DOCKER_DATA_DIR}/log/${repository_name}"
tag="${branch_name}"

export host_repo_dir
export repository_name
export container_name
export tag
export host_log_directory

Update: Please note that you can also just use docker cp nowadays, so the sshd container outlined above is probably not necessary anymore, except if you need remote access to the file system running in a container under a Windows host.

Solution – 4

When running linux based containers on a windows host, the actual volumes will be stored within the linux VM and will not be available on the host’s fs, otherwise windows running on windows => C:ProgramDataDockervolumes

Also docker inspect <container_id> will list the container configuration, under Mounts section see more details about the persistence layer.

Update:
Not applicable for Docker running on WSL.

Solution – 5

If you have wsl2 enabled, u can find it in file explorer under \wsl$docker-desktopmnthostwsldocker-desktop-datadatadocker

Solution – 6

I have found that my setup of Docker with WSL 2 (Ubuntu 20.04) uses this location at Windows 10:

C:UsersUsernameAppDataLocalDockerwsldataext4.vhdx

Where Username is your username.

Solution – 7

I am Windows + WSL 2 (Ubuntu 18.04).

Type in the Windows file explorer :

For Docker version 20.10.+ : \wsl$docker-desktop-datadatadockervolumes
For Docker Engine v19.03: \wsl$docker-desktop-dataversion-pack-datacommunitydockervolumes

You will have one directory per volume.

Solution – 8

If you’re on windows and use Docker For Windows then Docker works via VM (MobyLinuxVM). Your volumes (as everting else) are in this VM! It is how to find them:

# get a privileged container with access to Docker daemon
docker run --privileged -it --rm -v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/usr/bin/docker alpine sh

# in second power-shell run a container with full root access to MobyLinuxVM
docker run --net=host --ipc=host --uts=host --pid=host -it --security-opt=seccomp=unconfined --privileged --rm -v /:/host alpine /bin/sh

# switch to host FS
chroot /host

# and then go to the volume you asked for
cd /var/lib/docker/volumes/YOUR_VOLUME_NAME/_data

Solution – 9

If you’re searching where the data is actually located when you put a volume that is pointing to the docker «vm» like here:

version: '3.0'
services:
  mysql-server:
    image: mysql:latest
    container_name: mysql-server
    restart: always
    ports:
      - 3306:3306
    volumes:
      - /opt/docker/mysql/data:/var/lib/mysql

The "/opt/docker/mysql/data" or just the / is located in \wsl$docker-desktopmntversion-packcontainersservicesdockerrootfs

Hope it’s helping 🙂

Solution – 10

For Windows 10 + WSL 2 (Ubuntu 20.04), Docker version 20.10.2, build 2291f61

Docker artifacts can be found in

DOCKER_ARTIFACTS == \wsl$docker-desktop-dataversion-pack-datacommunitydocker

Data volumes can be found in

enter image description here

DOCKER_ARTIFACTSvolumes[VOLUME_ID]_data

enter image description here

Solution – 11

you can find the volume associated with host on below path for Docker Desktop(Windows)

\wsl$docker-desktop-dataversion-pack-datacommunitydockervolumes

Solution – 12

\wsl$docker-desktop-dataversion-pack-datacommunitydockervolumes

Worked for me as well (Windows 10 Home), great stuff.

Solution – 13

In my case, i install docker-desktop on wsl2, windows 10 home. i find my image files in

\wsl$docker-desktop-dataversion-pack-datacommunitydockeroverlay2
\wsl$docker-desktop-dataversion-pack-datacommunitydocker

Containers, images volumes infos are all there.

All image files are stored there, and have been seperated into several folders with long string names. When i look into every folder, i can find all the real image files in «diff» folders.

Although the terminal show the path «var/lib/docker», but the folder doesn’t exsit and the actual files are not stored there. i think there is no error, the «var/lib/docker» is just linked or mapped to the real folder, kind like that

Solution – 14

If your using windows, your docker files (in this case your volumes) exist on a virtual machine that docker uses for windows either Hyper-V or WSL. However if you need to access those files, you can copy your container files and store them locally on your machine and access the data this way.

docker cp container_Id_Here:/var/lib/mysql path_To_Your_Local_Machine_Here

Solution – 15

You can find it as follows, see the attached image:

enter image description here

Solution – 16

For me I found my volumes in

\wsl$docker-desktop-datadatadockervolumes

Using WSL2 and Windows 21H1

Источник

Update! From Windows 1809 onwards this is no longer an issue!

See 6 Things You Can Do with Docker in Windows Server 2019 That You Couldn’t Do in Windows Server 2016

You use Docker volumes to store state outside of containers, so your data survives when you replace the container to update your app. Docker uses symbolic links to give the volume a friendly path inside the container, like C:\data. Some application runtimes try to follow the friendly path to the real location — which is actually outside the container — and get themselves into trouble.

This issue may not affect all application runtimes, but I have seen it with Windows Docker containers running Java, Node JS, Go, PHP and .NET Framework apps. So it’s pretty widespread.

You can avoid that issue by using a mapped drive (say G:\) inside the container. Your app writes to the G drive and the runtime happily lets the Windows filesystem take care of actually finding the location, which happens to be a symlink to a directory on the Docker host.

Filesystems in Docker Containers

An application running in a container sees a complete filesystem, and the process can read and write any files it has access to. In a Windows Docker container the filesystem consists of a single C drive, and you’ll see all the usual file paths in there — like C:\Program Files and C:\inetpub. In reality the C drive is composed of many parts, which Docker assembles into a virtual filesystem.

It’s important to understand this. It’s the basis for how images are shared between multiple containers, and it’s the reason why data stored in a container is lost when the container is removed. The virtual filesystem the container sees is built up of many image layers which are read-only and shared, and a final writeable layer which is unique to the container:

When processes inside the container modify files from read-only layers, they’re actually copied into the writeable layer. That layer stores the modified version and hides the original. The underlying file in the read-only layer is unchanged, so images don’t get modified when containers make changes.

Removing a container removes its writeable layer and all the data in it, so that’s not the place to store data if you run a stateful application in a container. You can store state in a volume, which is a separate storage location that one or more containers can access, and has a separate lifecycle to the container:

Storing State in Docker Volumes

Using volumes is how you store data in a Dockerized application, so it survives beyond the life of a container. You run your database container with a volume for the data files. When you replace your container from a new image (to deploy a Windows update or a schema change), you use the same volume, and the new container has all the data from the original container.

The SQL Server Docker lab on GitHub walks you through an example of this.

You define volumes in the Dockerfile, specifying the destination path where the volume is presented to the container. Here’s a simple example which stores IIS logs in a volume:

#escape=`
FROM microsoft/iis  
VOLUME C:\inetpub\logs

You can build an image from that Dockerfile and run it in a container. When you run docker container inspect you will see that there is a mount point listed for the volume:

"Mounts": [
            {
                "Type": "volume",
                "Name": "cfc1ab55dbf6e925a1705673ff9f202d0ee2157dcd199c02111813b05ddddf22",
                "Source": "C:\\ProgramData\\docker\\volumes\\cfc1ab55dbf6e925a1705673ff9f202d0ee2157dcd199c02111813b05ddddf22\\_data",
                "Destination": "C:\\inetpub\\logs",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
        ]

The source location of the mount shows the physical path on the Docker host where the files for the volume are written — in C:\ProgramData\docker\volumes. When IIS writes logs from the container in C:\Inetpub\logs, they’re actually written to the directory in C:\ProgramData\docker\volumes on the host.

The destination path for a volume must be a new folder, or an existing empty folder. Docker on Windows is different from linux in that respect, you can’t use a destination folder which already contains data from the image, and you can’t use a single file as a destination.

Docker surfaces the destination directory for the volume as a symbolic link (symlink) inside the container, and that’s where the trouble begins.

Symlink Directories

Symbolic links have been a part of the Windows filesystem for a long time, but they’re nowhere near as popluar as they are in Linux. A symlink is just like an alias, which abstracts the physical location of a file or directory. Like all abstractions, it lets you work at a higher level and ignore the implementation details.

In Linux it’s common to install software to a folder which contains the version name — like /opt/hbase-1.2.3 and then create a sylmink to that directory, with a name that removes the version number — /opt/hbase. In all your scripts and shortcuts you use the symlink. When you upgrade the software, you change the symlink to point to the new version and you don’t need to change anything else. You can also leave the old version in place and rollback by changing the symlink.

You can do the same in Windows, but it’s much less common. The symlink mechanism is how Docker volumes work in Windows. If you docker container exec into a running container and look at the volume directory, you’ll it listed as a symlink directory (SYMLINKD) with a strange path:

c:\>dir C:\inetpub  
 Volume in drive C has no label.
 Volume Serial Number is 90D3-C0CE

 Directory of C:\inetpub

06/30/2017  10:08 AM    <DIR>          .  
06/30/2017  10:08 AM    <DIR>          ..  
01/18/2017  07:43 PM    <DIR>          custerr  
01/18/2017  07:43 PM    <DIR>          history  
01/18/2017  07:43 PM    <SYMLINKD>     logs [\\?\ContainerMappedDirectories\8305589A-2E5D...]  
06/30/2017  10:08 AM    <DIR>          temp  
01/18/2017  07:43 PM    <DIR>          wwwroot

The logs directory is actually a symlink directory, and it points to the path \\?\ContainerMappedDirectories\8305589A-2E5D... The Windows filesystem understands that symlink, so if apps write directly to the logs folder, Windows writes to the symlink directory, which is actually the Docker volume on the host.

The trouble really begins when you configure your app to use a volume, and the application runtime tries to follow the symlink. Runtimes like Go, Java, PHP, NodeJS and even .NET will do this — they resolve the symlink to get the real directory and try to write to the real path. When the «real» path starts with \\?\ContainerMappedDirectories\, the runtime can’t work with it and the write fails. It might raise an exception, or it might just silently fail to write data. Neither of which is much good for your stateful app.

DOS Devices to the Rescue

The solution — as always — is to introduce another layer of abstraction, so the app runtime doesn’t directly use the symlink directory. In the Dockerfile you can create a drive mapping to the volume directory, and configure the app to write to the drive. The runtime just sees a drive as the target and doesn’t try to do anything special — it writes the data, and Windows takes care of putting it in the right place.

I use the G drive in my Dockerfiles, just to distance it from the C drive. Ordinarily you use the subst utility to create a mapped drive, but that doesn’t create a map which persists between sessions. Instead you need to write a registry entry in your Dockerfile to permanently set up the mapped drive:

VOLUME C:\data

RUN Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\DOS Devices' -Name 'G:' -Value "\??\C:\data" -Type String;

This creates a fake G drive which maps to the volume directory C:\data. Then you can configure your app to write to the G drive and it won’t realise the target is a symlink, so it won’t try to resolve the path and it will write correctly.

I use this technique in these Jenkins and Bonobo Dockerfiles, where I also set up the G drive as the target in the app configuration.

How you configure the storage target depends on the app. Jenkins uses an environment variable, which is very easy. Bonobo uses Web.config, which means running some XML updates with PowerShell in the Dockerfile. This technique means you need to mentally map the fake G drive to a real Docker volume, but it works with all the apps I’ve tried, and it also works with volume mounts.

Mounting Volumes

Docker volumes on Windows are always created in the path of the graph driver, which is where Docker stores all image layers, writeable container layers and volumes. By default the root of the graph driver in Windows is C:\ProgramData\docker, but you can mount a volume to a specific directory when you run a container.

I have a server with a single SSD for the C drive, which is where my Docker graph is stored. I get fast access to image layers at the cost of zero redundancy, but that’s fine because I can always pull images again if the disk fails. For my application data, I want to use the E drive which is a RAID array of larger but slower spinning disks.

When I run my local Git server and Jenkins server in Docker containers I use a volume mount, pointing the Docker volume in the container to a location on my RAID array:

docker container run -v E:\bonobo:C:\data sixeyed/bonobo

Actually I use a compose file for my services, but that’s for a different post.

So now there are multiple mappings from the G drive the app uses to the Docker volume, and the underlying storage location:

G:\ -> C:\data -> \\?\ContainerMappedDirectories\xyz -> E:\bonobo

Book Plug

I cover volumes — and everything else to do with Docker on Windows — in my book Docker on Windows, which is out now.

If you’re not into technical books, all the code samples are on GitHub: sixeyed/docker-on-windows and every sample has a Docker image on the Hub: dockeronwindows.

Use the G Drive For Now

I’ve hit this problem with lots of different app runtimes, so I’ve started to do this as the norm with stateful applications. It saves a lot of time to configure the G drive first, and ensure the app is writing state to the G drive, instead of chasing down issues later.

The root of the problem actually seems to be a change in the file descriptor for symlink directories in Windows Server 2016. Issues have been logged with some of the application runtimes to work correctly with the symlink (like in Go and in Java), but until they’re fixed the G drive solution is the most robust that I’ve found.

It would be nice if the image format supported this, so you could write VOLUME G: in the Dockerfile and hide all this away. But this is a Windows-specific issue and Docker is a platform that works in the same way across multiple operating systems. Drive letters don’t mean anything in Linux so I suspect we’ll need to use this workaround for a while.

Источник

Locating Data Volumes In Docker Desktop (Windows) With Code Examples

With this article, we will examine several different instances of how to solve the Locating Data Volumes In Docker Desktop (Windows) problem.

\\wsl$\docker-desktop-data\version-pack-data\community\docker\volumes\

By examining various real-world cases, we’ve shown how to fix the Locating Data Volumes In Docker Desktop (Windows) bug.

Where are docker desktop volumes stored Windows?

Volumes are stored in a part of the host filesystem which is managed by Docker ( /var/lib/docker/volumes/ on Linux).

How do I find docker volume files?

View a Data Volume You can use the docker volume ls command to view a list of data volumes. Use the docker volume inspect command to view the data volume details.08-Feb-2022

Where are docker volumes stored on Windows wsl2?

ls -l /var/lib/docker/volumes/

Where is docker data folder?

The standard data directory used by docker is /var/lib/docker, and since this directory will store all your images, volumes, etc. it can become quite large in a relative small amount of time.

Where are Windows docker images stored?

C:\ProgramData\Docker\image\windowsfilter.
C:\ProgramData\Docker\windowsfilter.

Where are docker desktop images stored?

C:/ProgramData/DockerDesktop/

How do I list unused docker volumes?

You can list unused volumes using the filtering option of docker volume ls command. Once these are identified, it’s easy enough to remove such volumes altogether.12-Feb-2019

What is docker volume command?

The VOLUME command will specify a mount point in the container. This mount point will be mapped to a location on the host that is either specified when the container is created or if not specified chosen automatically from a directory created in /var/lib/docker/volumes .15-May-2017

How do I get rid of unused docker volumes?

Volumes are removed using the docker volume rm command. You can also use the docker volume prune command.15-Dec-2017

Where are Docker volumes stored in WSL?

I can actually find it via PowerShell at \\wsl$\docker-desktop-data\version-pack-data\community\docker\volumes .

Источник