--latest flag in `docker image list`

[lczyk]

Tell us about your request

i find that i docker container list --latest quite a bit, and it would be helpful to have the same functionality (aka the --latest flag) in docker image list. I imagine the --last flag would be straightforward to port from container to image at the same time.

Which service(s) is this request for?

docker

Are you currently working around the issue?

docker image list --format '{{.CreatedAt}} {{.Repository}} {{.Tag}} {{.ID}}' | sort | head -n1 | cut -d' ' -f5-

[thaJeztah]

Thanks for opening this ticket!

Can you describe a bit more what your use-case is? It would help to look at what improvements can be made.

The longer answer there is that the "created" date that's currently used to sort images is somewhat ambiguous (more below). There may be some other options, but those would require additional metadata to be kept in the image store provided by the daemon (also see moby/moby#4237), but even with that, there may be ambiguity and potential TOCTOU issues, so having a better understanding of your use-case may be useful.

The problem is that the CREATED date (sometimes depending on how the image was built) will show the date the image was created, which not always reflects "last" (depending on your use-case); we are even considering to hide the column by default, possibly providing more detailed information through other means.

Here's some examples; in the example below, 2 images are built;

echo -e 'FROM busybox\nRUN echo foo > foo.txt\n' | docker build -t myimage1 -

echo -e 'FROM busybox\nRUN echo bar > bar.txt\n' | docker build -t myimage2 -

So far so good; the last image built shows at the top (docker image ls currently sorts on CREATED);

docker image ls
REPOSITORY                            TAG                IMAGE ID       CREATED          SIZE
myimage2                              latest             ec2e63fffa5f   4 seconds ago    6.04MB
myimage1                              latest             ecef5240b936   43 seconds ago   6.04MB

However, there's many situations wherer the CREATED date may not match;

Continuing with the example above, if I would build the first image again and there's no changes, the image build would be fully cached, so no new image created;

echo -e 'FROM busybox\nRUN echo foo > foo.txt\n' | docker build -t myimage1 -
[+] Building 1.0s (6/6) FINISHED                                                                   docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                               0.0s
 => => transferring dockerfile: 74B                                                                                0.0s
 => [internal] load metadata for docker.io/library/busybox:latest                                                  0.1s
 => [internal] load .dockerignore                                                                                  0.0s
 => => transferring context: 2B                                                                                    0.0s
 => [1/2] FROM docker.io/library/busybox:latest@sha256:f85340bf132ae937d2c2a763b8335c9bab35d6e8293f70f606b9c6178d  0.1s
 => => resolve docker.io/library/busybox:latest@sha256:f85340bf132ae937d2c2a763b8335c9bab35d6e8293f70f606b9c6178d  0.1s
 => CACHED [2/2] RUN echo foo > foo.txt                                                                            0.0s
 => exporting to image                                                                                             0.4s
 => => exporting layers                                                                                            0.0s
 => => exporting manifest sha256:5a059e4f4f8501234adee8532a8435e2d2dee21c25196303bba9380bfc298a69                  0.0s
 => => exporting config sha256:1bc9413f73da5d5f3c0fd37b21afcc50dabeff10b24f2d386ea80f8f84576a00                    0.0s
 => => exporting attestation manifest sha256:99f69bac8a52b41ea8bd818ad2a7d41d4111fbe69115b9a6b6b1f47aef2dcea4      0.1s
 => => exporting manifest list sha256:95c5a999f4343f8e29c1cf588cf2eb19b57fdcd7f8ecc4a68341ec0487119b2b             0.1s
 => => naming to docker.io/library/myimage1:latest                                                                 0.0s
 => => unpacking to docker.io/library/myimage1:latest                                                              0.0s

Which means that the image would still be second last and considered "older" than image2;

docker image ls | head -n 3
REPOSITORY                            TAG                IMAGE ID       CREATED         SIZE
myimage2                              latest             c1b451eff3ac   3 minutes ago   6.04MB
myimage1                              latest             95c5a999f434   4 minutes ago   6.04MB

Similarly, the CREATED date would reflect the date the image was created; when pulling an image, this may be days, or months ago; for images using reproducible builds, it's even possible that the date is reset to a fixed date in the past, which can show many years in the past 😅

docker pull -q debian:bookworm
docker pull -q hello-world
docker pull -q paketobuildpacks/builder-jammy-base:latest

docker image ls
REPOSITORY                            TAG                IMAGE ID       CREATED         SIZE
myimage2                              latest             c1b451eff3ac   5 minutes ago   6.04MB
myimage1                              latest             95c5a999f434   6 minutes ago   6.04MB
debian                                bookworm           d42b86d7e24d   2 days ago      204MB
hello-world                           latest             940c619fbd41   5 months ago    17kB
paketobuildpacks/builder-jammy-base   latest             c0768d9659eb   45 years ago    2.4GB

When building images, it's possible to capture the ID of the image that was built, which would prevent TOCTOU issues;

echo -e 'FROM busybox\nRUN echo foo > foo.txt\n' | docker build -t myimage1 --iidfile ./image-id -
cat ./image-id
sha256:6975486945fd896897a42b718ea2c5d2985d48d5e3b2d2f846364c99d45fbb41

docker run --rm sha256:6975486945fd896897a42b718ea2c5d2985d48d5e3b2d2f846364c99d45fbb41 sh -c 'cat /foo.txt'
foo

When pulling an image, the output of the pull can be used;

docker pull -q hello-world
docker.io/library/hello-world:latest

Although the default is currently to show the name of the image (possibly a --iidfile option could be added to preserve the digest), or alternatively a --format option to allow printing the desired field (name or digest);

docker pull hello-world
Using default tag: latest
latest: Pulling from library/hello-world
Digest: sha256:940c619fbd418f9b2b1b63e25d8861f9cc1b46e3fc8b018ccfe8b78f19b8cc4f
Status: Image is up to date for hello-world:latest
docker.io/library/hello-world:latest

[lczyk]

thanks for a very thorough response 😮 👍

Can you describe a bit more what your use-case is?

my specific use pattern is to use skopeo to copy an oci image file to the docker daemon and then refer to it easily, from multiple terminals / windows, hence setting an env variable wouldn't work. maybe i ought to add something to my workflow to write the name to a file in tmp and then use that...

and yes, you're absolutely right about the image creation time. for me it happens to be good enough, but it is ambiguous in the context of what does 'latest' mean. i guess what i'd imagine is something equivalent to unix file accessed timestamp, aka last-interacted-with, but as you said: thats a piece of metadata which is not kept at the moment, and it is not clear as to what that would exactly mean.

tldr; i can see why that's not the option, at least at the moment, and don't see a clear solution. food for thought maybe.

happy to close

[thaJeztah]

and then refer to it easily, from multiple terminals / windows

Ah, hm, yes, that's slightly more tricky as there would be a need to communicate between those terminals. The --iidfile on docker pull or docker load / docker import (or what's used by skopeo), but I guess by that time, it may as well be an option to write to a file yourself if you know what's being copied.

It's worth mentioning there that we are in process of migrating the image store to use the containerd image store, which will be the default (for fresh installs) in an upcoming release;

• Docker Desktop: https://docs.docker.com/desktop/features/containerd/
• Docker Engine standalone: https://docs.docker.com/engine/storage/containerd/

While somewhat orthogonal, one of the benefits of the containerd image store is that we are able to preserve the image-digest of the OCI "distributable" image, which means that the digest as present in registries is preserved, and can be used to refer to a specific version of the image.

All that said; additional metadata could still be useful for these use-cases; while they won't solve TOCTOU issues, they may help to provide additional means of sorting / filtering.

happy to close

I think it's worth to keep the ticket (or possibly could be converted to a "discussion") to dig deeper into use-cases, and see what kind of improvements would be possible. Even if they're not addressed in the short term, it always helps to get a picture of how users are using Docker, which can help decide direction when working on enhancements in these areas.

Let me transfer this ticket to the docker/cli repository in the mean time.