Add namespace query parameter for mirrors

[dmcgowan]

Mirrored registries are design to serve content from upstreams. However, the mirror hostname will usually not match the hostname of the upstream, requiring the mirror to only use a single upstream or use its own pattern matching to determine the upstream. To solve this issue, the client will pass along the namespace which is being used for the request, allowing mirrors to easily map to multiple upstreams. This query parameter can safely be ignored if multiple upstreams are not supported.

Marking as draft to move along conversation in opencontainers/distribution-spec#12

[theopenlab-ci]

Build succeeded.

• containerd-build-arm64 : SUCCESS in 8m 25s (non-voting)

[codecov-io]

Codecov Report

Merging #3451 into master will increase coverage by <.01%.
The diff coverage is 53.65%.

@@            Coverage Diff             @@
##           master    #3451      +/-   ##
==========================================
+ Coverage   44.11%   44.11%   +<.01%     
==========================================
  Files         124      124              
  Lines       13760    13794      +34     
==========================================
+ Hits         6070     6085      +15     
- Misses       6759     6774      +15     
- Partials      931      935       +4

Flag	Coverage Δ
#linux	47.87% <56.25%> (ø)	⬆️
#windows	39.77% <53.65%> (+0.01%)	⬆️

Impacted Files	Coverage Δ
remotes/docker/registry.go	61.33% <100%> (+1.05%)	⬆️
remotes/docker/fetcher.go	47.54% <25%> (-2.46%)	⬇️
remotes/docker/resolver.go	53.29% <58.06%> (-0.24%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fdab4f4...7538d45. Read the comment docs.

[theopenlab-ci]

Build succeeded.

• containerd-build-arm64 : SUCCESS in 8m 31s (non-voting)

[Random-Liu]

@dmcgowan With today's containerd, the client is configured to talk with a specific registry MirrorHost, and resolving the original image ref UpstreamHost/library/busybox.

In this model, the mirror will get the original image ref, thus it will know whatever the upstream is, right?

I guess you don't like the regexp match of UpstreamHost/library/busybox on the mirror server?

[dmcgowan]

The problem with UpstreamHost/library/busybox is that no mirror supports it and there is no way today for the client to know whether the registry would understand that. The path to get that working would be to re-introduce the ping before every pull along with a defined configuration to figure out how the registry wants the client to interpret the "path" component. This would reintroduce the extra latency for ping, make the whole process more complicated, and fragment the registry ecosystem as now registries would need to be able to handle multiple types of clients to offer content on multiple URLs.

In this model, the mirror will get the upstream host information via a query parameter, which can be safely ignored if the mirror does not support arbitrary or multiple upstreams. (A path component cannot be ignored 😄 )

[Random-Liu]

With today's containerd, the client is configured to talk with a specific registry MirrorHost, and resolving the original image ref UpstreamHost/library/busybox.

@dmcgowan Hm, my bad. I didn't pay attention to the protocol before. I thought that we are just talking to different mirrors, and request UpstreamHost/library/busybox (as a parameter or sub-path), so that mirrors should know the upstream.

However, as you said (and based on the log and code), it seems that todays implementation is https://MirrorHost/v2/library/busybox. :)

Then adding UpstreamHost as a parameter of the request makes sense to me. :)

[dmcgowan]

@Random-Liu I see in your mirror PR on CRI you don't add the push capability to the hosts. This code would see that and add the query parameter on all requests from CRI. While it is harmless I am not sure if that is ideal

[Random-Liu]

@dmcgowan Kubernetes/CRI doesn't need the push capability from the client side.

If we want to add push capability, I think we can only add it to the default host.

[fuweid]

should parse first and then truncate the path

[dmcgowan]

Going to close this for now since it is not mergeable, changes need to be made for configurable and better logic for determining whether a registry is a global mirror.