Multiple levels of wildcards not supported or not working properly

[csMACnz]

I used to be able to copy all files of a specific type from two-levels nested. This no longer works in BuildKit.

Why? To avoid cache busting. We want to restore our dependencies from all *.csproj files, and not invalidate that cache layer when a source file changes.

see https://stackoverflow.com/a/58697043 as the defacto approach most dotnet docker builds will be using.

If we just copy everything over first, that means any change triggers cache-busting and a full package restore is again required. Without wildcards, we have to manually maintain a long list of csproj file COPYs that is error-prone and unmaintainable across many solutions.

# These all worked without buildkit but now fail with buildkit
COPY src/*/*.csproj ./
COPY */*/*.csproj ./

This puts all found files into a single folder (which at least works for the goal we are trying to achieve)

I now get errors like:

error from sender: readdir: open *: The filename, directory name, or volume label syntax is incorrect.

or sometimes nothing is copied, so i get missing files errors later in the next RUN command.

As far as I an tell, COPY */*.csproj ./ still works as expected, when csproj files are only one-level nested, but doesn't work for the two-level nested case.

As a workaround, I might be able to move to a one-level nested structure, though ideally i wouldn't have to change my structure to fit docker for something that used to work?

Thanks!

[pierregorce]

I'm having the same problem while configuring buildkit for same caching purpose in .NET. Is there an alternative ?

COPY */*.csproj ./ (with one-level nested) not working either, is there a tricks ?

Thanks 😊

[gvanbrakel]

I don't think it is about multiple level of wildcards. When I run the below script, that has only one level of wildcard, it fails:

mkdir project-x-dependencies

echo "line-of-text" > project-x-dependencies/file1.txt
	
echo FROM busybox > Dockerfile
echo COPY project-*-dependencies/file1.txt  /work/ >> Dockerfile

docker build .

It appears to me that it is about wildcards in directories.

[YoeriVD]

"needs investigation"

Any update on this? after the last update, no one in our team using the windows version of docker can build their images anymore :/

EDIT:
Fellow googlers (temp) fix can be found here
moby/moby#42133 (comment)

[sbabych]

Good morning,
in our case we have smth like:
COPY ./drivers/build/3rdparty-jdbc-drivers/**/* jars/3rdparty/
and with "buildkit": true it doesn't work

[qrjo]

If this is gonna be fixed, it would be nice have a way to have these wildcards and preserve the folder structure as well.

[ASHWINI-GUPTA]

Same issue happened while I was trying to build the - https://github.com/Squidex/squidex.
After setting "buildkit": false was able to build but resetting "buildkit": true its giving an error from sender: readdir: open backend\extensions\*: The filename, directory name, or volume label syntax is incorrect..

[vchirikov]

Same with

COPY src/dotnet/*/*.csproj ./

doesn't work with buildkit, works fine without buildkit

[Schwaller]

Please fix this - it's creating so much work and can't be that difficult to solve this - please. IMHO this ticket doesn't need any investigation, it needs solving :D (talking of labels)

[jahsome]

Stumbled into a workaround thanks to a very unique combination of sheer laziness and force of stubborn will.

I just added a "staging" layer and slammed my entire context to copy from It appears to have the added benefit of standardizing to *nixy / paths from the get-go, and just copying in --from=ctx instead:

FROM scratch as ctx
WORKDIR /app
COPY . /app

FROM mcr.microsoft.com/dotnet/sdk:6.0 as build
COPY --from=ctx "app/src/*/*.csproj" "./"

I'm sure this is bloating my caches with unneeded builds though. I'm trying, but I can't figure out how to explicitly exclude the stage from caching (via Dockerfile at least).