feat(buildkit): Export cache to registry

[gaocegege]

Description

The build cache can be exported to the registry. We need to investigate the benefits of AI/ML scenarios.

[gaocegege]

If we implement this, we can remove conda in base images. The dependencies can be installed during build process, and we can cache them in the registry.

[gaocegege]

/cc @kemingy

[gaocegege]

Ref

• Consistent cache misses when pulling images from a remote cache locally moby/buildkit#2383
• Cache pushed from one machine can not be reused on another machine moby/buildkit#1251

Tried to push the cache to the registry and import it in a clean build, it does not work. Not sure what happens:

eg.Go(func() error {
				_, err := b.Client.Build(ctx, client.SolveOpt{
					CacheExports: []client.CacheOptionsEntry{
						{
							Type: "inline",
							Attrs: map[string]string{
								"mode": "max",
							},
						},
					},
					CacheImports: []client.CacheOptionsEntry{
						{
							Type: "registry",
							Attrs: map[string]string{
								"ref": b.tag,
							},
						},
					},
					Exports: []client.ExportEntry{entry},
					LocalDirs: map[string]string{
						flag.FlagCacheDir: home.GetManager().CacheDir(),
					},
					Session: attachable,
					// TODO(gaocegege): Use llb.WithProxy to implement it.
					FrontendAttrs: map[string]string{
						"build-arg:HTTPS_PROXY": os.Getenv("HTTPS_PROXY"),
					},
				}, "envd", b.BuildFunc(), pw.Status())

				if err != nil {
					err = errors.Wrap(err, "failed to solve LLB")
					return err
				}
				b.logger.Debug("llb def is solved successfully")
				return nil
			})

[gaocegege]

moby/buildkit#2383 (comment)

Maybe it is related to the umask. We need to re-compile buildkit to know what happens here.

[gaocegege]

I simplified the LLB to:

func (g Graph) compilePython(aptStage llb.State) (llb.State, error) {
	systemStage := g.compileSystemPackages(aptStage)

	return systemStage, nil
}

There are only two layers: llb.Image and llb.Run. The remote still does not work. Then I added some debug info into buildit:

• Print cachemap of run state.

The cachemap is always the same.

I need to have a deep dive into the implementation about the remote cache.