Weird race condition with /home/build at startup w/ QEMU runner

[dannf]

I've seen a couple weird situations where a runs: at the top of a melange.yaml isn't seeing what I expect. Most recently, I added a test to make sure a file from a package directory was present, and it would always fail. But if I ran another command first, the file would be there.

runs: |
    test -x quickstart.py

[egibs]

@89luca89 -- are we maybe not waiting long enough when spinning up the QEMU VM here?

melange/pkg/build/build.go

Lines 689 to 705 in 02e2607

	if err := b.Runner.StartPod(ctx, cfg); err != nil {
	return fmt.Errorf("unable to start pod: %w", err)
	}
	if !b.DebugRunner {
	defer func() {
	if err := b.Runner.TerminatePod(context.WithoutCancel(ctx), cfg); err != nil {
	log.Warnf("unable to terminate pod: %s", err)
	}
	}()
	}
	// run the main pipeline
	log.Debug("running the main pipeline")
	pipelines := b.Configuration.Pipeline
	if err := pr.runPipelines(ctx, pipelines); err != nil {
	return fmt.Errorf("unable to run package %s pipeline: %w", b.Configuration.Name(), err)
	}

Pure conjecture but this is definitely interesting.

[egibs]

Maybe this isn't done before we run the pipeline?

melange/pkg/container/qemu_runner.go

Lines 851 to 865 in 02e2607

	err = sendSSHCommand(ctx,
	cfg.SSHClient,
	cfg,
	nil,
	stderr,
	stdout,
	false,
	[]string{"sh", "-c", "find /mnt/ -mindepth 1 -maxdepth 1 -exec cp -a {} /home/build/ \\;"},
	)
	if err != nil {
	err = qemuCmd.Process.Kill()
	if err != nil {
	return err
	}
	}

Although:

melange/pkg/container/qemu_runner.go

Lines 1226 to 1230 in 02e2607

	// Wait for the session to finish
	if err := session.Wait(); err != nil {
	clog.FromContext(ctx).Errorf("Failed wait for session running: %q: %v", cmd, err)
	return err
	}

[89luca89]

Maybe this isn't done before we run the pipeline?

melange/pkg/container/qemu_runner.go

Lines 851 to 865 in 02e2607

err = sendSSHCommand(ctx,
cfg.SSHClient,
cfg,
nil,
stderr,
stdout,
false,
[]string{"sh", "-c", "find /mnt/ -mindepth 1 -maxdepth 1 -exec cp -a {} /home/build/ \;"},
)
if err != nil {
err = qemuCmd.Process.Kill()
if err != nil {
return err
}
}
Although:

melange/pkg/container/qemu_runner.go

Lines 1226 to 1230 in 02e2607

// Wait for the session to finish
if err := session.Wait(); err != nil {
clog.FromContext(ctx).Errorf("Failed wait for session running: %q: %v", cmd, err)
return err
}

The first step is performed in the StartPod step, and pipelines will not run until this step is done

[dannf]

Another example, where I see that find command erroring out while git-checkout is in-progress.