nix 2.30 - "is not allowed to refer to the following paths"

[h0nIg]

Describe the bug

build failure on certain derivation which use advanced nix features: NixOS/nixpkgs#448752

nix 2.29.2 works, 2.30.3 not (including 2.31.*)

$ nix build ".#postgresql_17" --verbose
this derivation will be built:
  /nix/store/x8l5i34yws4jm9vi6nj1791xr3rjqr3g-postgresql-17.6.drv
building '/nix/store/x8l5i34yws4jm9vi6nj1791xr3rjqr3g-postgresql-17.6.drv'...
error: output '/nix/store/iaddlmk85rsykjpx54glm9vnzh66ixbw-postgresql-17.6-jit' is not allowed to refer to the following paths:
         /nix/store/02k5k9n7j04ckr0m8533zwkfvwp64gi3-clang-wrapper-20.1.8
         /nix/store/6fqij3sy1mlsnq1n679bzbl5bkqs7yvk-zlib-1.3.1-dev
         /nix/store/87m0bfli6f63fx1shw4sw59cjcqpvzb0-zstd-1.5.7-dev
         /nix/store/cn6yrlr63spymdnxf6m8gd9zkkxpcpas-krb5-1.22.1-dev
         /nix/store/cz9k6nhxjppa1kmyf5npd0g8l89xzilw-openssl-3.5.2-dev
         /nix/store/i94kq19cgv5c2phfj6gzkkl84pgs9pas-util-linux-minimal-2.41.1-dev
         /nix/store/pk1kxabxmwnr5d2gm778haqrsivkpp1j-systemd-minimal-libs-257.9-dev
         /nix/store/rrpmwmachqkv4cn5i0azqw8dqxyx9wrj-libxml2-2.14.6-dev
         /nix/store/sy2vbl9113xlz1w47jk5gx7rg8rk089p-llvm-20.1.8
         /nix/store/ys2c150xm8rfnwi15pkl6a3ic49xlb6g-lz4-1.10.0-dev
         /nix/store/zc7w3ii28j4kagmlznjdajf0nzd0z60a-icu4c-76.1-dev

Steps To Reproduce

recent build on nixpkgs staging:

git checkout b7516ddb2fa10be3e3e711634c76748ba1646e21
nix build ".#postgresql_17" --verbose

Expected behavior

Metadata

Additional context

Checklist

• checked latest Nix manual (source)
• checked open bug issues and pull requests for possible duplicates

---

Add 👍 to issues you find important.

[xokdvium]

As a data-point, I can't reproduce this with 7ba3ef2.

nix path-info -f . postgresql_17.jit -L -r
/nix/store/0ja5daq2yay3svg6f48spyfby7pgy5bm-xgcc-14.3.0-libgcc
/nix/store/7ndy5820h69n1pgd13viph1qjq6zhhkj-libunistring-1.3
/nix/store/wwqyxsvmg41q5ckfvvv6wa50nwvz9dv4-libidn2-2.3.8
/nix/store/qhw0sp183mqd04x5jp75981kwya64npv-glibc-2.40-66
/nix/store/6qi8skh85ci2k9gvl27nnh3kiy32qnsz-zlib-1.3.1
/nix/store/8b9srwwmrwmh1yl613cwwj7gydl87br6-gcc-14.3.0-libgcc
/nix/store/jwjf14rgh6ms0vn2c4qvx2b6h97cg3ps-libffi-3.5.2
/nix/store/xp989kyfg52803fmkzbz5py35jphcpgd-gcc-14.3.0-lib
/nix/store/yw9ks7f8sx384yw37xrsqdxn2ljw2d1y-libxml2-2.14.6
/nix/store/nbk2z7yjm0wm29idcqhqp64b1p5hkllb-llvm-20.1.8-lib
/nix/store/iaddlmk85rsykjpx54glm9vnzh66ixbw-postgresql-17.6-jit

@h0nIg are you able to reproduce this issue reliably?

[fzakaria]

I ran this on an older version too and could not reproduce it.

> ./build/src/nix/nix --version
nix (Nix) 2.31.0

> ./build/src/nix/nix build github:NixOS/nixpkgs/nixpkgs-unstable#postgresql_17

> echo $status
0

[fzakaria]

Okay @xokdvium reminded me that I didn't swap my daemon

I ran this on an older version too and could not reproduce it.

I did do that here and still no reproduction.

> ./build/src/nix/nix --version
nix (Nix) 2.31.0

> ./build/src/nix/nix build github:NixOS/nixpkgs/nixpkgs-unstable#postgresql_17 --store /tmp/issue-14157

> echo $status
0

[fzakaria]

> ./build/src/nix/nix build github:NixOS/nixpkgs?commit=b7516ddb2fa10be3e3e711634c76748ba1646e21#postgresql_17 --store /tmp/issue-14157

> echo $status
0

> ./build/src/nix/nix --version
nix (Nix) 2.30.3

Even at 2.30.3 couldn't reproduce with exact commit listed.... hmm.

[h0nIg]

@fzakaria in my example i used the latest staging which is not present on a binary cache. With a binary cache hit, it should look like this:

$ docker run -ti nixos/nix:2.30.3
bash-5.2# nix build "nixpkgs#postgresql_17" --rebuild --extra-experimental-features nix-command --extra-experimental-features flakes
[45.3 MiB DL] unpacking 'github:NixOS/nixpkgs/d7f52a7a640bc54c7bb414cca603835bf8dd4b10' into the Git cache
error: derivation '/nix/store/ggdg2kxcv7rzj7k5z4gikb8qaj5l00l6-postgresql-17.6.drv' may not be deterministic: output '/nix/store/msjxcqa4x2f52dyq10rbrbw6k0m0hi90-postgresql-17.6-lib' differs

does this makes sense?

[xokdvium]

@h0nIg Interesting, I did build postgresql_17 without substituters on staging and wasn't able to reproduce. Is postgre known to be nondeterministic? Could you run diffoscope on those store paths?

[h0nIg]

@h0nIg Interesting, I did build postgresql_17 without substituters on staging and wasn't able to reproduce. Is postgre known to be nondeterministic? Could you run diffoscope on those store paths?

@wolfgangwalther @Ma27 ?

[wolfgangwalther]

I'm not exactly sure what you're asking for @h0nIg. We were already not able to reproduce this.

You were repeatedly asked to look at these store paths yourself. Please do so?

[h0nIg]

I'm not exactly sure what you're asking for @h0nIg. We were already not able to reproduce this.

You were repeatedly asked to look at these store paths yourself. Please do so?

i provided a better way to reproduce this, you can just use the nix docker image for it (see above). would you mind taking a look at it again please, since you are the experts and a second set of eyes would be beneficial

[Ma27]

Ehm, this is not the error we've been talking about though. I'm not too surprised that the package isn't bit-by-bit reproducible (though this is something we should take a look at eventually).

Which brings me back to what I asked to before in nixpkgs:

Does the store-path still exist? Can you share, which files in the store-path reference these files specifically? Ideally share the affected files somewhere. Because I'm afraid I don't have a good explanation for this right now.

If we have a store-path there that actually references disallowedReferences, we have a bad impurity in the build (which will be fun considering that nobody could reproduce it so far). Otherwise, it's a Nix bug.

[h0nIg]

@fzakaria in my example i used the latest staging which is not present on a binary cache. With a binary cache hit, it should look like this:

``` $ docker run -ti nixos/nix:2.30.3 bash-5.2# nix build "nixpkgs#postgresql_17" --rebuild --extra-experimental-features nix-command --extra-experimental-features flakes [45.3 MiB DL] unpacking 'github:NixOS/nixpkgs/d7f52a7a640bc54c7bb414cca603835bf8dd4b10' into the Git cache error: derivation '/nix/store/ggdg2kxcv7rzj7k5z4gikb8qaj5l00l6-postgresql-17.6.drv' may not be deterministic: output '/nix/store/msjxcqa4x2f52dyq10rbrbw6k0m0hi90-postgresql-17.6-lib' differs ```

does this makes sense?

ok, postgres is non deterministic, but this is not the root cause here and just a side-effect finding which can get ignored for now. diffoscope just revealed different buildID's.

I was able to compare the broken postgresql-jit with the old postgresql-jit nix store path and they are completely different (does not make sense to post the diffoscope output here). Looks like we really need to find a reproducible way to trigger this behaviour

[h0nIg]

the following shows the problem again. I was able to continously reproduce the issue within a x64 docker image running nix. I just had to cleanup the store path and triggered an offline nix build. By using a different nix CLI (no nix daemon), i was able to show it:

the flake.lock was locked with the following content

    "nixpkgs-repo": {
      "locked": {
        "lastModified": 1759709826,
        "narHash": "sha256-Z9ZBXUiVA6JWPoNsz06SchLKp/2bRYg8ujdbeLl3Ybk=",
        "owner": "NixOS",
        "repo": "nixpkgs",
        "rev": "f2ebc99bb035b412d78d5c8537e5a471c7f6c6bc",
        "type": "github"
      },
      "original": {
        "owner": "NixOS",
        "ref": "staging",
        "repo": "nixpkgs",
        "type": "github"
      }
    },

[wolfgangwalther]

@RaitoBezarius you mentioned something on Matrix that makes me believe this issue could be related to path length of the build directory or so?