Nix build hangs with "download thread waiting for 10000 ms" loop

[thomasjm]

Describe the bug

I'm having this issue very similar to #11249, where I try to run a Nix build in a container-like environment and get an endless loop like this:

download thread waiting for 10000 ms
download thread waiting for 10000 ms
download thread waiting for 10000 ms
download thread waiting for 10000 ms
download thread waiting for 10000 ms

In my case, rather than devcontainer, i'm using Bubblewrap. Also, I'm mounting a local binary cache constructed with mkBinaryCache and trying to make my build use it as a substituter.

Steps To Reproduce

I put together a full repro here, which constructs a script that shows the problem: https://github.com/thomasjm/nix-deadlock-repro. All you have to do is

git clone git@github.com:thomasjm/nix-deadlock-repro.git
cd nix-deadlock-repro
nix run .#go

(If this succeeds the first time, maybe try it once or twice more and you should see the deadlock.)

Expected behavior

The Nix build should complete successfully and promptly.

Metadata

Everything is fixed in my repro flake. I'm using a static build of Nix 2.28.

Additional context

Issue #11249 said it was fixed by #11171, but I'm testing on the latest Nix and that solution is not fixing it for me.

Checklist

• checked latest Nix manual (source)
• checked open bug issues and pull requests for possible duplicates

---

Add 👍 to issues you find important.

[NaN-git]

Has cores = 0 a special meaning in nix.conf? I can reproduce the issue with these settings, but I'm not sure whether the config is correct. If I set cores = 1, then the issue seems to disappear.

[thomasjm]

From the docs:

The cores setting determines the value of NIX_BUILD_CORES. NIX_BUILD_CORES is equal to cores, unless cores equals 0, in which case NIX_BUILD_CORES will be the total number of cores in the system.

My intention with cores = 0 is to have Nix autodetect the number of cores. I wonder if something could be going wrong with that inside the bubblewrap container.

[thomasjm]

I just tried cores = 1 and got the issue still. It's not always perfectly reproducible.

[thomasjm]

CC @roberth who was involved in some similar issues like #11258

[thomasjm]

Ah, I'm seeing it happen sometimes even without the Bubblewrap sandbox.

The common denominator seems to be that it gets stuck when using a local substituter, like --extra-substituters 'file://...'.

[thomasjm]

Update: it can get stuck with a local HTTP substituter too! I made a branch to try this variation: https://github.com/thomasjm/nix-deadlock-repro/tree/http. (See comparison).