Clean updating the crates.io index sets 'refs/remotes/origin/' to HEAD insted of master

[meltinglava]

Running any cargo add {crate} would fail with:

Command failed due to unhandled error: reference 'refs/remotes/origin/master' not found; class=Reference (4); code=NotFound (-3)

After some debugging i found that the cargo index had its 'refs/remotes/origin' set to HEAD instead of master. I tried rebuilding the index by running rm -rf ~/.cargo/registry/index/github.com-*/ and then cargo add <any dependency>. This updated the index, however now refs/remotes/origin/ still contains HEAD insted of master.

Found the line in that failed in this lib to be:

cargo-edit/src/fetch.rs

Line 271 in 034f6ef

.find_reference("refs/remotes/origin/master")?

Confirmed that running mv HEAD master temporarily fixed the problem.

[ordian]

Thanks for the report. Hmm, this is strange, I can't reproduce the issue. Have you tried removing the index and rebuilding it with cargo check instead of cargo add?
This is what I got after that:

❯ tree ~/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/refs/remotes/origin
~/.cargo/registry/index/github.com-1ecc6299db9ec823/.git/refs/remotes/origin
└── master

[ordian]

Updating the index is done here:

cargo-edit/src/fetch.rs

Lines 126 to 159 in 034f6ef

	let refspec = "refs/heads/master:refs/remotes/origin/master";
	fetch_with_cli(&repo, registry.as_str(), refspec)?;
	Ok(())
	}
	// https://github.com/rust-lang/cargo/blob/57986eac7157261c33f0123bade7ccd20f15200f/src/cargo/sources/git/utils.rs#L758
	fn fetch_with_cli(repo: &git2::Repository, url: &str, refspec: &str) -> Result<()> {
	let cmd = subprocess::Exec::shell("git")
	.arg("fetch")
	.arg("--tags") // fetch all tags
	.arg("--force") // handle force pushes
	.arg("--update-head-ok") // see discussion in rust-lang/cargo#2078
	.arg(url)
	.arg(refspec)
	// If cargo is run by git (for example, the `exec` command in `git
	// rebase`), the GIT_DIR is set by git and will point to the wrong
	// location (this takes precedence over the cwd). Make sure this is
	// unset so git will look at cwd for the repo.
	.env_remove("GIT_DIR")
	// The reset of these may not be necessary, but I'm including them
	// just to be extra paranoid and avoid any issues.
	.env_remove("GIT_WORK_TREE")
	.env_remove("GIT_INDEX_FILE")
	.env_remove("GIT_OBJECT_DIRECTORY")
	.env_remove("GIT_ALTERNATE_OBJECT_DIRECTORIES")
	.cwd(repo.path());
	let _ = cmd.capture().map_err(|e| match e {
	subprocess::PopenError::IoError(io) => ErrorKind::Io(io),
	_ => unreachable!("expected only io error"),
	})?;
	Ok(())
	}

[meltinglava]

Did indeed fail on cargo check as well. Thow I narrowed down the issue. This works as intended on Stable, but not on nightly.

rustup show
Default host: x86_64-unknown-linux-gnu
rustup home:  /home/vagrant/.rustup

installed toolchains
--------------------

stable-x86_64-unknown-linux-gnu
nightly-x86_64-unknown-linux-gnu (default)

active toolchain
----------------

nightly-x86_64-unknown-linux-gnu (default)
rustc 1.46.0-nightly (346aec9b0 2020-07-11)

Might be connected to rust-lang/cargo#8364

[kavirajk]

I'm facing same issue on nightly.

-bash-5.0$ cargo add clap
Command failed due to unhandled error: reference 'refs/remotes/origin/master' not found; class=Reference (4); code=NotFound (-3)

can I revert back cargo-edit to older version as temporary workaround?

[meltinglava]

@kavirajk no. Temporarly solution is to rename the file in ~/.cargo/registry/index/github.com-*/.git/refs/remotes/origincalled HEAD to master

[meltinglava]

Have thought about some fixes. One would be to allow cargo add to use any of the origins. Think we always expects this to have 1 file in that directory.

[futurist]

+1, Same problem with OP, hope for the new version

[fireairforce]

How about this problem i missed here: Command failed due to unhandled error: reference 'refs/heads/master' not found; class=Reference (4); code=NotFound (-3)

[zyansheep]

I am getting a similar error:
Error: The repo at path /home/zyansheep/.cargo/registry/index/github.com-1ecc6299db9ec823 is unusable due to having an invalid HEAD reference: reference 'refs/heads/master' not found; class=Reference (4); code=NotFound (-3)

[timabell]

I notice that others are posting that this error has reappeared. I'd be happy to open a new issue if that's useful but thought it might be useful to add the details of the error I see here and specific versions:

Error: The repo at path
 /home/tim/.asdf/installs/rust/1.77.1/registry/index/github.com-1ecc6299db9ec823
 is unusable due to having an invalid HEAD reference: 
 reference 'refs/heads/master' not found; class=Reference (4); 
 code=NotFound (-3)

It works on rust 1.77.0 but not on my asdf-vm installed 1.77.1

I have a scripted update of rust + crates for gitopolis which you can find here: https://github.com/rustworkshop/gitopolis/blob/main/upgrades.sh

I've been running it fairly regularly, and the last rust upgrade worked fine (rust update, cargo update done with cargo-edit)

I've just run it again this evening, and it's upgraded rust to 1.77.1, and then the cargo upgrade has failed as follows:

tim@fox:~/repo/gitopolis  (main*)
$ ./upgrade-crates.sh 
#!/bin/sh -v
set -e # exit on error
cargo update
    Updating crates.io index
cargo install cargo-edit
    Updating crates.io index
     Ignored package `cargo-edit v0.12.2` is already installed, use --force to override
cargo upgrade --incompatible # from cargo-edit
Error: The repo at path /home/tim/.asdf/installs/rust/1.77.1/registry/index/github.com-1ecc6299db9ec823 is unusable due to having an invalid HEAD reference: reference 'refs/heads/master' not found; class=Reference (4); code=NotFound (-3)

tim@fox:~/repo/gitopolis  (main*)
$ cargo upgrade
Error: The repo at path /home/tim/.asdf/installs/rust/1.77.1/registry/index/github.com-1ecc6299db9ec823 is unusable due to having an invalid HEAD reference: reference 'refs/heads/master' not found; class=Reference (4); code=NotFound (-3)

tim@fox:~/repo/gitopolis  (main*)
$ cargo --version
cargo 1.77.1 (e52e36006 2024-03-26)
 tim@fox:~/repo/gitopolis  (main*)

$ cargo upgrade --version  
cargo-edit-upgrade 0.12.2
 tim@fox:~/repo/gitopolis  (main*)

$ which cargo            
/home/tim/.asdf/shims/cargo
 tim@fox:~/repo/gitopolis  (main*)

$ cat .tool-versions 
rust 1.77.1

Looking at the referenced git folder, it appears to have no heads at all, which would explain the error, though I don't know why it would be in that state

$ tree -a ~/.asdf/installs/rust/1.77.1/registry/index/github.com-1ecc6299db9ec823
/home/tim/.asdf/installs/rust/1.77.1/registry/index/github.com-1ecc6299db9ec823
└── .git
    ├── config
    ├── description
    ├── HEAD
    ├── hooks
    │   └── README.sample
    ├── info
    │   └── exclude
    ├── objects
    │   ├── info
    │   └── pack
    └── refs
        ├── heads            # <= empty folder
        └── tags

I don't know what this repo/folder is for, I don't know much about the innards of asdf, rust and cargo.

The same folder in the v1.77.0 install has HEAD and master:

$ tree -a ~/.asdf/installs/rust/1.77.0/registry/index/github.com-1ecc6299db9ec823 
/home/tim/.asdf/installs/rust/1.77.0/registry/index/github.com-1ecc6299db9ec823
└── .git
    ├── config
    ├── description
    ├── FETCH_HEAD
    ├── HEAD
    ├── hooks
    │   └── README.sample
    ├── info
    │   └── exclude
    ├── logs
    │   └── refs
    │       └── remotes
    │           └── origin
    │               ├── HEAD
    │               └── master
    ├── objects
    │   ├── info
    │   └── pack
    │       ├── pack-c936aed47ba938432ada913079443dfbd2c5fff0.idx
    │       └── pack-c936aed47ba938432ada913079443dfbd2c5fff0.pack
    └── refs
        ├── heads
        ├── remotes     # <== entirely missing in 1.77.1
        │   └── origin
        │       ├── HEAD
        │       └── master
        └── tags

15 directories, 12 files

Hope that's in some way helpful to someone