Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.5.0 breaks h2 build #144

Closed
crackcomm opened this issue Jul 30, 2020 · 19 comments
Closed

1.5.0 breaks h2 build #144

crackcomm opened this issue Jul 30, 2020 · 19 comments

Comments

@crackcomm
Copy link

crackcomm commented Jul 30, 2020

After re-installing my environment I'm getting an error on same code base:

error[E0107]: wrong number of type arguments: expected 3, found 2
  --> /home/pah/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/h2-0.2.6/src/proto/streams/store.rs:15:10
   |
15 |     ids: IndexMap<StreamId, SlabIndex>,
   |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected 3 type arguments

error: aborting due to previous error

For more information about this error, try `rustc --explain E0107`.
error: could not compile `h2`.

Is there a way to somehow set has_std for indexmap crate so deep in dependency tree?

Edit: It works on windows.

@cuviper
Copy link
Member

cuviper commented Jul 30, 2020

I suppose your lock file had an older version before, but do you have any idea what that was? You could try cargo update -p indexmap --precise 1.4.0 or older, if the rest of your dependencies will allow it.

You say Windows works, so what is the target which fails? If that's using cross or a custom spec, it could be related to cuviper/autocfg#15.

Right now has_std is detected with an autocfg test, but we could add a manual feature to force that on.

@crackcomm
Copy link
Author

@cuviper that's exactly what happened, Cargo.lock was old.

Windows works and stable on Linux works as well.

Now a strange thing happened. I tried to build the repo with stable which failed because of unstable features in my code but Checking h2 v0.2.6 passed. Once I switched back to nightly everything compiled just fine.

@crackcomm
Copy link
Author

crackcomm commented Jul 30, 2020

I'd close this issue because I think it should be in autocfg repo. Thanks for help to you @cuviper and few people from the discord server.

Edit: If I can help somehow let me know.

@cuviper
Copy link
Member

cuviper commented Jul 30, 2020

Ultimately, whether the issue is here or in autocfg, I'm going to need a way to reproduce the failure. If you're doing something with with rustc flags that's not picked up by autocfg, that's the known issue, otherwise I don't know what's wrong.

@crackcomm
Copy link
Author

crackcomm commented Jul 30, 2020

Fortunately, it was fresh Ubuntu 18.04 on WSL initialized with a single script so you can reproduce this.

[dependencies.hyper]
version = "^0.13.7"
features = ["runtime", "stream"]
#!/bin/sh

sudo add-apt-repository -y ppa:avsm/ppa
sudo apt-get update &&
  sudo apt-get install -qy \
    m4 \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common \
    build-essential \
    git \
    wget patch diffutils zlib1g-dev libtinfo-dev \
    opam libffi-dev \
    pkg-config libssl-dev python3.8 python3.8-dev python3-pip &&
  sudo apt-get clean autoclean &&
  sudo apt-get autoremove -y

echo "alias python3=python3.8" >>$HOME/.bashrc

python3.8 -m pip install --upgrade pip
python3.8 -m pip install -r requirements-dev.txt

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88
sudo add-apt-repository \
  "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"
sudo apt-get update
sudo apt-get install -qy docker-ce docker-ce-cli containerd.io
sudo usermod -a -G docker $USER

# Rust
curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain nightly-2020-07-27 --no-modify-path -y
echo "source $HOME/.cargo/env" >>$HOME/.bashrc
source $HOME/.cargo/env
rustup component add rls rust-analysis rust-src

wget https://github.com/Kitware/CMake/releases/download/v3.11.0/cmake-3.11.0-Linux-x86_64.tar.gz
mkdir -p /tmp/cmake
tar -xf cmake-3.11.0-Linux-x86_64.tar.gz -C /tmp/cmake
sudo cp -r /tmp/cmake/cmake-3.11.0-Linux-x86_64/bin /usr/
sudo cp -r /tmp/cmake/cmake-3.11.0-Linux-x86_64/share /usr/
sudo cp -r /tmp/cmake/cmake-3.11.0-Linux-x86_64/doc /usr/share/
sudo cp -r /tmp/cmake/cmake-3.11.0-Linux-x86_64/man /usr/share/
rm -rf cmake-3.11.0-Linux-x86_64.tar.gz /tmp/cmake

opam init -a --disable-sandboxing
opam install -y dune core core_bench alcotest conf-rust-2018 ppx_jane cohttp lwt odoc torch || true
eval $(opam env)
echo "eval \$(opam env)" >>$HOME/.bashrc

@cuviper
Copy link
Member

cuviper commented Jul 30, 2020

I'm running Fedora 32, but I tried your setup in a ubuntu:18.04 container.

python3.8 -m pip install -r requirements-dev.txt

I don't have the file for this step -- pressing on...

[dependencies.hyper]
version = "^0.13.7"
features = ["runtime", "stream"]

Are you suggesting an new project with just that dependency? Because this worked fine for me.

Now a strange thing happened. I tried to build the repo with stable which failed because of unstable features in my code but Checking h2 v0.2.6 passed. Once I switched back to nightly everything compiled just fine.

This fickle behavior makes me wonder if it's a problem with WSL, like a race in the filesystem or something.

If you manage to get back to a failed build, you can try to see what the build script did -- I get:

$ cat target/debug/build/indexmap-*/output
cargo:rustc-cfg=has_std
cargo:rerun-if-changed=build.rs

$ cat target/debug/build/indexmap-*/stderr
(nothing)

@crackcomm
Copy link
Author

I removed target, ~/.cargo/registry and stable cargo.

$ cat target/debug/build/indexmap-*/output
cargo:rustc-cfg=has_std
cargo:rerun-if-changed=build.rs
$ cat target/debug/build/indexmap-*/stderr

Looks like I myself cannot reproduce that. Did I miss removing any directory?

This fickle behavior makes me wonder if it's a problem with WSL, like a race in the filesystem or something.

I was using the rust language server on vscode through WSL, maybe that could be the reason.

It is highly likely that the problem is related to WSL more than anything.

I will re-install Ubuntu again.

@crackcomm
Copy link
Author

crackcomm commented Jul 30, 2020

It failed with the same error.

$ cat target/debug/build/indexmap-*/output
cat: 'target/debug/build/indexmap-*/output': No such file or directory
$ cat target/debug/build/indexmap-*/stderr
cat: 'target/debug/build/indexmap-*/stderr': No such file or directory

Maybe you could try changing cargo to nightly:

curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain nightly --no-modify-path -y

Edit: I also skipped opam install step.

Edit: FYI requirements-dev.txt:

pip>=18.1
pytest>=5.4.3
setuptools-rust>=0.10.6
toml>=0.10.1
pydoctor>=20.7.1
pyre-check>=0.0.46
nox>=2020.5.24
pathlib>=1.0

@cuviper
Copy link
Member

cuviper commented Jul 30, 2020

$ cat target/debug/build/indexmap-*/output
cat: 'target/debug/build/indexmap-*/output': No such file or directory
$ cat target/debug/build/indexmap-*/stderr
cat: 'target/debug/build/indexmap-*/stderr': No such file or directory

If those don't exist, it's as if the build script didn't run at all... 😕

Maybe you could try changing cargo to nightly:

In my container I used the exact nightly you specified:

curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain nightly-2020-07-27 --no-modify-path -y
$ rustup show
Default host: x86_64-unknown-linux-gnu
rustup home:  /home/user/.rustup

nightly-2020-07-27-x86_64-unknown-linux-gnu (default)
rustc 1.47.0-nightly (6c8927b0c 2020-07-26)

$ rustc -Vv
rustc 1.47.0-nightly (6c8927b0c 2020-07-26)
binary: rustc
commit-hash: 6c8927b0cf80ceee19386026cf9d7fd4fd9d486f
commit-date: 2020-07-26
host: x86_64-unknown-linux-gnu
release: 1.47.0-nightly
LLVM version: 10.0

@crackcomm
Copy link
Author

@cuviper before posting an issue I tried with fresh ubuntu twice: once with nightly in script and reinstalling to nightly-2020-07-27 and second time with nightly-2020-07-27 in script.

All following commands would fail with same error:

$ cargo test --release
(error)
$ cargo test
(error)
$ cargo check
(error)

I did (all in a mounted directory):

$ rm -rf ~/.cargo/registry
$ rustup toolchain install stable
$ rustup default stable
$ cargo test
(lots of errors about features not supported on stable)
For more information about this error, try `rustc --explain E0554`.
error[E0107]: wrong number of type arguments: expected 3, found 2
  --> /home/pah/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/h2-0.2.6/src/proto/streams/store.rs:15:10
   |
15 |     ids: IndexMap<StreamId, SlabIndex>,
   |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected 3 type arguments

error: aborting due to previous error

For more information about this error, try `rustc --explain E0107`.
error: build failed
$ rustup show
Default host: x86_64-unknown-linux-gnu
rustup home:  /home/pah/.rustup

installed toolchains
--------------------

stable-x86_64-unknown-linux-gnu (default)
nightly-2020-07-27-x86_64-unknown-linux-gnu
nightly-x86_64-unknown-linux-gnu

active toolchain
----------------

stable-x86_64-unknown-linux-gnu (default)
rustc 1.45.0 (5c1f21c3b 2020-07-13)

Now in ~/hyper, part of a virtual hard drive.

$ rustup default stable
$ git clone https://github.com/hyperium/hyper.git
$ cargo test --features stream --features runtime
(...)
test result: ok. 18 passed; 0 failed; 6 ignored; 0 measured; 0 filtered out

I copied my repo to $HOME and it just works!

@cuviper
Copy link
Member

cuviper commented Jul 30, 2020

I've seen issues with Windows virtual drives before: rust-lang/rust#54216

@crackcomm
Copy link
Author

Does autocfg rely in any way on file system modification time?

@crackcomm
Copy link
Author

I got something!

$ cat target/debug/build/indexmap-*/output
cargo:rerun-if-changed=build.rs
$ cat target/debug/build/indexmap-*/stderr
error: could not copy "/mnt/d/crypto-market/cxmr/cxmr-master/cxmr/target/debug/build/indexmap-62ff17e61db67c74/out/probe0.probe0.3a1fbbbh-cgu.0.rcgu.ll" to "/mnt/d/crypto-market/cx
mr/cxmr-master/cxmr/target/debug/build/indexmap-62ff17e61db67c74/out/probe0.ll": Operation not permitted (os error 1)

error: aborting due to previous error

error: could not copy "/mnt/d/crypto-market/cxmr/cxmr-master/cxmr/target/debug/build/indexmap-62ff17e61db67c74/out/probe1.probe1.3a1fbbbh-cgu.0.rcgu.ll" to "/mnt/d/crypto-market/cx
mr/cxmr-master/cxmr/target/debug/build/indexmap-62ff17e61db67c74/out/probe1.ll": Operation not permitted (os error 1)

error: aborting due to previous error

warning: autocfg could not probe for `std`
error: could not copy "/mnt/d/crypto-market/cxmr/cxmr-master/cxmr/target/debug/build/indexmap-62ff17e61db67c74/out/probe2.probe2.3a1fbbbh-cgu.0.rcgu.ll" to "/mnt/d/crypto-market/cx
mr/cxmr-master/cxmr/target/debug/build/indexmap-62ff17e61db67c74/out/probe2.ll": Operation not permitted (os error 1)

error: aborting due to previous error

@crackcomm
Copy link
Author

It tries to copy a file which doesn't exist.

The directory does exist though.

$ ls -lh target/debug/build/indexmap-62ff17e61db67c74/out
total 0
-rwxrwxrwx 1 root root 0 Jul 30 21:07 probe0.ll
-rwxrwxrwx 1 root root 0 Jul 30 21:07 probe1.ll
-rwxrwxrwx 1 root root 0 Jul 30 21:07 probe2.ll

The root is WSL thingie I can't chown it.

@cuviper
Copy link
Member

cuviper commented Jul 30, 2020

All autocfg does is invoke rustc, so I'm afraid I'll have to deflect the problem there. You could try a workaround of setting a local CARGO_TARGET_DIR, as we did in rust-lang/rust#54216.

@crackcomm
Copy link
Author

This workaround works and is actually much faster, thanks.

@cuviper
Copy link
Member

cuviper commented Jul 30, 2020

Ok, great! If you want to chime in on that rust bug or file a new one, feel free, but I don't think there's anything we can really do in indexmap or autocfg to fix that.

@cuviper cuviper closed this as completed Jul 30, 2020
cuviper added a commit to cuviper/indexmap that referenced this issue Aug 6, 2020
If you know you need "std", e.g. relying on default `S = RandomState`,
then you can enable this feature to force it on, bypassing target
detection. This may help scenarios where `autocfg` fails detection, like
issue indexmap-rs#144, but at the very least it will speed up the build script by
avoiding extra `rustc` invocations.
cuviper added a commit to cuviper/indexmap that referenced this issue Sep 1, 2020
If you know you need "std", e.g. relying on default `S = RandomState`,
then you can enable this feature to force it on, bypassing target
detection. This may help scenarios where `autocfg` fails detection, like
issue indexmap-rs#144, but at the very least it will speed up the build script by
avoiding extra `rustc` invocations.
@Xiphoseer
Copy link

Just FYI: I got here via the rust-server generator of https://openapi-generator.tech. They depend on hyper = "0.13" => h2 = "0.2.2" => indexmap = "1.0" and produce a .cargo/config with

[build]
rustflags = [
    "-W", "missing_docs",  # detects missing documentation for public members
    "-W", "trivial_casts",  # detects trivial casts which could be removed
    "-W", "trivial_numeric_casts",  # detects trivial casts of numeric types which could be removed
    "-W", "unsafe_code",  # usage of `unsafe` code
    "-W", "unused_qualifications",  # detects unnecessarily qualified names
    "-W", "unused_extern_crates",  # extern crates that are never used
    "-W", "unused_import_braces",  # unnecessary braces around an imported item
    "-D", "warnings",
]

which causes

warning: autocfg could not probe for `std`
error: missing documentation for the crate
 --> <anon>:1:1
  |
1 | extern crate std as probe;
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: `-D missing-docs` implied by `-D warnings`

error: aborting due to previous error

with the same result (std not detected, missing 3rd generic parameter on IndexMap):

error[E0107]: this struct takes 3 generic arguments but 2 generic arguments were supplied
   --> C:\Users\<username>\.cargo\registry\src\gh.neting.cc-1ecc6299db9ec823\h2-0.2.7\src\proto\streams\store.rs:15:10
    |
15  |     ids: IndexMap<StreamId, SlabIndex>,
    |          ^^^^^^^^ --------  --------- supplied 2 generic arguments
    |          |
    |          expected 3 generic arguments
    |
note: struct defined here, with 3 generic parameters: `K`, `V`, `S`
   --> C:\Users\<username>\.cargo\registry\src\gh.neting.cc-1ecc6299db9ec823\indexmap-1.4.0\src\map.rs:303:12
    |
303 | pub struct IndexMap<K, V, S> {
    |            ^^^^^^^^ -  -  -
help: add missing generic argument
    |
15  |     ids: IndexMap<StreamId, SlabIndex, S>,
    |                                      +++

The solution for this case is to remove the -D warnings line.

IIUC this is essentially what https://rust-lang.github.io/rfcs/1193-cap-lints.html intends to avoid, so the solution would probably be to have autocfg add --cap-lints warn here.

@cuviper
Copy link
Member

cuviper commented Mar 22, 2022

@Xiphoseer can you open an issue on autocfg about --cap-lints?

For indexmap, you can now enable the std feature and it will bypass the check. You can do this as a workaround from your own crate, and it will affect the build for your entire dependency tree. Then 2.0 in #219 plans to stop using autocfg at all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants