Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rustdoc output is not deterministic for failed doctests #76442

Closed
jyn514 opened this issue Sep 7, 2020 · 3 comments · Fixed by #76445
Closed

Rustdoc output is not deterministic for failed doctests #76442

jyn514 opened this issue Sep 7, 2020 · 3 comments · Fixed by #76445
Assignees
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc C-bug Category: This is a bug. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.

Comments

@jyn514
Copy link
Member

jyn514 commented Sep 7, 2020

This is causing spurious errors in the testsuite.

Details
failures:

---- [ui] rustdoc-ui\failed-doctest-output.rs stdout ----
\rustdoc-ui\failed-doctest-output.rs
\rustdoc-ui\failed-doctest-output.rs
\rustdoc-ui\failed-doctest-output.rs
thread 'main' panicked at 'Some tests failed', src\tools\compiletest\src\main.rs:354:22
\rustdoc-ui\failed-doctest-output.rs
\rustdoc-ui\failed-doctest-output.rs
\rustdoc-ui\failed-doctest-output.rs
\rustdoc-ui\failed-doctest-output.rs
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
\rustdoc-ui\failed-doctest-output.rs
diff of stdout:

1	
2	running 2 tests
-	test $DIR/failed-doctest-output.rs - OtherStruct (line 21) ... FAILED
4	test $DIR/failed-doctest-output.rs - SomeStruct (line 11) ... FAILED
+	test $DIR/failed-doctest-output.rs - OtherStruct (line 21) ... FAILED
5	
6	failures:
7	

-	---- $DIR/failed-doctest-output.rs - OtherStruct (line 21) stdout ----
-	error[E0425]: cannot find value `no` in this scope
-	  --> $DIR/failed-doctest-output.rs:22:1
-	   |
-	LL | no
-	   | ^^ not found in this scope
-	
-	error: aborting due to previous error
-	
-	For more information about this error, try `rustc --explain E0425`.
-	Couldn't compile the test.
19	---- $DIR/failed-doctest-output.rs - SomeStruct (line 11) stdout ----
20	Test executable failed (exit code 101).
21	

30	note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
31	
32	
+	---- $DIR/failed-doctest-output.rs - OtherStruct (line 21) stdout ----
+	error[E0425]: cannot find value `no` in this scope
+	  --> $DIR/failed-doctest-output.rs:22:1
+	   |
+	LL | no
+	   | ^^ not found in this scope
+	
+	error: aborting due to previous error
+	
+	For more information about this error, try `rustc --explain E0425`.
+	Couldn't compile the test.
33	
34	failures:
35	    $DIR/failed-doctest-output.rs - OtherStruct (line 21)

Originally posted by @mati865 in #76423 (comment)

@jyn514 jyn514 added A-testsuite Area: The testsuite used to check the correctness of rustc C-bug Category: This is a bug. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Sep 7, 2020
@Mark-Simulacrum
Copy link
Member

We should probably just remove/ignore these tests if they're causing spurious failures at a relatively high rate.

@jyn514
Copy link
Member Author

jyn514 commented Sep 7, 2020

Maybe instead of removing them we could just split them into separate files? That way the order doesn't have to be deterministic, but we're still testing the output is correct.

@mati865
Copy link
Contributor

mati865 commented Sep 9, 2020

Somehow this started to appear quite often recently, usually on i686-mingw-1 builder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc C-bug Category: This is a bug. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants