Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tracking issue for not crashing on the rustc test suite #145

Open
Nadrieril opened this issue Apr 23, 2024 · 37 comments
Open

Tracking issue for not crashing on the rustc test suite #145

Nadrieril opened this issue Apr 23, 2024 · 37 comments
Labels
C-tracking-issue

Comments

@Nadrieril
Copy link
Member

Nadrieril commented Apr 23, 2024 •
edited
Loading

In #136 we added a script that runs charon on the rustc test suite and displays the results. As we'd like to support a large portion of the rust language, we track in this issue our results on these tests.

Our first aim is to never panic. Initial results:

   5269 expected-failure
   2585 failure
   3105 panic
   4207 success
     27 timeout

(Please don't post on this conversation, it's mean for automated posting)
@Nadrieril Nadrieril pinned this issue Apr 23, 2024
@Nadrieril Nadrieril changed the title Tracking issue for passing the rustc test suite Tracking issue for not crashing on the rustc test suite Apr 23, 2024
@AeneasVerif AeneasVerif locked as off-topic and limited conversation to collaborators Apr 23, 2024
@pnmadelaine pnmadelaine mentioned this issue Apr 23, 2024
Merged
@AeneasVerif AeneasVerif unlocked this conversation Apr 23, 2024
@github-actions GitHub Actions
Copy link

Summary of the results:
4673 expected-failure
2065 failure
1731 panic
3546 success
30 timeout
3148 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
4672 expected-failure
2060 failure
1731 panic
3556 success
25 timeout
3149 unsupported-build-settings

@github-actions GitHub Actions
Copy link

github-actions bot commented May 5, 2024

Summary of the results:
4692 expected-failure
2152 failure
1609 panic
3563 success
28 timeout
3149 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
4692 expected-failure
2150 failure
1608 panic
3565 success
29 timeout
3149 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
4500 expected-failure
1604 failure
1620 panic
4289 success
31 timeout
3149 unsupported-build-settings

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 5, 2024

Summary of the results:
5725 expected-failure
2122 failure
4531 success
29 timeout
3306 unsupported-build-settings

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 9, 2024

Summary of the results:
5740 expected-failure
2130 failure
4501 success
36 timeout
3306 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
5717 expected-failure
2057 failure
4606 success
26 timeout
3307 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
5706 expected-failure
2033 failure
4641 success
26 timeout
3307 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
5711 expected-failure
2017 failure
4654 success
23 timeout
3308 unsupported-build-settings

@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 7, 2024

Summary of the results:
5480 expected-failure
1442 failure
5451 success
35 timeout
3305 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
5506 expected-failure
1444 failure
5424 success
35 timeout
3304 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
5507 expected-failure
1446 failure
5423 success
32 timeout
3305 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
5510 expected-failure
1448 failure
5420 success
30 timeout
3305 unsupported-build-settings

1 similar comment
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 5, 2024

Summary of the results:
5510 expected-failure
1448 failure
5420 success
30 timeout
3305 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
5531 expected-failure
1456 failure
5392 success
29 timeout
3305 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
5531 expected-failure
1475 failure
5373 success
29 timeout
3305 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
5526 expected-failure
1471 failure
5385 success
26 timeout
3305 unsupported-build-settings

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 1, 2024

Summary of the results:
5381 expected-failure
1245 failure
5751 success
31 timeout
3305 unsupported-build-settings

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 8, 2024

Summary of the results:
5315 expected-failure
1140 failure
5922 success
31 timeout
3305 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
5274 expected-failure
997 failure
6105 success
34 timeout
3303 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
5275 expected-failure
997 failure
6106 success
31 timeout
3304 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
5275 expected-failure
1001 failure
6098 success
36 timeout
3303 unsupported-build-settings

@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 6, 2024

Summary of the results:
5169 expected-failure
780 failure
6425 success
36 timeout
3303 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
5163 expected-failure
764 failure
6448 success
35 timeout
3303 unsupported-build-settings

@Nadrieril Nadrieril mentioned this issue Oct 18, 2024
Merged
@github-actions GitHub Actions
Copy link

Summary of the results:
13 expected-failure
11 failure
5181 panic
7173 success
30 timeout
3305 unsupported-build-settings

@Nadrieril
Copy link
Member Author

Nadrieril commented Oct 21, 2024 •
edited
Loading

I might have been too lenient in my recent PRs: I think we're now treating rust errors as panics. See also #306.

@github-actions GitHub Actions
Copy link

Summary of the results:
15 expected-failure
11 failure
5174 panic
7174 success
35 timeout
3304 unsupported-build-settings

@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 3, 2024

Summary of the results:
15 expected-failure
11 failure
5164 panic
7186 success
33 timeout
3304 unsupported-build-settings

@sonmarcho
Copy link
Member

I just noticed we almost have 7.2k success now, to be compared with the 4.2k success from when we started tracking this issue :)

@github-actions GitHub Actions
Copy link

Summary of the results:
16 expected-failure
11 failure
5165 panic
7188 success
28 timeout
3305 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
16 expected-failure
11 failure
5164 panic
7187 success
30 timeout
3305 unsupported-build-settings

@Nadrieril Nadrieril mentioned this issue Nov 22, 2024
Merged
@github-actions GitHub Actions
Copy link

Summary of the results:
16 expected-failure
11 failure
5164 panic
7186 success
31 timeout
3305 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
15 expected-failure
11 failure
5164 panic
7186 success
32 timeout
3305 unsupported-build-settings

@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 1, 2024

Summary of the results:
16 expected-failure
11 failure
5164 panic
7186 success
31 timeout
3305 unsupported-build-settings

@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 8, 2024

Summary of the results:
15 expected-failure
11 failure
5163 panic
7183 success
38 timeout
3303 unsupported-build-settings

@github-actions GitHub Actions
Copy link

Summary of the results:
15 expected-failure
11 failure
5164 panic
7185 success
34 timeout
3304 unsupported-build-settings

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-tracking-issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Nadrieril @sonmarcho