Buildbot grammar #37825

dns2utf8 · 2016-11-17T10:19:14Z

This is the implementation of #28592.

rust-highfive · 2016-11-17T10:19:28Z

(rust_highfive has picked a reviewer for you, use r? to override)

est31 · 2016-11-17T11:25:24Z

Dose it checks the commit messages too?

dns2utf8 · 2016-11-17T12:51:52Z

I am sorry, I do not understand which checks do you mean?
As far as I can tell, the bots do not parse the commit messages.

alexcrichton · 2016-11-17T15:29:35Z

Thanks for the PR! In light of #37817 we're probably going to want to eventually migrate this to rustbuild at some point as well if we're interested in continually gating on it. For now I'll wait for this to go green and then looks good to me!

cc @brson, expanding our matrix of what's tested

alexcrichton · 2016-11-17T15:55:57Z

So thinking a bit about this bot, I wonder if we can perhaps try to speed up its execution time? We could take strategies like:

Use a system LLVM instead of our bundled version
Perhaps only compile the stage1 compiler, not all the way to stage2

Thoughts?

dns2utf8 · 2016-11-17T21:52:31Z

Good news: it works 😄

For the speedup, I think using the system llvm would be perfect.

I am not sure about not building the compiler because if I had broken something inside the compiler, how would I know when the compiler used for the checkes is still a good one?

Should I disable the ALLOW_PR=1 flag again? Because right now the rust-part of the verifyer gets builded with make tidy so breaking changes against the infrastructure are detected.

Running the grammar checks on every commit of every PR might be overkill. If I understand it correctly the script runs all tests as soon as bors is attempting to merge everything on the auto branch, yes?

Before we merge this, I guess I should switch back the ALLOW_PR=1 to the llvm-3.7 build?

alexcrichton · 2016-11-17T22:39:58Z

Oh installing llvm-3.9 is fine actually, should even get us a bit more coverage. But yeah before merging we'd want to turn ALLOW_PR=1 as we in theory just want this to run for merges, not all PRs.

The verify.rs script which this runs only needs three crates, syntax, syntax_pos, and rustc. In that sense we only need to build those three crates, not the entire compiler in three stages? We can also skip make tidy if necessary.

alexcrichton · 2016-11-18T15:32:40Z

Hm so this line seems suspicious:

writing 7877 files that did not yield the correct result with /tmp/obj//grammar/parser-lalr to parser-lalr.bad

Does that mean that we're actually differing on a bunch of files? Does this mean that the verification doesn't generate an error if a difference is detected?

shepmaster · 2016-11-19T14:28:59Z

I might be missing the joke, but it should be spelled "grammar", not "grammer" in the PR title.

dns2utf8 · 2016-12-01T13:54:00Z

I think the reason is bash:

if [ -z "42" ] ; then true; else false; fi

this returns false but this:

if [ -z "42" ] ; then true; elif [ -z "42" ]; then false else false; fi

does not

dns2utf8 · 2016-12-01T14:14:50Z

I found the bug in the shell script:

if [ -z "42" ] ; then echo 1; false; elif [ -z "42" ]; then echo 2; false fi; else echo 3; false; fi
                                                                          ^^^

this fi is required.
I do not know why bash behaves like this, but I think it will require a little more.

Fun fact, with this code there is no path to echo 2.5:

if [ -z "42" ] ; then echo 1; false; elif [ -z "42" ]; then echo 2; false else echo 2.5; false fi; else echo 3; false; fi

Do you have an idea how to solve this problem?

alexcrichton · 2016-12-01T16:55:37Z

Ah I was wondering what was wrong with that! Nice find :)

Wanna switch back the ALLOW_PR=1 and we can r+?

dns2utf8 · 2016-12-01T23:46:38Z

Allright I rebased to current master.

I reverted my changes to the .travis.yml script, because my bash handles elif diffrently from travises bash and it works as is: build with failing make check-grammar

With this rebase we are at 625 failing grammar tests.

Unforunalty the verify.rs is broken in the current master and my fix (#38029) did not merge yet.

That said I expect this PR to fail on the auto branch and, once it is merged, all following PRs until the grammar test suite is fixed. 😃

alexcrichton · 2016-12-01T23:57:51Z

Hm I'd prefer to not check in a matrix entry which fails tests. Could we set ALLOW_PR=1 temporarily and avoid merging until the tests are fixed on this PR? We could perhaps add whitelists with FIXMEs attached for test cases that need to be fixed.

dns2utf8 · 2016-12-01T23:59:57Z

src/grammar/testparser.py

    with open(filename, "w") as f:
        for p in bad[parser]:
+            print("bad test: {}".format(p))


I added this output because the output file will be lost with the container.

dns2utf8 · 2016-12-02T00:04:44Z

This works for me, but I may take some time and I would appreciate help with fixing the tests.

I guess in theory interestes people could also send me PRs with fixied tests directly and I could merge them into this PR.

alexcrichton · 2016-12-02T00:07:07Z

Oh yeah sure, we can definitely help out. Want to set ALLOW_PR=1 as part of this PR on that entry in the travis matrix, and then we can see the failures we're dealing with?

dns2utf8 · 2016-12-02T00:22:01Z

Cool 👍

dns2utf8 · 2016-12-02T11:06:40Z

I looked into some of the errors and it looks like some of them are the parser not understanding the new error format.

I tried to fix one, but I guess somebody with knowledge of bison could help.

alexcrichton · 2016-12-02T17:49:27Z

Hm this does raise an interesting question. If we enable and start gating on this then we're requiring that all changes and extensions to the grammar are implemented as well (or ignored). If we require changes, then this is essentially starting out RFC 1331. I'd just want to make sure that we're all on board with that and consciously making that step..

cc @rust-lang/compiler, this may make it slightly more difficult to extend the grammar in the future (the actual grammar has to be updated), but that seems like it may be a good thing. If that's true, is the current implementation of src/grammar something that we're willing to gate on?

bors · 2016-12-07T16:30:52Z

☔ The latest upstream changes (presumably #37817) made this pull request unmergeable. Please resolve the merge conflicts.

dns2utf8 · 2016-12-08T15:11:42Z

The make check-grammar and make check-lexer targes are gone with the new build infrastructure.

Is there a replacement yet?
I think switching to the new infrastructure will speedup the process when using --stage 1 or higher.

alexcrichton · 2016-12-09T00:12:06Z

The old infrastructure still exists temporarily with --disable-rustbuild but otherwise yeah it'll need to be migrated into rustbuild.

- Made with ubuntu 16.10 and python3 - Include antlr-complete.jar - Switch to system llvm 3.9

…ation

sanxiyn · 2017-01-01T03:49:10Z

You may want to update Dockerfile as in #38632.

bors · 2017-01-13T05:46:46Z

☔ The latest upstream changes (presumably #38748) made this pull request unmergeable. Please resolve the merge conflicts.

alexcrichton · 2017-01-17T21:09:56Z

Closing due to inactivity, but feel free to resubmit with a rebase!

rust-highfive assigned alexcrichton Nov 17, 2016

dns2utf8 mentioned this pull request Nov 17, 2016

Automatically run grammar tests (grammar bot) #28592

Closed

dns2utf8 force-pushed the buildbot_grammer branch from 0651a00 to 98770aa Compare November 17, 2016 14:16

dns2utf8 changed the title ~~Buildbot grammer~~ Buildbot grammar Nov 29, 2016

dns2utf8 force-pushed the buildbot_grammer branch 2 times, most recently from b578a8d to cc35c89 Compare December 1, 2016 23:21

dns2utf8 commented Dec 1, 2016

View reviewed changes

dns2utf8 force-pushed the buildbot_grammer branch from 26b222f to a198afb Compare December 2, 2016 00:09

dns2utf8 force-pushed the buildbot_grammer branch from 91e62b3 to ee785b9 Compare December 2, 2016 11:02

dns2utf8 force-pushed the buildbot_grammer branch from ee785b9 to aa76e62 Compare December 4, 2016 00:22

dns2utf8 force-pushed the buildbot_grammer branch from aa76e62 to 2685c66 Compare December 8, 2016 14:40

dns2utf8 added 8 commits December 27, 2016 11:12

Add a new Dockerfile for the grammer-check bot

048570a

- Made with ubuntu 16.10 and python3 - Include antlr-complete.jar - Switch to system llvm 3.9

Add env to travis

78d6f08

Return with error

1c9fffa

Add explicit int to unsigned

1a7d3e4

Forward declare strdup from <string.h> to avoid warning during compil…

b759443

…ation

Temporarily switch ALLOW_PR=1 to grammartest

2b4c9fd

Add section for running a single test

d058d41

First try

819ab04

dns2utf8 force-pushed the buildbot_grammer branch from 2685c66 to 819ab04 Compare December 27, 2016 10:12

alexcrichton closed this Jan 17, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Buildbot grammar #37825

Buildbot grammar #37825

dns2utf8 commented Nov 17, 2016

rust-highfive commented Nov 17, 2016

est31 commented Nov 17, 2016

dns2utf8 commented Nov 17, 2016 •

edited

Loading

alexcrichton commented Nov 17, 2016

alexcrichton commented Nov 17, 2016

dns2utf8 commented Nov 17, 2016 •

edited

Loading

alexcrichton commented Nov 17, 2016

alexcrichton commented Nov 18, 2016

shepmaster commented Nov 19, 2016

dns2utf8 commented Dec 1, 2016

dns2utf8 commented Dec 1, 2016

alexcrichton commented Dec 1, 2016

dns2utf8 commented Dec 1, 2016

alexcrichton commented Dec 1, 2016

dns2utf8 Dec 1, 2016

dns2utf8 commented Dec 2, 2016

alexcrichton commented Dec 2, 2016

dns2utf8 commented Dec 2, 2016

dns2utf8 commented Dec 2, 2016

alexcrichton commented Dec 2, 2016

bors commented Dec 7, 2016

dns2utf8 commented Dec 8, 2016

alexcrichton commented Dec 9, 2016

sanxiyn commented Jan 1, 2017

bors commented Jan 13, 2017

alexcrichton commented Jan 17, 2017

Buildbot grammar #37825

Buildbot grammar #37825

Conversation

dns2utf8 commented Nov 17, 2016

rust-highfive commented Nov 17, 2016

est31 commented Nov 17, 2016

dns2utf8 commented Nov 17, 2016 • edited Loading

alexcrichton commented Nov 17, 2016

alexcrichton commented Nov 17, 2016

dns2utf8 commented Nov 17, 2016 • edited Loading

alexcrichton commented Nov 17, 2016

alexcrichton commented Nov 18, 2016

shepmaster commented Nov 19, 2016

dns2utf8 commented Dec 1, 2016

dns2utf8 commented Dec 1, 2016

alexcrichton commented Dec 1, 2016

dns2utf8 commented Dec 1, 2016

alexcrichton commented Dec 1, 2016

dns2utf8 Dec 1, 2016

Choose a reason for hiding this comment

dns2utf8 commented Dec 2, 2016

alexcrichton commented Dec 2, 2016

dns2utf8 commented Dec 2, 2016

dns2utf8 commented Dec 2, 2016

alexcrichton commented Dec 2, 2016

bors commented Dec 7, 2016

dns2utf8 commented Dec 8, 2016

alexcrichton commented Dec 9, 2016

sanxiyn commented Jan 1, 2017

bors commented Jan 13, 2017

alexcrichton commented Jan 17, 2017

dns2utf8 commented Nov 17, 2016 •

edited

Loading

dns2utf8 commented Nov 17, 2016 •

edited

Loading