Update and improve test262 integration #372
Conversation
1. Update the test262 submodule to the latest version This brings more test cases to already enabled folders, but also requires more exclusions of currently failing checks. 2. Improve test262 runner (Test262SuiteTest class) * Allow arbitrary names of expected exceptions and support the new structure of the 'negative' section in tests' metadata * Distinguish exception's phase, early vs runtime, for negative tests * Support for a 'raw' flag: no harness files, implies 'noStrict' * Add filtering by the 'features' lists * Exclude tests with a 'module' flag, as this isn't supported by Rhino * Exclude tests with an 'async' flag, which isn't yet supported by the Test262SuiteTest itself * Couple of other smaller fixes 3. Enable more tests, mostly from the 'language' folder This brings the total number of passing tests to 44k.
|
I seem to be getting an error parsing some of the metadata, below: Do you have a chance to look at this? java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map |
|
Sure. It’s a bit late here already, I’ll check this tomorrow. But as a guess: have you updated the test262 submodule when you got this exception? (I always forget that |
|
yes, that was it -- I did a "sync" and not "update." Looks good now. Thanks!
…On Tue, Jan 16, 2018 at 1:06 PM, Ivan Vyshnevskyi ***@***.***> wrote:
Sure. It’s a bit late here already, I’ll check this tomorrow.
But as a guess: have you updated the test262 submodule when you got this
exception? (I always forget that git checkout isn’t enough and you have
to do git submodule --update.)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#372 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAf0awWfc5Pb7TOM-hRAMIxOrP5mtpj-ks5tLQ9SgaJpZM4RMBjA>
.
|
|
Great! Thanks for merging! |
Update the test262 submodule to the latest version
This brings more test cases to already enabled folders, but also
requires more exclusions of currently failing checks. I wanted to
avoid accidentally excluding passing files, so exclusion lists for
some directories/features are quiet large. At some point I'd like
to be able to generate the
test262.propertiesautomatically, butwith more logging manual edits are good enough for now.
Improve test262 runner (Test262SuiteTest class)
structure of the 'negative' section in tests' metadata
the Test262SuiteTest itself
Enable more tests, mostly from the 'language' folder
This brings the total number of passing test cases to 45k, generated from 7779 files.
The execution time incises, but not by that much. Before it took around 1min 15s
and now it's about 1min 50s on my laptop.
Parameters generation (reading
test262.properties, recursively walking folders,reading files and parsing their YAML headers) takes slightly more than 2s. For
26k tests (4454 files) in master it takes 1.4s. I believe there are some possible
improvements, but it's such a small component I didn't spent much time on it.