Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fuzz v3 #1697

Merged
merged 1 commit into from
Mar 31, 2017
Merged

Fuzz v3 #1697

merged 1 commit into from
Mar 31, 2017

Conversation

pvdz
Copy link
Contributor

@pvdz pvdz commented Mar 26, 2017

Various improvements to the fuzzer.

This is probably my last (major) iteration of the fuzzer for now.

@pvdz
Copy link
Contributor Author

pvdz commented Mar 26, 2017

The major bottleneck in the number of cycles per second is without doubt the minification process. Is there a particular way to speed that up? Is that desirable at all? Or is perf of the minifier a low prio?

@alexlamsl
Copy link
Collaborator

Quick tip: use Node.js 4.x instead of 7.x will give you 100% speed boost

Performance is not critical in the sense that I only focus on pathological cases, i.e. so slow it makes uglify-js essentially unusable. Once the dust settles, I can always do a bit of profiling and see what can be done.

But before that, I'll need to update this test/ufuzz.js to test a battery of minify() options. Also, might be good once it found an error case, drill down to see which combination of options makes a difference - that would help with investigation.

test/ufuzz.js Outdated
@@ -440,7 +502,7 @@ for (var round = 0; round < num_iterations; round++) {
var beautify_result = run_code(beautify_code);

try {
var uglify_code = minify(beautify_code, {
var uglify_code = minify(original_code, {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC there was a prior discussion involving @kzc and we ended up keeping this unchanged.

If we are going to change this, might as well get rid of beautify_result and move computation of beautify_code into log(). That way you should get a nice speed-up.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect it to test both the beautifier and the minifier this way. I assume they use completely different code paths after parsing. Parsing doesn't appear to be a bottleneck because the beautification is about as fast as I'd expect it and I expect that to use the same parser.

I don't mind either way, you have to use the tool so it's really up to you.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing beautify on input revealed a couple of bugs. It would be nice if were an option, disabled by default to speed up fuzzing.

When there's a failure can both the minified output code as well as beautified output code be logged? That way it's easier to diagnose. We have to beautify the output anyway.

@pvdz
Copy link
Contributor Author

pvdz commented Mar 26, 2017

Also, might be good once it found an error case, drill down to see which combination of options makes a difference - that would help with investigation.

I like that. If there's a bounded number of combinations to test then it could easily be done and show you a nice table of "works v doesnt work" kind of result. Should not be too difficult to make and fairly separate from the generation code.

In such case you'd probably split the generation to its own file etc.

@pvdz
Copy link
Contributor Author

pvdz commented Mar 26, 2017

fwiw, some numbers on my machine:

MAX_GENERATED_TOPLEVELS_PER_RUN= 5 , MAX_GENERATION_RECURSION_DEPTH= 5
generating: 0.981ms
code len= 857 , running 1: 3.495ms
beautifying: 20.470ms
running 2: 2.320ms
uglifying: 42.727ms
running 3: 0.992ms
<cycle>: 72.542ms
MAX_GENERATED_TOPLEVELS_PER_RUN= 10 , MAX_GENERATION_RECURSION_DEPTH= 5
generating: 1.354ms
code len= 2887
running 1: 4.698ms
beautifying: 33.981ms
running 2: 2.096ms
uglifying: 106.762ms
running 3: 1.620ms
<cycle>: 152.387ms
MAX_GENERATED_TOPLEVELS_PER_RUN= 2 , MAX_GENERATION_RECURSION_DEPTH= 15
generating: 1.068ms
code len= 14831
running 1: 8.265ms
beautifying: 25.972ms
running 2: 7.232ms
uglifying: 301.376ms
running 3: 3.060ms
<cycle>: 347.240ms
MAX_GENERATED_TOPLEVELS_PER_RUN= 50 , MAX_GENERATION_RECURSION_DEPTH= 5
generating: 5.092ms
code len= 167313
running 1: 61.268ms
beautifying: 268.447ms
running 2: 57.770ms
uglifying: 2173.699ms
running 3: 40.059ms
<cycle>: 2606.751ms
MAX_GENERATED_TOPLEVELS_PER_RUN= 5 , MAX_GENERATION_RECURSION_DEPTH= 20
generating: 4.429ms
codel len= 36034
running 1: 9.025ms
beautifying: 136.061ms
running 2: 5.971ms
uglifying: 396.245ms
running 3: 2.237ms
<cycle>: 556.290ms
MAX_GENERATED_TOPLEVELS_PER_RUN= 50 , MAX_GENERATION_RECURSION_DEPTH= 20
generating: 133.457ms
codel len= 3908819
running 1: 778.072ms
beautifying: 4721.689ms
running 2: 752.243ms
uglifying: 38221.894ms
running 3: 231.465ms
<cycle>: 44841.879ms

@alexlamsl
Copy link
Collaborator

alexlamsl commented Mar 26, 2017

Once I've rebased #1460, you can see if it gives you better performance. It handles sequences more efficiently, which I imagine is part of the problem when the fuzzer generates a large body of statements for instance.

Edit: rebased

@kzc
Copy link
Contributor

kzc commented Mar 27, 2017

When I tried to run the script with this PR:

test/ufuzz.js:484
        createTopLevelCodes(rng(MAX_GENERATED_TOPLEVELS_PER_RUN) + 1) +
                                ^

ReferenceError: MAX_GENERATED_TOPLEVELS_PER_RUN is not defined

test/ufuzz.js Outdated
@@ -122,6 +129,8 @@ var TYPEOF_OUTCOMES = [
'crap' ];

var FUNC_TOSTRING = 'Function.prototype.toString=function(){return"function(){}"};';
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kzc I shall wait on this PR before putting in #1688 (comment) to avoid merge conflicts.

@pvdz
Copy link
Contributor Author

pvdz commented Mar 27, 2017

I shall wait on this PR before putting

That's fine. My stuff affects one file so I can easily rebase it. Go ahead.

@kzc
Copy link
Contributor

kzc commented Mar 27, 2017

I tried testing the fuzzer in this PR and noticed that each iteration is several times slower than the existing version. I assume it's because of the complexity of the test produced by each iteration increased?

Disabling beautify on the input code by default should speed it up a bit. Having an option to enable it would be nice.

@pvdz
Copy link
Contributor Author

pvdz commented Mar 27, 2017

I tried testing the fuzzer in this PR and noticed that each iteration is several times slower than the existing version. I assume it's because of the complexity of the test produced by each iteration increased?

It's quite exponentially tied to the size of the generated code. Try lowering the MAX_GENERATED_TOPLEVELS_PER_RUN and in particular MAX_GENERATION_RECURSION_DEPTH.

As the numbers above should tell you the beautifier doesn't add a lot to the runtime, most of the time is spent in the minifier. So yeah, we can make that optional, but runtime perf won't be noticeably affected by turning it off.

@pvdz
Copy link
Contributor Author

pvdz commented Mar 27, 2017

Mind you, it could also be caused by new additions that trigger slow paths in the minifier. I didn't profile it. The numbers above tell me that the generating part is fine (just like the beautifier).

@kzc
Copy link
Contributor

kzc commented Mar 27, 2017

If the input code beautifier in the fuzzer on master is disabled it increases fuzzer performance by 16%.

@alexlamsl
Copy link
Collaborator

@kzc I think @qfox is seeing minify() slowing down greatly with this PR due to the bigger test code the fuzzer now generates, which in turn beginning to dominate the runtime.

@qfox as discussed in #1700 (comment), I think deeper code generations may not bring about the best efficiency in catching issues. So may be we can back off a bit and avoid this performance issue altogether?

@kzc
Copy link
Contributor

kzc commented Mar 27, 2017

Try lowering the MAX_GENERATED_TOPLEVELS_PER_RUN and in particular MAX_GENERATION_RECURSION_DEPTH

Indeed - using these settings I was able to get 140 iterations per second for this PR, compared to 125 on master:

+var MAX_GENERATED_TOPLEVELS_PER_RUN = 1;
 var MAX_GENERATED_FUNCTIONS_PER_RUN = 1;
-var MAX_GENERATION_RECURSION_DEPTH = 15;
+var MAX_GENERATION_RECURSION_DEPTH = 3;
 var INTERVAL_COUNT = 100;
+var loops = 0;
+var funcs = 0;

By the way, I had to declare a few variables to get this script to run without error: MAX_GENERATED_TOPLEVELS_PER_RUN, loops and funcs.

@pvdz
Copy link
Contributor Author

pvdz commented Mar 27, 2017

You should run with V to see what's actually being generated. I think with depth=3 you'll find the functions rather basic. Which explains the 140/sec ;)

I just scrubbed the options a bit so it'll be -V now. Just run -? for the rest.

@pvdz
Copy link
Contributor Author

pvdz commented Mar 27, 2017

I think deeper code generations may not bring about the best efficiency in catching issues. So may be we can back off a bit and avoid this performance issue altogether?

@alexlamsl yeah that makes sense. I've made the recursion depth optional. You can play around (I suggest running with -V to see what's generated) to get to an acceptable runtime :)

@pvdz
Copy link
Contributor Author

pvdz commented Mar 27, 2017

(updated, rebased)

@pvdz
Copy link
Contributor Author

pvdz commented Mar 27, 2017

@alexlamsl btw that toString code doesn't seem very effective;

original result:
[ 100, [Function: a], 0 ]

uglified result:
[ 100, [Function: n], 0 ]

!!!!!! Failed...

@pvdz
Copy link
Contributor Author

pvdz commented Mar 27, 2017

As an addendum to the recursion depth (and amount of toplevels generated); since the nature of a fuzzer is very probabilistic generally more means better. There is definitely merit in running with a low recursion value but if that runs stable for a few thousand cycles then a higher recursion depth allows for randomly generating code that refers to other generated variables, weird constructions, etc. With -r 1 you'll never get a try inside a switch for example. It'll run fast, but definitely miss some important cases. Again, it's a good canary to run with a low recursion value but you'll also want to run with a higher number (10 ~ 20). That's all :)

@kzc
Copy link
Contributor

kzc commented Mar 27, 2017

Can you post some fuzzed generated code that is producing output with [Function: a]?

test/ufuzz.js Outdated

function run_code(code) {
var stdout = "";
var original_write = process.stdout.write;
process.stdout.write = function(chunk) {
process.stdout.write = function (chunk) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I appreciate the updating of two-space to four-space indentation for consistency, can we avoid these extraneous whitespaces getting inserted in the process?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, can you be more specific? Your comment is ambiguous to me. Did you mean;

  • this particular line that has process
  • just code according to the style guide already
  • dont do this kind of whitespace-only update in the future
  • d: something else?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(FWIW I've updated my editor for this project to use 4 instead of 2 tabs, which is the biggest reason for the fluctuating indents, so those shouldn't appear anymore. If there's other style guide rules to adhere I'll configure them for sure. I can also replace this commit with those style changes instead. Or just drop the commit if you prefer it that way. It's just an automatic beautification really.)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry about the confusion.

What I meant in this particular case is don't change any whitespaces except for the indentation.

There are a few other places where this happened as well.

@pvdz
Copy link
Contributor Author

pvdz commented Mar 27, 2017

@kzc took a while :)

//=============================================================
// !!!!!! Failed...
// original code
//
var a = 100, b = 10, c = 0;
try {(a++); } catch (Math_3) { {{var arguments_7 = ( - b), Math_3 = (b = a);}for (var brake8 = 5; ((--b) + new function(){ 
return Math;
}*(a++) + (typeof undefined_9 != "number")) && brake8 > 0; --brake8)var bar;}var eval = (a++) + (delete b), Math_3_12 = (c = c + 1) + undefined; } finally { (void a)>>>(--b) + ((c = c + 1) + (b |= a))||(delete b);if (new function(){ 
return Math;
})c = c + 1; }try {if ((a++) + !function Math(){}){var brake18 = 5; while (((c = c + 1) + (--b) + ( - b)?(!function Math_3_12_19(){}):(new function(){ 
return Math;
})) && --brake18 > 0)var foo_21 = (--b) + (1 === 1 ? a : b);} else {var brake22 = 5; while (((a++) + +function (){}) && --brake22 > 0)var eval_24 = 0x26.toString()?((a--)):((1 === 1 ? a : b)), eval_25 = new function(){ 
new function(){ 
return Math;
}
return Math;
};} } catch (foo_26) { {(--b) + new function(){ 
(--b) + (delete b)
return Math;
};var undefined_9 = (a++) + (0 === 1 ? a : b);(--/* ignore */b);;}var bar_33; }

//$$$$$$$$$$$$$$

function f0(undefined_9_34){new function(){ 
(c = c + 1) + ((c = c + 1) + ((a++) + (a++)))
return Math;
};{ /*2*/ return
(--b)}}
f0();


//$$$$$$$$$$$$$$

{var brake37 = 5; while (((--b) + ((--b) + ((c = c + 1) + ((--b) + (0 === 1 ? a : b)?((a++) + (typeof bar_33_38 === "boolean")):((c = c + 1) + new function(){ 
(b = a)
return Math;
}))))) && --brake37 > 0)(--/* ignore */b);}(function a(){if (new function(){ 
(--b)
return Math;
}){var brake42 = 5; do {(a++) + new function(){ 
return Math;
}^(a++) + (a/* ignore */++);} while (((--b) + (a++)) && --brake42 > 0);} else {!function f1(arguments){{!function f2(Math_3_12_19_46){;}
((c = 1 + c, 24 .toString()))}{!function f3(Math_3_12_48){;}
((c = 1 + c, []))}}
((a--))}c = c + 1;for (var brake50 = 5; ((a++) + (b = a)) && brake50 > 0; --brake50){var brake51 = 5; while (((c = c + 1) + ((--b) + (typeof arguments_7 === "number"))?(((typeof c === "unknown"))):((--b) + (typeof foo_21_52))) && --brake51 > 0)var bar_33_38_54 = (--b) + new function(){ 
+function Math_3_12_19_55(){(c = 1 + c, -0);(c = 1 + c, Infinity);(c = 1 + c, 23..toString());}
return Math;
};}try {for (var brake60 = 5; ((c = c + 1) + (++/* ignore */a)) && brake60 > 0; --brake60){!function f4(eval_24_62){if ((c = 1 + c, 25. ))(c = 1 + c, 0); else (c = 1 + c, null);{ /*1*/ return (c = 1 + c, ([,0].length === 2))}}
((c = 1 + c, 25. )?((c = 1 + c, 23..toString())):((c = 1 + c, true))|(c = c + 1) + (--(b)))} } catch (eval_24_67) { { /*1*/ return (c = c + 1) + (a++)}var Infinity_70 = (--b) + (((c = c + 1) + +function NaN_71(){;})); }})();

//$$$$$$$$$$$$$$

console.log([a, b, c]);


//-------------------------------------------------------------
// uglified code
//
function f0(n){new function(){c+=1,c+=1,a++,a++,Math}}var a=100,b=10,c=0;try{a++}catch(n){for(var arguments_7=-b,Math_3=b=a,brake8=5;--b+new function(){return Math}*a+++("number"!=typeof undefined_9)&&brake8>0;--brake8)var bar;var eval=a+++delete b,Math_3_12=(c+=1)+void 0}finally{void 0>>>--b+((c+=1)+(b|=a))||delete b,new function(){return Math}&&(c+=1)}try{if(a+++!function(){})for(var brake18=5;((c+=1)+--b-b?!function(){}:new function(){return Math})&&--brake18>0;)var foo_21=--b+a;else for(var brake22=5;a+++ +function(){}&&--brake22>0;)var eval_24=38..toString()?a--:a,eval_25=new function(){return new function(){Math},Math}}catch(n){--b,new function(){--b,delete b,Math};var undefined_9=a+++b;--b;var bar_33}f0();for(var brake37=5;--b+(--b+((c+=1)+(--b+b?a+++("boolean"==typeof bar_33_38):(c+=1)+new function(){return b=a,Math})))&&--brake37>0;)--b;!function n(){if(new function(){return--b,Math}){var a=5;do{n++,new function(){Math},n++,n++}while(--b+n++&&--a>0)}else!function(arguments){c=1+c,24..toString(),c=1+c}(n--);c+=1;for(var t=5;n+++(b=n)&&t>0;--t)for(var e=5;((c+=1)+(--b+("number"==typeof arguments_7))?"unknown"==typeof c:--b+typeof foo_21_52)&&--e>0;)--b,new function(){Math};try{for(var r=5;(c+=1)+ ++n&&r>0;--r)!function(n){c=1+c,c=1+c,c=1+c,[,0].length}((c=1+c,c=1+c,23..toString()))}catch(a){return(c+=1)+n++}}(),console.log([a,b,c]);


original result:
[ 109, [Function: a], 37 ]

uglified result:
[ 109, [Function: n], 37 ]

!!!!!! Failed...

@kzc
Copy link
Contributor

kzc commented Mar 27, 2017

[Function: a] was not produced by Function.prototype.toString:

$ node -e 'function a(){}; console.log(a); console.log(a.toString());'
[Function: a]
function a(){}

@kzc
Copy link
Contributor

kzc commented Mar 27, 2017

Any idea how to override how console.log() emits the following?

$ node -e 'console.log(function a(){})'
[Function: a]

If not, we have to disable mangle.

@alexlamsl
Copy link
Collaborator

Hey guys, I already did that in #1688, but only if you pass it a function directly 😉

https://github.com/mishoo/UglifyJS2/pull/1688/files#diff-723be892ae46591e4816c74a1fe0cf21R137

So if you don't mind, don't pass it an array like console.log([a, b, c]) and just give me console.log(a, b, c), may be? 👻

@kzc
Copy link
Contributor

kzc commented Mar 27, 2017

It's valueOf() I think:

$ node -e 'console.log(function a(){}.valueOf())'
[Function: a]

Can we override that in Function.prototype?

@kzc
Copy link
Contributor

kzc commented Mar 27, 2017

@alexlamsl

  •                    return typeof arg == "function" ? "[Function]" : arg;
    

Could you change it to:

+                        return typeof arg == "function" ? arg.toString() : arg;

@alexlamsl
Copy link
Collaborator

So there is one solution to the NaN / Infinity issue - do what lib/output.js already does to AST_Undefined, i.e. always output void 0

So that would be 0/0 and 1/0 respectively.

(null is a reserved keyword so we won't suffer the same issue.)

@alexlamsl
Copy link
Collaborator

OT: I wonder if we can get rid of this now...

https://github.com/mishoo/UglifyJS2/blob/master/lib/output.js#L657-L663

@kzc
Copy link
Contributor

kzc commented Mar 29, 2017

OT: I wonder if we can get rid of this now...
https://github.com/mishoo/UglifyJS2/blob/master/lib/output.js#L657-L663

Workaround not needed for Safari 9 which I'm using.
2013 bug report date is probably Safari 7 which has 0.13% market use.

https://en.wikipedia.org/wiki/Safari_version_history#Mac
https://data.gov.uk/data/site-usage?month=2017-03#browsers_versions

@kzc
Copy link
Contributor

kzc commented Mar 29, 2017

Don't get rid of this one - probably still affects 25% of Safari users:

AST_Function.DEFMETHOD("next_mangled", function(options, def){
    // #179, #326
    // in Safari strict mode, something like (function x(x){...}) is a syntax error;
    // a function expression's argument cannot shadow the function expression's name

@kzc
Copy link
Contributor

kzc commented Mar 29, 2017

Considering IE8 was released in 2009, I think we should probably keep both of these Safari workarounds.

@kzc
Copy link
Contributor

kzc commented Mar 29, 2017

I am not aware of a general correct solution to the +function(){} Function.prototype.toString side effect problem.

The naïve implementation of Function.prototype.toString returning the same string for all functions "works" in the sense that there are no runtime errors in tests, but it could mask all manner of other bugs when comparing the string representation of functions.

@alexlamsl
Copy link
Collaborator

@kzc indeed - I see it as a choice between false positives (currently) or false negatives (same string).

Actually, come to think of it, my naïve implementation at first was me not wanting to restart the tests manually whenever it hits a false positive. But we can workaround that by test/ufuzz.js continuing to run beyond a failed test, since it's not like we have anything unrecoverable after such an event.

That way, combined with my proposed enhancement of singling out option flags that cause a given test failure, should make life easier while still maintaining the best coverage.

@kzc
Copy link
Contributor

kzc commented Mar 29, 2017

we can workaround that by test/ufuzz.js continuing to run beyond a failed test, since it's not like we have anything unrecoverable after such an event.

True.

combined with my proposed enhancement of singling out option flags that cause a given test failure, should make life easier while still maintaining the best coverage

Yeah, it would reduce the burden of isolating the problem.

@kzc
Copy link
Contributor

kzc commented Mar 29, 2017

Additionally, perhaps after a bad test case is found the original code AST could be scanned for instances of [+-~]AST_Function and issue a warning. (Not to be confused with a function expression call which is different.)

@pvdz
Copy link
Contributor Author

pvdz commented Mar 29, 2017

The latest commit to the fuzzer has been running without a problem or bug for over a million iterations. Maybe it's time to squash and merge?

@kzc
Copy link
Contributor

kzc commented Mar 29, 2017

continuing to run beyond a failed test

If tests continue beyond a failure please remember to have a non-zero exit code (when max iterations specified).

@alexlamsl
Copy link
Collaborator

@kzc I think it should stop by default, and a flag to enable it to run beyond failures.

@qfox as long as you are happy with it, go right ahead 😉

@kzc
Copy link
Contributor

kzc commented Mar 29, 2017

the fuzzer has been running without a problem or bug for over a million iterations

Impressive. Well done @qfox and @alexlamsl!

@alexlamsl
Copy link
Collaborator

@qfox would you mind incorporating these two line changes?

        new vm.Script(FUNC_TOSTRING + code).runInNewContext({
            console: {
                log: function() {
                    return console.log.apply(console, [].map.call(arguments, function(arg) {
-                        return typeof arg == "function" ? "[Function]" : arg;
+                        return typeof arg == "function" ? arg.toString() : arg;
                    }));
                }
            }
-        }, { timeout: 5000 });
+        }, { timeout: 30000 });
        return stdout;

First one is #1697 (comment), while second one just bumps the execution timeout - if a 4.5GHz core can't run the test code within 5 seconds, I think it's safe to say we need more time 😉

@kzc
Copy link
Contributor

kzc commented Mar 30, 2017

I guess the only remaining ES5 features not in the fuzzer are:

  • for-in loops
  • Random Object literals
  • setters and getters
  • Random Arrays
  • instanceof
  • new (on variables)

Anything else I missed?

@pvdz
Copy link
Contributor Author

pvdz commented Mar 31, 2017

Btw if your test case really needs 30 seconds to run you need to use a lower -r or -m param, a faster computer, or really start thinking about optimizing the code. Also, if you can only run one test every so many seconds you maybe may as well not fuzz.

@alexlamsl
Copy link
Collaborator

@qfox I agree, though that 30 second is just me getting this check out of the way, but not removing it totally in case something does get stuck in an infinite loop.

And BTW, systems with ARM processors do exist 😉

@alexlamsl
Copy link
Collaborator

alexlamsl commented Mar 31, 2017

In addition, that 30 second is saying "we might get a rare outliner which takes a long time to run". Even if it means say 0.01%, I certainly don't want to restart the fuzzer every ~10,000 runs just because of a false positive.

@pvdz
Copy link
Contributor Author

pvdz commented Mar 31, 2017

Anything else I missed?

  • property access (dynamic, keywords, __proto__)
  • function calls (other than the simple declare-and-call cases)
  • prototype stuff
  • defineproperty stuff (maybe, not sure if this can screw up)
  • eval/with traps
  • getters/setters
  • call/apply ?

mmm can't think of anything else right now

@pvdz
Copy link
Contributor Author

pvdz commented Mar 31, 2017

And BTW, systems with ARM processors do exist

Sure, but you wouldn't run this fuzzer on them. That's what super computers are built for ;)

Fix bug where a `throw` was generated without expression

Reenable try/catch/finally and fix them up

Skip serialization errors

Allow function decl in other funcs but not in blocks etc

Rename function to be more appropriate

Fix global functions not getting certain names

Make the canaries more likely to appear as expressions

Add a silly rounding edge case

Add a new canary, `c`, which should only ever be incremented

Refactoring

Fix (another) iife not actually being invoked

When a statement hits recursion max return an expression instead of `;`

When a expression hits recursion max also inc `c`

Generate global code as well as function code

Also fixes some argument juggling related bugs.
No longer reduces the recursion max when generating sub functions.
Generates a function arg.

Add used names to var name pool while in that scope

This is a little wonky, possibly a hack, but since it's synchronous code I think it's alright to do this. The alternative is to slice the varnames array and juggle them through almost all the generator functions and there are various reasons why this patch is a better alternative.

Minify generated code, not beautified code. Prevents beautifier bias.

Prevent unnecessary duplication

Remove serialization protection because I think it got handled elsewhere

Abstract toplevel code generation

Add example line of running test case

Add poor man options parser, and some options

Reindent to 4 spaces

Lower chance of `default` generation

Comment example of testing a case and output improvement

Enable `default` clause appearing at any clause index

Removing some training wheels; dont add parens where we dont absolutely need them

Support `-s1` and `-s2` to force specific statements being generated at that recursion level

Add round number to output when failing. For stats and fun and profit.

Solidify statement depth counting. The argument juggling is real.

Renamed option to something long. -scf was ugly and probably confusing.

Fix missing arguments causing `canThrow` to be truthy, generating crashing code

Generate more binary nested expressions

Add black and white list cli options for statement generation

Allows you to explicitly require or forbid certain statements from/to being made.

```
node test/ufuzz.js --without-stmt switch,try -t 5 -r 5 -V
```

```
node test/ufuzz.js --only-stmt ifelse,expr -t 5 -r 5 -V
```

Similar granularity for expression may be added later.

There can be no comma between names; it just does a split on that arg.

Trim down the binary expression generator

Prevent scoping issues in nodejs by preventing certain names in global space

Oh this list was incomplete?

Allow bin-expr to generate assignments too. More vigilant with storing and reusing vars.

Add more global builtin names

Update wrapper code

Also patch Function valueOf
@pvdz
Copy link
Contributor Author

pvdz commented Mar 31, 2017

sasquatched 👹 ready for commit

@alexlamsl alexlamsl merged commit e6b76a4 into mishoo:master Mar 31, 2017
@alexlamsl
Copy link
Collaborator

@qfox thanks for all the work - much appreciated 😉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants