Fuzz large offsets through sequence compression api

[daniellerozenblit]

This PR introduces a few changes to the sequence compression api fuzzer in order to better test cases with large offsets without generating many MB of input data.

• Rather than generating a new dictionary of max size 256KB with each fuzzer call, we now generate a huge global dictionary of size 1 << ZSTD_WINDOWLOG_MAX_32 that is reused between calls. The dictionary is calloc'd in order to fit the memory constraints of the oss-fuzz environment.
• We now use a maximum window size of ZSTD_WINDOWLOG_MAX, rather than ZSTD_WINDOWLOG_MAX_32.
• We now generate the literalBuffer using a randomly generated seed, rather than using the same seed (0) each time.

This PR also fixes a bug exposed by #3439 and ensures that the seqStore bounds check is accurate forZSTD_copySequencesToSeqStoreNoBlockDelim().

• We now increment the sequence idx after checking whether or not we have room to store an additional sequence.

[terrelln]

@daniellerozenblit lets simplify this a little bit and just calloc() the dictBuffer.

Suggested change

	FILE* dictFile;
	ZSTD_compressionParameters cParams;
	/* Generate a large dictionary file and mmap to buffer */
	generateDictFile(ZSTD_FUZZ_GENERATED_DICT_MAXSIZE, producer);
	dictFile = fopen(ZSTD_FUZZ_DICT_FILE, "r");
	dictBuffer = mmap(NULL, ZSTD_FUZZ_GENERATED_DICT_MAXSIZE, PROT_READ, MAP_PRIVATE, fileno(dictFile), 0);
	FUZZ_ASSERT(dictBuffer);
	fclose(dictFile);
	dictBuffer = calloc(ZSTD_FUZZ_GENERATED_DICT_MAXSIZE, 1);

This should drastically reduce the memory usage, and should make the initialization basically free, as long as we never write to this memory (which we don't).

Any generality we lose through this simplification should be worth it for the simplicity. And we can always extend it later if we need to.

[terrelln]

I don't think this check is right when the else branch is taken and we split the sequence (we set finalMatchSplit = 1; on line 6352. Since we don't increment idx but still store a sequence.

You could simplify this a bit by moving the idx++ to the bottom of the loop and doing:

if (!finalMatchSplit)
    ++idx;

[daniellerozenblit]

Oh thanks, great catch!

[terrelln]

Awesome!