Golang internal library fuzzers #2188

guidovranken · 2019-02-27T13:46:44Z

This plugs the fuzzers in https://github.com/dvyukov/go-fuzz-corpus into libFuzzer.

Is there any interest in this?

kcc · 2019-02-27T19:33:30Z

exciting!
This needs more work before it can be submitted (see comments), but this is certainly interesting.

I wish we could do it the right way (golang/go#19109) but this way is better than none.

Expect lots of things to not work. E.g. the coverage dashboard.
Or various types of reporting. E.g. all the fuzz targets have cov: 1:

#65210	REDUCE cov: 1 ft: 887 corp: 124/10236b lim: 551 exec/s: 9315 rss: 198Mb L: 3/338 MS: 1 EraseBytes-

(obviously, because you are using libFuzzer's __libfuzzer_extra_counters)

You will need a project.yaml

Is this expected to work with AFL?

You probably don't need AddressSanitizer, but there is currently no way to not have the asan build.
(not a huge problem)

I would appreciate if you could add a short readme to https://github.com/guidovranken/libfuzzer-go
I think I understand how it works, but it's worth documenting it. :)

@dvyukov FYI

kcc · 2019-02-27T19:12:47Z

projects/golang/build.sh

@@ -0,0 +1,54 @@
+# Get golang
+wget https://dl.google.com/go/go1.11.1.linux-amd64.tar.gz


this should go to the Docker file.
Also, why fixed revision and not the head?

kcc · 2019-02-27T19:13:18Z

projects/golang/build.sh

+
+mkdir -p $GOPATH/src/github.com/dvyukov/
+cd $GOPATH/src/github.com/dvyukov/
+git clone https://github.com/dvyukov/go-fuzz-corpus


same here and below: all download should go to Dockerfile

kcc · 2019-02-27T19:14:30Z

projects/golang/Dockerfile

@@ -0,0 +1,21 @@
+# Copyright 2016 Google Inc.


fix the year.

kcc · 2019-02-27T19:33:13Z

projects/golang/build.sh

+    fuzzer=$(basename $1)
+    echo "Fuzzer is $fuzzer"
+    $CC $CFLAGS -Wall -Wextra $GOPATH/src/github.com/guidovranken/libfuzzer-go/C/main_libFuzzer_extra_counters.c -g -O3 -c -o main.o
+    ./go-fuzz-build -o $fuzzer.a github.com/dvyukov/go-fuzz-corpus/$fuzzer


maybe add some comments on how the code gets instrumented here.
I assume this is source-to-source thing that go-fuzz uses, but not sure, and other readers won't know.

kcc · 2019-02-27T19:43:04Z

Just an example, here is how a fuzzing session with a bug report will look like in this project:

...
#2804	NEW    cov: 1 ft: 905 corp: 47/169b lim: 8 exec/s: 350 rss: 46Mb L: 4/7 MS: 1 ChangeByte-
#2805	REDUCE cov: 1 ft: 905 corp: 47/167b lim: 8 exec/s: 350 rss: 46Mb L: 4/7 MS: 1 EraseBytes-
#2913	NEW    cov: 1 ft: 906 corp: 48/175b lim: 8 exec/s: 323 rss: 46Mb L: 8/8 MS: 3 InsertByte-ShuffleBytes-ChangeByte-
panic: asn1: string not valid UTF-8

goroutine 17 [running, locked to thread]:
github.com/dvyukov/go-fuzz-corpus/asn1.Fuzz(0x602000030190, 0x6, 0x6, 0x10c00015dea0)
	/tmp/go-fuzz-build479809458/gopath/src/github.com/dvyukov/go-fuzz-corpus/asn1/asn1.go:46 +0x825
main.fuzzer_run(0x602000030190, 0x6, 0x6)
	/tmp/go-fuzz-build479809458/gopath/src/github.com/dvyukov/go-fuzz-corpus/asn1/go.fuzz.main/main.go:13 +0x41
main._cgoexpwrap_c53586c27493_fuzzer_run(0x602000030190, 0x6, 0x6)
	_cgo_gotypes.go:45 +0x41
==160894== ERROR: libFuzzer: deadly signal
    #0 0x4ff8f3 in __sanitizer_print_stack_trace /src/llvm/projects/compiler-rt/lib/asan/asan_stack.cc:37
    #1 0x6e942a in fuzzer::PrintStackTrace() /src/libfuzzer/FuzzerUtil.cpp:205:5
    #2 0x69f7eb in fuzzer::Fuzzer::CrashCallback() /src/libfuzzer/FuzzerLoop.cpp:234:3
    #3 0x7fa0fe5e20bf  (/lib/x86_64-linux-gnu/libpthread.so.0+0x110bf)
    #4 0x589b13 in runtime.raise /tmp/go-fuzz-build479809458/goroot/src/runtime/sys_linux_amd64.s:145

NOTE: libFuzzer has rudimentary signal handlers.
      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal
MS: 1 ChangeBit-; base unit: 0d33cba94bf7c5b7b042dc97b1a2d50ccb4a3e42
0x14,0x4,0xff,0xff,0xff,0xff,
\x14\x04\xff\xff\xff\xff
artifact_prefix='./'; Test unit written to ./crash-7dbf1774dcb05cfa72766eb3651f869afcdbb3dd
Base64: FAT/////
==160894==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7fff0e9fa000; bottom 0x10c00015c000; size: 0x6f3f0e89e000 (122316617539584)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189

kcc · 2019-02-27T19:53:32Z

one more.
How actionable is this report?
Can it be improved?

panic: parse //%B9%B9%B9%B9%B9%01%00%00%00%00%00%00%00%B9%B9%B9%B9%B9%B9%B9%B9%B9%B9%B9%B9%B9%B9%B9: invalid URL escape "%01"

goroutine 17 [running, locked to thread]:
github.com/dvyukov/go-fuzz-corpus/url.Fuzz(0x6030001458a0, 0x20, 0x20, 0x10c0000c8ea0)
        /tmp/go-fuzz-build242808228/gopath/src/github.com/dvyukov/go-fuzz-corpus/url/main.go:24 +0x3d5
main.fuzzer_run(0x6030001458a0, 0x20, 0x20)
        /tmp/go-fuzz-build242808228/gopath/src/github.com/dvyukov/go-fuzz-corpus/url/go.fuzz.main/main.go:13 +0x41
main._cgoexpwrap_9bd49841752b_fuzzer_run(0x6030001458a0, 0x20, 0x20)
        _cgo_gotypes.go:45 +0x41
==158476== ERROR: libFuzzer: deadly signal

guidovranken · 2019-02-27T20:20:03Z

one more.
How actionable is this report?
Can it be improved?

panic: parse //%B9%B9%B9%B9%B9%01%00%00%00%00%00%00%00%B9%B9%B9%B9%B9%B9%B9%B9%B9%B9%B9%B9%B9%B9%B9: invalid URL escape "%01"

goroutine 17 [running, locked to thread]:
github.com/dvyukov/go-fuzz-corpus/url.Fuzz(0x6030001458a0, 0x20, 0x20, 0x10c0000c8ea0)
        /tmp/go-fuzz-build242808228/gopath/src/github.com/dvyukov/go-fuzz-corpus/url/main.go:24 +0x3d5
main.fuzzer_run(0x6030001458a0, 0x20, 0x20)
        /tmp/go-fuzz-build242808228/gopath/src/github.com/dvyukov/go-fuzz-corpus/url/go.fuzz.main/main.go:13 +0x41
main._cgoexpwrap_9bd49841752b_fuzzer_run(0x6030001458a0, 0x20, 0x20)
        _cgo_gotypes.go:45 +0x41
==158476== ERROR: libFuzzer: deadly signal

The only way I can think of to improve it is to generate more verbose panics in the fuzzer, eg here: https://github.com/dvyukov/go-fuzz-corpus/blob/master/url/main.go#L24

In the stack trace, url/main.go:24 is sufficient information to locate the source of the panic.

I'm working on addressing your remarks now.

dvyukov · 2019-02-28T10:09:26Z

projects/golang/Dockerfile

+MAINTAINER guidovranken@gmail.com
+
+RUN apt-get update && apt-get install -y build-essential wget
+RUN wget https://dl.google.com/go/go1.11.1.linux-amd64.tar.gz


Most of go-fuzz-corpus tests the standard library, so if we download the fixed revision, we test the fixed revision.
I would assume that all existing fuzzers should checkout a git with the project to get latest sources for testing?

And FWIW 1.12 is already out.

For some reason I get an import cycle error when I run it on 1.12. Any idea why? To test s/1.11.1/1.12/g in Dockerfile and build.

What import cycle? On what binary? What is "build"?

failed to execute go build: exit status 1 import cycle not allowed package github.com/dvyukov/go-fuzz-corpus/url/go.fuzz.main imports github.com/dvyukov/go-fuzz-corpus/url imports fmt imports go-fuzz-dep imports syscall imports runtime imports runtime/internal/math imports go-fuzz-dep

Happens for 1.12 but not for 1.11.1.

To test:

git clone https://github.com/google/oss-fuzz cd oss-fuzz [change downloaded Go version from 1.11.1 to 1.12 in projects/golang/Dockerfile] python infra/helper.py build_fuzzers golang

Adding runtime/internal/math here should help:
https://github.com/dvyukov/go-fuzz/blob/master/go-fuzz-build/main.go#L302-L304

That appears to fix it, thanks!

With regards to using the latest Go version, does the project offer nightly binaries? If not I'll look into https://golang.org/doc/install/source and try if I get build the latest version from the repo (I've never done that before).

Do you mind sending PR for go-fuzz with the runtime/internal/math change?

Go does not offer nightly releases, but it offer simple and working build :)
git clone; cd go/src; ./make.bash

I’m actively updating Go-fuzz-build to work with go/packages, which should fix this issue and similar ones in the future. I hope to send a PR in the next few days. I’d love help testing and reviewing it—it ends up being a pretty major rewrite.

Do you mind sending PR for go-fuzz with the runtime/internal/math change?

dvyukov/go-fuzz#211 should fix this, in a much deeper way. Still needs testing (and probably fixes) on Windows. Help there--even just bug reports--would be deeply appreciated.

Reviews from any/all are also very welcome. It's a large change.

dvyukov · 2019-02-28T10:12:08Z

This is awesome!
How/where will this file bugs?

guidovranken · 2019-02-28T15:11:53Z

This is awesome!
How/where will this file bugs?

I propose we review your fuzzers and raise the threshold for panicking where applicable (eg. should url crash ? see @kcc 's stack trace above. there are a couple of other ones that crash).

For my other Go fuzzer at oss-fuzz (bignum-fuzzer), security@golang.org is in the bug report CC.
I can open an issue at the Golang repo notifying them of this effort and ask if that same process should be followed here.

dvyukov · 2019-02-28T15:19:58Z

I propose we review your fuzzers and raise the threshold for panicking where applicable (eg. should url crash ? see @kcc 's stack trace above. there are a couple of other ones that crash).

For my other Go fuzzer at oss-fuzz (bignum-fuzzer), security@golang.org is in the bug report CC.
I can open an issue at the Golang repo notifying them of this effort and ask if that same process should be followed here.

I guess asking them is the right option. There can be lots of bugs, so they should be prepared. Maybe they will say to open all them (easily discoverable by anybody anyway).

josharian · 2019-02-28T16:03:49Z

IMHO go-fuzz-corpus is in need of some attention before it can be unleashed in an automated system.

(1) There are existing false positives. Off the top of my head, the json fuzz function incorrectly reports round trip failures when the fuzz data contains duplicate keys. And the gofmt fuzz has false positives around \r and comments. You’ll find these quickly enough if you start fuzzing. :) But I think they should be fixed before turning it on—there has been resistance to fuzzer bugs in the past, and it’d be better to start on a good footing.

(2) The fuzz functions contain a bunch of exceptions for bugs that have been fixed, particularly the go/types one.

Some other things while I’m here, and apologies for drifting off topic. (1) Naive question: will new generate corpus entries be upstreamed somewhere discoverable? I’m working on improving go-fuzz, and I’d love to have a throughly saturated corpus to try to improve. (2) If I want to request oss-fuzz add support for starlark, how do I do that? I have a decently developed fuzz function and a pretty large corpus already. (3) How do I get (moderately) more involved here? I found this only by accident, backlinked from the Go repo issue you filed.

dvyukov · 2019-02-28T16:06:23Z

IMHO go-fuzz-corpus is in need of some attention before it can be unleashed in an automated system.

Agree. It can make sense to start with just 1, safe fuzzer (fmt?) to get everything else working. And then add other fuzzers one-by-one or in batches.

FiloSottile · 2019-02-28T16:52:41Z

I would love for the Fuzz functions to live in the standard library, so that we can ask people to add them or update them with other CLs. I'm happy for the crypto packages to be the guinea pigs here, too, and those can just email me for now.

It looks like the easy way to do that would be to have Fuzz functions in files with the go-fuzz build tag, and then for these scripts to fetch the most recent git tree, find them and run them. Does that sound right?

@dvyukov @josharian How do you feel about go-fuzz-corpus slowly migrating into the standard library? Is it too soon/not something you want to happen?

josharian · 2019-02-28T17:09:05Z

How do you feel about go-fuzz-corpus slowly migrating into the standard library? Is it too soon/not something you want to happen?

I'd love fuzzing to be officially supported in the toolchain. There are important unanswered questions, though, among them: Do you check in the corpus entries? If so, where do they live? Does cmd/go need to be aware of them? If so, can we do that independently of fully integrating fuzzing into the toolchain? These seem pretty apropos for moving go-fuzz-corpus into the main repo.

dvyukov · 2019-02-28T17:09:40Z

I would love for the Fuzz functions to live in the standard library, so that we can ask people to add them or update them with other CLs. I'm happy for the crypto packages to be the guinea pigs here, too, and those can just email me for now.

It looks like the easy way to do that would be to have Fuzz functions in files with the go-fuzz build tag, and then for these scripts to fetch the most recent git tree, find them and run them. Does that sound right?

@dvyukov @josharian How do you feel about go-fuzz-corpus slowly migrating into the standard library? Is it too soon/not something you want to happen?

I think that would be great!
Since this will be excluded from build, we don't commit to anything, so it looks like quite safe change.
We could also try to support Fuzz functions in *_test.go files before this move (as I think that's the eventual plan). But that would require some non-trivial changes to go-fuzz-build.

dvyukov · 2019-02-28T17:13:30Z

How do you feel about go-fuzz-corpus slowly migrating into the standard library? Is it too soon/not something you want to happen?

I'd love fuzzing to be officially supported in the toolchain. There are important unanswered questions, though, among them: Do you check in the corpus entries? If so, where do they live? Does cmd/go need to be aware of them? If so, can we do that independently of fully integrating fuzzing into the toolchain? These seem pretty apropos for moving go-fuzz-corpus into the main repo.

But we don't burn any bridges by committing Fuzz functions into std lib, so we don't have to answer these questions right now.

josharian · 2019-02-28T17:16:25Z

Then where does the generated corpus live?

dvyukov · 2019-02-28T17:17:46Z

Then where does the generated corpus live?

Somewhere in OSS-Fuzz.

josharian · 2019-02-28T17:18:35Z

I dunno. Seems awkward to document to me, which is not usually a good sign. But I'm just one guy. :)

dvyukov · 2019-02-28T17:44:04Z

I dunno. Seems awkward to document to me, which is not usually a good sign. But I'm just one guy. :)

We don't document it. The Fuzz function can be used without corpus by anybody. Corpus is not checked-in. If you want it, run it locally and collect the corpus.
It's like benchmarks are checked-in but benchmark results are not. If you want benchmark results, build it and run locally.

Even if we decide on the format, I am not sure OSS-Fuzz will be able to follow it (both format and permissions to check-in). So it looks mostly orthogonal to me. We can check in Fuzz functions, use them on OSS-Fuzz, and in parallel start figuring out how we want to store corpus for manual runs. Once we figure out the first option to try, we can manually check-in the corpus according to this scheme and see how well it works. But OSS-Fuzz will work with own corpus meanwhile.

I wrote down my thoughts re corpus location in the proposal doc:
https://docs.google.com/document/u/1/d/1zXR-TFL3BfnceEAWytV8bnzB2Tfp6EPFinWVJ5V4QC8/pub
I still think the same now. And nobody provide any objection so far.

FiloSottile · 2019-02-28T18:11:05Z

I say we start by moving reliable Fuzz functions to build-tagged files in the standard library, with no other change. The tree is open so it's a good time to start. Then, we should prioritize moving them to test files so they don't get broken by new changes. (Build tagged files are not compiled, so people will not notice they break. We should make the TryBots do a go-fuzz build in the meantime.)

As a start, the corpus can be in OSS-Fuzz and the reports can go to my inbox. Then we iterate to find better answers to those questions and to make the reports more actionable, the inputs more flexible, the build more performant, etc. :)

dvyukov · 2019-02-28T18:19:15Z

Sounds good to me!

guidovranken · 2019-02-28T19:52:13Z

To be clear, @dvyukov did all the hard work in terms of instrumentation and library fuzzers. For personal purposes, I only hacked his instrumentation tool for direct usage with libFuzzer. Seeing that it was technically possible to fuzz Go libraries with oss-fuzz, I created this PR. That said, my understanding of Go intricacies and efforts to implement built-in Go fuzzing report are limited, but I will try to read up on it.

As said, the PR as it stands uses https://github.com/guidovranken/libfuzzer-go, which is my fork/hack of @dvyukov https://github.com/dvyukov/go-fuzz. In short, it acts as a conduit between CoverTab (https://github.com/dvyukov/go-fuzz/blob/master/go-fuzz-dep/main.go#L27) and libFuzzer's extra counters (https://github.com/guidovranken/libfuzzer-go/blob/master/C/main_libFuzzer_extra_counters.c#L14).
@dvyukov , to the extent that your go-fuzz does not natively support this yet, would you be willing to integrate this feature (copying CoverTab data to libFuzzer's extra counters through a C shim) into your project? It would be nicer to have libFuzzer support in your own project rather than working on two trees in parallel.

Another feature that is preferable to have is libFuzzer's value profile/autodict support. In short, it would be nice that if a comparison is performed in Go (eg if i == 0xAABBCCDD { ...), libFuzzer gets notified of the value 0xAABBCCDD, so that it can use it during generation of inputs. This is particularly useful in targets where hardcoded strings/values are compared (for example magic values in the header of a file format), and code coverage depends on it.
In my fork of go-fuzz, values are intercepted and sent to (a customized) libFuzzer: https://github.com/guidovranken/libfuzzer-go/blob/master/go-fuzz-dep/sonar.go#L67
Another option could be to implement a custom mutator and insert intercepted values there.
Ideally, libFuzzer would offer a callback function for this purpose. @kcc thoughts on this?

(1) Naive question: will new generate corpus entries be upstreamed somewhere discoverable?

The up-to-date corpus is accessible through https://oss-fuzz.com/ (once the project is online). It can be manually be pushed to a public repo. See for example OpenSSL, which frequently does this: https://github.com/openssl/openssl/commits/master/fuzz/corpora

(2) If I want to request oss-fuzz add support for starlark, how do I do that? I have a decently developed fuzz function and a pretty large corpus already.

To build a libFuzzer-backed Go fuzzer:

Implement func Fuzz(input []byte) {
(see https://github.com/guidovranken/golang-fuzzers/blob/master/encoding_json/fuzz.go for a basic example)
go build github.com/guidovranken/libfuzzer-go/go-fuzz-build
Compile https://github.com/guidovranken/libfuzzer-go/blob/master/C/main_libFuzzer_extra_counters.c to main.o
run ./go-fuzz-build -o libname.a github.com/path/to/fuzz.go
Link main.o + libname.a + libFuzzer.a

For oss-fuzz integration, see Dockerfile + build.sh in this PR.
Feel free to ping me for any help.

One more thought: strong seed corpora can be generated from existing public corpora (like that of OpenSSL) for generic formats like ASN1, PEM, JSON, X509, etc.

dvyukov · 2019-03-01T08:10:27Z

@dvyukov , to the extent that your go-fuzz does not natively support this yet, would you be willing to integrate this feature (copying CoverTab data to libFuzzer's extra counters through a C shim) into your project?

What exactly are talking about? Where is this implemented in your fork?
I mean generally it would be great to add libfuzzer support and have a single repo rather than fork, but I would like to understand what exactly we are talking about first.

dvyukov · 2019-03-01T08:17:00Z

Another feature that is preferable to have is libFuzzer's value profile/autodict support. In short, it would be nice that if a comparison is performed in Go (eg if i == 0xAABBCCDD { ...), libFuzzer gets notified of the value 0xAABBCCDD, so that it can use it during generation of inputs.

It sounds reasonable for libfuzzer to provide some callback for this along with __libfuzzer_extra_counters to make it possible to fully replace instrumentation.

In your current implementation I see some hooks for this, but it is not glued to libfuzzer, right? I see that memcmpCBPtr is always set to NULL.

dvyukov · 2019-03-01T08:19:49Z

One more thought: strong seed corpora can be generated from existing public corpora (like that of OpenSSL) for generic formats like ASN1, PEM, JSON, X509, etc.

I see there are at least 3 SSL implementations tests, is there a way to share common input formats on OSS-Fuzz?

guidovranken · 2019-08-14T20:54:04Z

So should I add golang-fuzz@googlegroups.com to project.yaml or is the intention to manually forward reports there? And should I add view_restrictions: none ?

FiloSottile · 2019-08-14T21:12:05Z

Please add golang-fuzz@googlegroups.com directly, and view_restrictions: none sounds ok (although I don't know the details).

guidovranken · 2019-08-14T21:24:51Z

It means that all bugs are immediately publicly disclosed at https://bugs.chromium.org/p/oss-fuzz/issues/list

Dor1s · 2019-08-15T22:10:52Z

Alright, the stack parsing PR is almost landed. One last thing to mention here before landing this as well. We should be able to suppress libFuzzer's stacktrace from the log by using -handle_abort=0, e.g. see below:

root@3fbf31a34517:/out# ./fuzzer-asn1 -handle_bus=0 ./crash-11dbe891affdb59595d548581bf3cae1f2cba0a8 
INFO: Seed: 204859258
INFO: 65536 Extra Counters
./fuzzer-asn1: Running 1 inputs 1 time(s) each.
Running: ./crash-11dbe891affdb59595d548581bf3cae1f2cba0a8
panic: asn1: string not valid UTF-8

goroutine 17 [running, locked to thread]:
github.com/dvyukov/go-fuzz-corpus/asn1.Fuzz(0x26c9da0, 0x16, 0x16, 0x7fff84fd8e08)
	/src/go/packages/src/github.com/dvyukov/go-fuzz-corpus/asn1/asn1.go:46 +0x994
main.LLVMFuzzerTestOneInput(0x26c9da0, 0x16, 0x89f390)
	github.com/dvyukov/go-fuzz-corpus/asn1/go.fuzz.main/main.go:35 +0x66
main._cgoexpwrap_0a73695ed89a_LLVMFuzzerTestOneInput(0x26c9da0, 0x16, 0x26c98c0)
	_cgo_gotypes.go:64 +0x37
==54== ERROR: libFuzzer: deadly signal
    #0 0x427800 in __sanitizer_print_stack_trace /src/llvm/projects/compiler-rt/lib/ubsan/ubsan_diag_standalone.cpp:33:3
    #1 0x46271f in fuzzer::PrintStackTrace() /src/libfuzzer/FuzzerUtil.cpp:205:5
    #2 0x4419e3 in fuzzer::Fuzzer::CrashCallback() /src/libfuzzer/FuzzerLoop.cpp:232:3
    #3 0x7f83a432138f  (/lib/x86_64-linux-gnu/libpthread.so.0+0x1138f)
    #4 0x4bed70 in runtime.raise runtime/sys_linux_amd64.s:149

NOTE: libFuzzer has rudimentary signal handlers.
      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal
======================================================
#### Compare with:
======================================================
root@3fbf31a34517:/out# ./fuzzer-asn1 -handle_abrt=0 ./crash-11dbe891affdb59595d548581bf3cae1f2cba0a8 
INFO: Seed: 209483495
INFO: 65536 Extra Counters
./fuzzer-asn1: Running 1 inputs 1 time(s) each.
Running: ./crash-11dbe891affdb59595d548581bf3cae1f2cba0a8
panic: asn1: string not valid UTF-8

goroutine 17 [running, locked to thread]:
github.com/dvyukov/go-fuzz-corpus/asn1.Fuzz(0x1990da0, 0x16, 0x16, 0x7fff704b43b8)
	/src/go/packages/src/github.com/dvyukov/go-fuzz-corpus/asn1/asn1.go:46 +0x994
main.LLVMFuzzerTestOneInput(0x1990da0, 0x16, 0x89f390)
	github.com/dvyukov/go-fuzz-corpus/asn1/go.fuzz.main/main.go:35 +0x66
main._cgoexpwrap_0a73695ed89a_LLVMFuzzerTestOneInput(0x1990da0, 0x16, 0x19908c0)
	_cgo_gotypes.go:64 +0x37
Aborted (core dumped)

But I don't think it's too big of a deal. The go frames will be parsed first and used in the crash state anyways. Plus, in case of some really weird and unexpected crash, libFuzzer logs may appear to be more useful compared to the standard Aborted (core dumped) message?

If you would like to get rid of these frames, we can add .options file for every fuzz target with the following contents:

[libfuzzer]
handle_abrt = 0

but I think it's fine for now, we can do it later.

Dor1s · 2019-08-16T16:20:38Z

FYI, we have to use ASan, otherwise corpus pruning won't work. I'm doing some speed measurements locally to decide whether we'll proceed with ASan only or with both ASan and UBSan (in case UBSan works much faster).

Dor1s · 2019-08-16T16:52:40Z

Don't observe any significant performance difference, likely because there isn't much code being instrumented by either of the sanitizers. I'll merge this and update the project.yaml myself right after that. Bad build check succeeded locally.

CF deployment is in the progress, btw.

dvyukov · 2019-08-17T20:09:11Z

Good work, everyone!
@Dor1s says we are already getting new bug reports (just need to find them!)

dvyukov · 2019-08-17T20:12:31Z

They should appear at:
https://groups.google.com/forum/#!forum/golang-fuzz
but they are not there. Is there moderation set up or something? @FiloSottile please add some more trusted people to owners of the group for redundancy.

This seems to be the query to find them:
https://bugs.chromium.org/p/oss-fuzz/issues/list?colspec=ID%20Type%20Component%20Status%20Proj%20Reported%20Owner%20Summary&q=%22Project%3A%20golang%22&can=2
and, yes, they have the group in CC.

FiloSottile · 2019-08-17T22:09:58Z

The group is set up like all golang-* to moderate the first posting by any new author. I have now allowed forever you and Monorail, we can see if that's a problem and worth changing as time goes.

Dor1s · 2019-08-20T23:22:07Z

FYI, we've improved stack parsing for Go crashes on ClusterFuzz side, and a couple of previously reported issues may be reported again with another (better) crash type. Could update the existing crashes manually, but didn't bother as there are not too many being affected and in general it's a one-off change as we've just started running Go fuzzers.

dgryski · 2019-08-21T17:56:39Z

Is there a doc listing the steps for getting more fuzzing functions running by default? Is this something we could effectively crowd-source?

Dor1s · 2019-08-21T18:06:33Z

I'll document OSS-Fuzz side of things later today after landing and testing #2735.

guidovranken · 2019-08-21T19:29:22Z

Is there a doc listing the steps for getting more fuzzing functions running by default? Is this something we could effectively crowd-source?

Do you mean stdlib fuzzing, or fuzzing Go software generally?

As for the stdlib, only a fraction of https://github.com/dvyukov/go-fuzz-corpus is currently running on OSS-Fuzz.

Adding more would simply consist of adding a compile_fuzzer command for each additional fuzzer to https://github.com/google/oss-fuzz/blob/master/projects/golang/build.sh

One caveat is that it shouldn't crash on the seed corpus.

dgryski · 2019-08-21T19:45:23Z

I was specifically thinking of stdlib fuzzing. Good to know adding more fuzz functions are easy. So it's just a question of getting new fuzz functions into go-fuzz-corpus.

Dor1s · 2019-08-21T23:02:52Z

I'll document OSS-Fuzz side of things later today after landing and testing #2735.

Update: looks like updating the OSS-Fuzz documentation may take a bit longer. So, for the time being:

if you want to add more fuzz targets to golang stdlib project, refer to @guidovranken's comment above: Golang internal library fuzzers #2188 (comment)
if you want to fuzz another Go project, take a look at the syzkaller integration with OSS-Fuzz, it's very concise and should be easy to follow as an example: https://github.com/google/oss-fuzz/tree/master/projects/syzkaller

josharian · 2021-02-12T06:09:09Z

How can I add @odeke-em as a registered maintainer for these fuzzers?

dvyukov · 2021-02-12T07:54:40Z

The metadata is here:
https://github.com/google/oss-fuzz/blob/master/projects/golang/project.yaml
But I see only CC field.

inferno-chromium · 2021-02-12T08:14:56Z

How can I add @odeke-em as a registered maintainer for these fuzzers?

The metadata is here:
https://github.com/google/oss-fuzz/blob/master/projects/golang/project.yaml
But I see only CC field.

yes @odeke-em in the auto_ccs field, docs here - https://google.github.io/oss-fuzz/getting-started/new-project-guide/#primary

odeke-em · 2021-02-12T21:55:29Z

Thank you @josharian @dvyukov and @inferno-chromium! I've mailed this PR #5189 adding myself as a maintainer.

odeke-em · 2021-02-12T22:41:46Z

For a context of the error I got, I tried to go download the testcase for https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=30042&q=label%3AProj-golang and instead got this

and for a couple of other bugs. My goal is to get them filed for the Go project and to fix them too.

guidovranken added 3 commits February 27, 2019 01:15

Add golang fuzzer

0b48d41

Use dvyukov's fuzzers

20aa4a7

Add more fuzzers

a42a928

kcc reviewed Feb 27, 2019

View reviewed changes

Move data retrieval to Dockerfile + cosmetic changes

6c46a41

dvyukov reviewed Feb 28, 2019

View reviewed changes

guidovranken mentioned this pull request Feb 28, 2019

all: oss-fuzz integration golang/go#30474

Closed

auto_ccs

c13838e

howardjohn mentioned this pull request Aug 16, 2019

Continuous Fuzzing istio/istio#16324

Closed

Dor1s mentioned this pull request Aug 16, 2019

Support corpus pruning with different sanitizers google/clusterfuzz#851

Open

jonathanmetzman mentioned this pull request Aug 16, 2019

[django] No ASAN #2696

Merged

Dor1s approved these changes Aug 16, 2019

View reviewed changes

Dor1s merged commit aebba43 into google:master Aug 16, 2019

josharian mentioned this pull request Aug 29, 2019

make go-fuzz-build work with modules (for #195) dvyukov/go-fuzz#263

Closed

thepudds mentioned this pull request Sep 22, 2019

Explicitly disable Go modules via GO111MODULE=off for go-fuzz corpus #2878

Closed

odeke-em mentioned this pull request Feb 12, 2021

projects/golang: add Emmanuel Odeke as a Go maintainer #5189

Merged

DavidKorczynski mentioned this pull request Mar 4, 2024

[golang] restrict access to golang issues #11663

Merged

		@@ -0,0 +1,54 @@
		# Get golang
		wget https://dl.google.com/go/go1.11.1.linux-amd64.tar.gz

Golang internal library fuzzers #2188

Golang internal library fuzzers #2188

Conversation

guidovranken commented Feb 27, 2019

kcc commented Feb 27, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kcc commented Feb 27, 2019

kcc commented Feb 27, 2019

guidovranken commented Feb 27, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dvyukov commented Feb 28, 2019

guidovranken commented Feb 28, 2019

dvyukov commented Feb 28, 2019

josharian commented Feb 28, 2019

dvyukov commented Feb 28, 2019

FiloSottile commented Feb 28, 2019

josharian commented Feb 28, 2019

dvyukov commented Feb 28, 2019

dvyukov commented Feb 28, 2019

josharian commented Feb 28, 2019

dvyukov commented Feb 28, 2019

josharian commented Feb 28, 2019

dvyukov commented Feb 28, 2019

FiloSottile commented Feb 28, 2019

dvyukov commented Feb 28, 2019

guidovranken commented Feb 28, 2019 • edited Loading

dvyukov commented Mar 1, 2019

dvyukov commented Mar 1, 2019

dvyukov commented Mar 1, 2019

guidovranken commented Aug 14, 2019

FiloSottile commented Aug 14, 2019

guidovranken commented Aug 14, 2019

Dor1s commented Aug 15, 2019

Dor1s commented Aug 16, 2019

Dor1s commented Aug 16, 2019

dvyukov commented Aug 17, 2019

dvyukov commented Aug 17, 2019

FiloSottile commented Aug 17, 2019

Dor1s commented Aug 20, 2019

dgryski commented Aug 21, 2019

Dor1s commented Aug 21, 2019

guidovranken commented Aug 21, 2019

dgryski commented Aug 21, 2019

Dor1s commented Aug 21, 2019

josharian commented Feb 12, 2021

dvyukov commented Feb 12, 2021

inferno-chromium commented Feb 12, 2021

odeke-em commented Feb 12, 2021

odeke-em commented Feb 12, 2021

guidovranken commented Feb 28, 2019 •

edited

Loading