internal/fuzz: regression finding interesting inputs in simple tests #48179
Labels
FrozenDueToAge
fuzz
Issues related to native fuzzing support
NeedsFix
The path to resolution is known, but the work has not been done.
Milestone
Comments
|
I was a bit blocked on investigating this by coverage not working for fuzzing runs: #48178. I suspected that the fuzzer was making progress but nothing was being labelled as interesting. To give an idea of the coverage, I used bpftrace to instrument the function under test where the function returns (with offsets to RET instructions from the function address). This confirmed that the fuzzer generated an input which reached this line which I think qualifies as finding new coverage. In a checkout of https://github.com/stevenjohnstone/go-beta-fuzzer-vs-libfuzzer, I built a test executable I then used my own hacky tool to generate a skeleton bpftrace program which ended up like this // exit from github.com/stevenjohnstone/fuzztests.loopmagic
uprobe:/home/stevie/code/go/fuzztests.test:"github.com/stevenjohnstone/fuzztests.loopmagic" + 169 {
@exit[169] = count();
}
// exit from github.com/stevenjohnstone/fuzztests.loopmagic
uprobe:/home/stevie/code/go/fuzztests.test:"github.com/stevenjohnstone/fuzztests.loopmagic" + 352 {
@exit[352] = count();
}
// exit from github.com/stevenjohnstone/fuzztests.loopmagic
uprobe:/home/stevie/code/go/fuzztests.test:"github.com/stevenjohnstone/fuzztests.loopmagic" + 377 {
@exit[377] = count();
}
This counts the exit points for go tool objdump -s 'fuzztests.loopmagic' fuzztests.testAfter tracing the fuzzer for a few seconds, the output was but the fuzzer displays I think the bpftrace output indicates that new coverage has been found. I think those should turn up in the "interesting" count displayed when fuzzing? |
cherrymui
added
the
NeedsInvestigation
|
cc @golang/fuzzing |
|
Here's a simpler setup which doesn't need "unsafe" support in bpftrace. I built a test executable in a checkout of https://github.com/stevenjohnstone/go-beta-fuzzer-vs-libfuzzer This is the function under test var leet = []byte{1, 3, 3, 7}
func loopmagic(input []byte) bool {
if len(input) != 4 {
return false
}
for i, v := range leet {
if input[i] != v {
return false
}
}
return true
}When I wrote a bpftrace script which captures the slice input to I ran this with Then run the fuzzer with Fuzzer says there's no interesting input: However, the bpftrace script output suggests that the fuzzer has guessed 6 slices of length 4 matching the first two bytes: Going twice around the loop in if len(input) != 4 {
return false
}(552 times) is interesting coverage? |
jayconrod
changed the title
rsc
changed the title
|
@rolandshoemaker the tests at https://github.com/stevenjohnstone/go-beta-fuzzer-vs-libfuzzer now complete in under a second when I use gotip from https://go-review.googlesource.com/c/go/+/364214/ (#49601) 👍 |
|
Change https://golang.org/cl/364214 mentions this issue: |
dmitshur
added
NeedsFix
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
What version of Go are you using (
go version)?Does this issue reproduce with the latest release?
n/a
What operating system and processor architecture are you using (
go env)?go envOutputWhat did you do?
Ran tests from https://github.com/stevenjohnstone/go-beta-fuzzer-vs-libfuzzer. Specifically, the test "FuzzLoop" which started to pass for the beta fuzzer after commit 7b6893d has stopped working with commit 5bc273a.
The fuzzer runs indefinitely (well, several hours so far) without finding a crasher. Notably the "interesting" column remains at zero for the duration of the run.
What did you expect to see?
Prior to 5bc273a, this test case would end with the fuzzer finding a crasher with the "interesting" count increasing through the run.
What did you see instead?
Now, no crasher is found and the "interesting" count remains zero.
The text was updated successfully, but these errors were encountered: