Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

internal/fuzz: "panic: unmarshaling fuzz input value after call: cannot unmarshal empty string" #48721

Closed
bcmills opened this issue Oct 1, 2021 · 6 comments
Labels
FrozenDueToAge fuzz Issues related to native fuzzing support NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Milestone

Comments

@bcmills
Copy link
Contributor

bcmills commented Oct 1, 2021

greplogs --dashboard -md -l -e 'panic: unmarshaling fuzz input value after call:' --since=2021-09-01

2021-09-30T18:10:18-205640e/windows-arm64-10
2021-09-29T16:45:42-2fad7db/windows-amd64-race

@bcmills bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker fuzz Issues related to native fuzzing support labels Oct 1, 2021
@bcmills bcmills added this to the Go1.18 milestone Oct 1, 2021
@katiehockman
Copy link
Contributor

/cc @golang/fuzzing

@katiehockman
Copy link
Contributor

katiehockman commented Oct 22, 2021

It looks like this is still occurring. Last time it occurred was 4 days ago (https://build.golang.org/log/b5e821bc571fbcb208e626adc15cca8cd0a9a036)

I'm only seeing this error on Windows

@rolandshoemaker
Copy link
Member

In two days I've managed to trigger this once, but that was before I had a chance to add any instrumentation, so not entirely sure what is happening.

cc @bufflig for windows knowledge. I suspect what is happening is the memory region is being zero'd (or shifted in some way between the processes) such that after writing something to it in one process in the other process it appears empty. None of the documentation I read for CreateFileMapping and MapViewOfFile indicated we were doing anything that could trigger this (perhaps something with the commit levels?) but entirely possible I missed something obvious.

Link to the relevant memory mapping code in question: https://go.googlesource.com/go/+/refs/heads/master/src/internal/fuzz/sys_windows.go#20

@gopherbot
Copy link

Change https://golang.org/cl/359354 mentions this issue: internal/fuzz: use atomics to synchronize shared memory read/write

@rolandshoemaker
Copy link
Member

This turns out to be a file read/write race.

When multiple workers operate on the same input, it is possible that multiple workers find the same new input, returning it to the coordinator. It is also possible to process two identical inputs such that they provide different coverage maps, due to inherent flakiness in how we implement counters.

In the described case CoordinateFuzzing and workerClient.fuzz can race on writing the interesting input and reading the cached input, respectively. This can cause the read in workerClient.fuzz to return an empty slice, triggering the unmarshaling panic. This appears to trigger more frequently on Windows, due to some difference in how file concurrent writes/reads are handled.

@gopherbot
Copy link

Change https://golang.org/cl/359755 mentions this issue: internal/fuzz: don't add duplicate corpus entries

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge fuzz Issues related to native fuzzing support NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Projects
Status: No status
Development

No branches or pull requests

4 participants