Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dev.fuzz] panic when corpus file exists #48228

Closed
kokes opened this issue Sep 7, 2021 · 6 comments
Closed

[dev.fuzz] panic when corpus file exists #48228

kokes opened this issue Sep 7, 2021 · 6 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@kokes
Copy link

kokes commented Sep 7, 2021

What version of Go are you using (go version)?

$ gotip version
go version devel go1.17-9c56a64673 Fri Sep 3 18:34:56 2021 +0000 darwin/amd64

Does this issue reproduce with the latest release?

Yes (release meaning commit in this case)

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/okokes/Library/Caches/go-build"
GOENV="/Users/okokes/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/okokes/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/okokes/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.16.6/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.16.6/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16.6"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/okokes/git/smda2/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/81/4jydp7kn51n6p68z88sqnkzc0000gn/T/go-build3291749728=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I updated my gotip (dev.fuzz) version and re-ran my fuzzing script (that previously ran for days and created a few testdata files), it crashed immediately. The only way I could make it not crash was to delete all my corpus files. Here's a full reproducer:

git clone -b experiments/fuzzing https://github.com/kokes/smda
cd smda

mkdir -p src/query/expr/testdata/corpus/FuzzSQLParser
echo -e 'go test fuzz v1\nstring("SELECT r FROM E@vdCE33333333333333333")' > src/query/expr/testdata/corpus/FuzzSQLParser/a779a4e81a21b0d1fcfa220bf260ef10ec172416e4391a4299896bfd6c7a5406

gotip clean -fuzzcache
gotip test -v -run=NONE -fuzz=FuzzSQL ./src/query/expr

What did you expect to see?

I expected fuzzing to start

What did you see instead?

$ gotip test -v -run=NONE -fuzz=FuzzSQL ./src/query/expr
=== FUZZ  FuzzSQLParser
--- FAIL: FuzzSQLParser (0.00s)
panic: must have at least one value to marshal [recovered]
	panic: must have at least one value to marshal

goroutine 19 [running]:
testing.fRunner.func1.2({0x12ce9e0, 0x13786b0})
	/Users/okokes/sdk/gotip/src/testing/fuzz.go:701 +0x1eb
testing.fRunner.func1()
	/Users/okokes/sdk/gotip/src/testing/fuzz.go:704 +0x1da
panic({0x12ce9e0, 0x13786b0})
	/Users/okokes/sdk/gotip/src/runtime/panic.go:1038 +0x215
internal/fuzz.marshalCorpusFile({0x0, 0x1, 0x0})
	/Users/okokes/sdk/gotip/src/internal/fuzz/encoding.go:23 +0x59a
internal/fuzz.newCoordinator({{0x137a600, 0xc0000a2010}, 0x0, 0x0, 0xdf8475800, 0x0, 0xc, {0xc00018b800, 0x23, 0x45}, ...})
	/Users/okokes/sdk/gotip/src/internal/fuzz/fuzz.go:582 +0x98
internal/fuzz.CoordinateFuzzing({0x137f780, 0xc0000fea80}, {{0x137a600, 0xc0000a2010}, 0x0, 0x0, 0xdf8475800, 0x0, 0xc, {0xc00018b800, ...}, ...})
	/Users/okokes/sdk/gotip/src/internal/fuzz/fuzz.go:97 +0x118
testing/internal/testdeps.TestDeps.CoordinateFuzzing({}, 0x0, 0x0, 0xdf8475800, 0x0, 0xc, {0xc00018b800, 0x23, 0x45}, {0xc000099760, ...}, ...)
	/Users/okokes/sdk/gotip/src/testing/internal/testdeps/deps.go:151 +0x278
testing.(*F).Fuzz(0xc0001a6000, {0x12d2660, 0x13368e0})
	/Users/okokes/sdk/gotip/src/testing/fuzz.go:406 +0x8b5
github.com/kokes/smda/src/query/expr.FuzzSQLParser(0x0)
	/Users/okokes/git/smda2/src/query/expr/parser_fuzz_test.go:163 +0x752
testing.fRunner(0xc0001a6000, 0x13368e8)
	/Users/okokes/sdk/gotip/src/testing/fuzz.go:738 +0xd1
created by testing.runFuzzing
	/Users/okokes/sdk/gotip/src/testing/fuzz.go:633 +0x925
exit status 2
FAIL	github.com/kokes/smda/src/query/expr	1.098s
FAIL
@thanm thanm added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 7, 2021
@thanm
Copy link
Contributor

thanm commented Sep 7, 2021

@golang/fuzzing

@katiehockman katiehockman added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Sep 7, 2021
@kokes
Copy link
Author

kokes commented Sep 8, 2021

Bisected this and found e5247f7 to be the first bad commit.

@katiehockman
Copy link
Contributor

@kokes Great thanks for bisecting and finding the culprit CL. We'll take a look and plan to fix this soon.

@katiehockman katiehockman added this to the Go1.18 milestone Sep 8, 2021
@katiehockman katiehockman self-assigned this Sep 8, 2021
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/348381 mentions this issue: [dev.fuzz] internal/fuzz: fix panic when marshaling data

gopherbot pushed a commit that referenced this issue Sep 8, 2021
The coordinator needs to marshal data that was provided
via f.Add. However, it was also attempting to marshal data
that was in testdata, which was not needed,
and was causing a panic. This change fixes this.

Fixes #48228

Change-Id: I1256c5a287b5a09d2f8cca59beb0f0fc06cc3554
Reviewed-on: https://go-review.googlesource.com/c/go/+/348381
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
@kokes
Copy link
Author

kokes commented Sep 10, 2021

@katiehockman good stuff, can confirm go test -fuzz no longer panics in my use case! (Not closing as not sure if there's any pending work related to this, feel free to do so yourself.)

@katiehockman
Copy link
Contributor

Thanks for verifying! I don't actually know why gopherbot didn't close it, but I'll do so now.

@golang golang locked and limited conversation to collaborators Jun 23, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

4 participants