Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: go clean --fuzzcache failure in TestScript/test_fuzz_limit_dup_entry #59062

Closed
bcmills opened this issue Mar 15, 2023 · 5 comments
Closed

cmd/go: go clean --fuzzcache failure in TestScript/test_fuzz_limit_dup_entry #59062

bcmills opened this issue Mar 15, 2023 · 5 comments
Assignees
@rolandshoemaker
Labels
FrozenDueToAge GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done. Testing An issue that has been verified to require only test changes, not just a test failure.
Milestone
Go1.21

Comments

@bcmills
Copy link
Contributor

bcmills commented Mar 15, 2023 

#!watchflakes
post <- pkg == "cmd/go" && `go clean --fuzzcache: exit status`

Pulled out from #55311 (comment).

I think the fix is to set env GOCACHE=$WORK/cache or similar explicitly in script tests that need a clean fuzzcache.
@bcmills bcmills added Testing An issue that has been verified to require only test changes, not just a test failure. NeedsFix The path to resolution is known, but the work has not been done. GoCommand cmd/go labels Mar 15, 2023
@bcmills bcmills added this to the Go1.21 milestone Mar 15, 2023
@bcmills bcmills mentioned this issue Mar 15, 2023
Closed
@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "cmd/go" && `go clean --fuzzcache: exit status`
2023-03-14 21:49 windows-amd64-longtest go@ebf8e26d cmd/go.TestScript (log) 
vcs-test.golang.org rerouted to http://127.0.0.1:54147
https://vcs-test.golang.org rerouted to https://127.0.0.1:54148
go test proxy running at GOPROXY=http://127.0.0.1:54149/mod
--- FAIL: TestScript (0.16s)
    --- FAIL: TestScript/test_fuzz_limit_dup_entry (0.18s)
        script_test.go:134: 2023-03-14T22:22:59Z
        script_test.go:136: $WORK=C:\Users\gopher\AppData\Local\Temp\1\cmd-go-test-2509494391\tmpdir1435811394\test_fuzz_limit_dup_entry3766149940
        script_test.go:154: 
            > [!fuzz] skip
            [condition not met]
            > [short] skip
            [condition not met]
            # FuzzA attempts to cause the mutator to create duplicate inputs that generate
            # new coverage. Previously this would trigger a corner case when the fuzzer
            # had a execution limit, causing it to deadlock and sit in the coordinator
            # loop indefinitely, failing to exit once the limit had been exhausted. (0.088s)
            > go clean --fuzzcache
            [stderr]
            go: remove C:\Users\gopher\AppData\Local\go-build\fuzz\fuzz\two\FuzzTwo\a8f6d909ad64d071: The process cannot access the file because it is being used by another process.
        script_test.go:154: FAIL: test_fuzz_limit_dup_entry.txt:9: go clean --fuzzcache: exit status 1

— watchflakes

@gopherbot
Copy link

Change https://go.dev/cl/476755 mentions this issue: cmd/go/testdata/script: consistently set GOCACHE in fuzz tests

@gopherbot
Copy link

Change https://go.dev/cl/476815 mentions this issue: internal/fuzz: release lock when reading file fails

gopherbot pushed a commit that referenced this issue Mar 16, 2023
@rolandshoemaker @gopherbot
cmd/go/testdata/script: consistently set GOCACHE in fuzz tests
f594a3e 
The fuzzing cache for interesting inputs is shared across all
invocations of scripts by default. When 'go clean -fuzzcache' is called,
or fuzz targets in different scripts have the same names, we can get
race-y unexpected behavior.

Since there isn't a easy way to set just the fuzz cache directory (test
has the flag -test.fuzzcachedir, but it requires setting it on each call
to 'go test'), instead we just consistently set GOCACHE to point to a
directory in the WORK dir. As a byproduct this also prevents usage of a
shared build cache, so we see an increase in build time for these tests.

Updates #59062

Change-Id: Ie78f2943b94f3302c5bdf1f8a1e93b207853666a
Reviewed-on: https://go-review.googlesource.com/c/go/+/476755
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Roland Shoemaker <roland@golang.org>
Auto-Submit: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopherbot pushed a commit that referenced this issue Mar 16, 2023
@rolandshoemaker @gopherbot
internal/fuzz: release lock when reading file fails
e91876f 
When corpusEntryData failed in workerClient.fuzz and
workerClient.minimize, the shared memory mutex wasn't properly given up,
which would cause a deadlock when worker.cleanup was called.

This was tickled by #59062, wherein the fuzz cache directory would be
removed during operation of the fuzzer, causing corpusEntryData to fail
because the entry files no longer existed.

Updates #51484

Change-Id: Iea284041c20d1581c662bddbbc7e12191771a364
Reviewed-on: https://go-review.googlesource.com/c/go/+/476815
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Roland Shoemaker <roland@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
@bcmills
Copy link
Contributor Author

bcmills commented Apr 19, 2023 •
edited

@rolandshoemaker, was this fixed by CL 476815?

@rolandshoemaker
Copy link
Member

It seems so 🎉.

@bcmills bcmills closed this as completed Apr 19, 2023
@golang golang locked and limited conversation to collaborators Apr 18, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@rolandshoemaker rolandshoemaker

Labels
FrozenDueToAge GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done. Testing An issue that has been verified to require only test changes, not just a test failure.
Projects
Test Flakes
Status: Done
Milestone
Go1.21
Development

No branches or pull requests
3 participants
@rolandshoemaker @bcmills @gopherbot