Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: flaky TestScript/test_regexps failure #36664

Closed
josharian opened this issue Jan 21, 2020 · 4 comments
Closed

cmd/go: flaky TestScript/test_regexps failure #36664

josharian opened this issue Jan 21, 2020 · 4 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. Testing An issue that has been verified to require only test changes, not just a test failure.
Milestone
Go1.15

Comments

@josharian
Copy link
Contributor

Noticed in a trybot run, sandwiched between a bunch of commits in which the test passed.

https://storage.googleapis.com/go-build-log/254cd30d/openbsd-amd64-64_892a8698.log

--- FAIL: TestScript (0.00s)
    --- FAIL: TestScript/test_regexps (1.68s)
        script_test.go:193: 
            # Test the following:
            #	TestX is run, twice (0.001s)
            #	TestX/Y is run, twice (0.001s)
            #	TestXX is run, twice (0.000s)
            #	TestZ is not run (0.000s)
            #	BenchmarkX is run but only with N=1, once (0.000s)
            #	BenchmarkXX is run but only with N=1, once (0.000s)
            #	BenchmarkX/Y is run in full, twice (0.001s)
            > stdout -count=2 '    BenchmarkX/Y: x_test.go:15: LOG: Y running N=1\n    BenchmarkX/Y: x_test.go:15: LOG: Y running N=100\n    BenchmarkX/Y: x_test.go:15: LOG: Y running N=10000\n    BenchmarkX/Y: x_test.go:15: LOG: Y running N=1000000\n    BenchmarkX/Y: x_test.go:15: LOG: Y running N=100000000\n    BenchmarkX/Y: x_test.go:15: LOG: Y running N=1000000000'
            FAIL: testdata/script/test_regexps.txt:29: have 1 matches for `(?m)    BenchmarkX/Y: x_test.go:15: LOG: Y running N=1\n    BenchmarkX/Y: x_test.go:15: LOG: Y running N=100\n    BenchmarkX/Y: x_test.go:15: LOG: Y running N=10000\n    BenchmarkX/Y: x_test.go:15: LOG: Y running N=1000000\n    BenchmarkX/Y: x_test.go:15: LOG: Y running N=100000000\n    BenchmarkX/Y: x_test.go:15: LOG: Y running N=1000000000`, want 2
            
FAIL

cc @bcmills
@josharian josharian added this to the Go1.15 milestone Jan 21, 2020
@josharian josharian added the Testing An issue that has been verified to require only test changes, not just a test failure. label Jan 21, 2020
@ianlancetaylor
Copy link
Contributor

CC @jayconrod @matloob

@gopherbot
Copy link

Change https://golang.org/cl/215578 mentions this issue: cmd/go: make Script/test_regexps less flaky under load

@gopherbot
Copy link

Change https://golang.org/cl/215597 mentions this issue: cmd/stress: use an ellipsis when truncating output

@bcmills
Copy link
Contributor

bcmills commented Jan 21, 2020

This is the same underlying issue as #36074, transformed by the conversion to a test script in CL 213130.

@bcmills bcmills mentioned this issue Jan 21, 2020
Closed
@toothrot toothrot added the NeedsFix The path to resolution is known, but the work has not been done. label Jan 21, 2020
gopherbot pushed a commit to golang/tools that referenced this issue Jan 22, 2020
@mvdan
cmd/stress: use an ellipsis when truncating output
bf1340f 
While investigating golang/go#36664, the entire output of a failure
could get pretty long - past fifty lines of content. I was getting
confused as to why the output was being cut off.

I had to discard a number of theories (the test being buggy, my expected
output being wrong, etc) before I realised that something was truncating
the output for the sake of the reader.

Since that's what stress is doing, make it add an ellipsis character
too, to further hint to the user that the output is being truncated for
readability. Separate it with a newline to make it easier to notice.

Change-Id: Ie052c13bb48d6fd842d6054848161266d9ea17e6
Reviewed-on: https://go-review.googlesource.com/c/tools/+/215597
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@golang golang locked and limited conversation to collaborators Jan 22, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. Testing An issue that has been verified to require only test changes, not just a test failure.
Projects
None yet
Milestone
Go1.15
Development

No branches or pull requests
5 participants
@toothrot @josharian @ianlancetaylor @bcmills @gopherbot