Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

testing: streaming output loses parallel subtest associations [1.14 backport] #39308

Closed
gopherbot opened this issue May 28, 2020 · 10 comments
Closed

testing: streaming output loses parallel subtest associations [1.14 backport] #39308

gopherbot opened this issue May 28, 2020 · 10 comments
Labels
CherryPickApproved Used during the release process for point releases FrozenDueToAge
Milestone
Go1.14.6

Comments

@gopherbot
Copy link

@bcmills requested issue #38458 to be considered for backport to the next 1.14 minor release.

@gopherbot, please backport to Go 1.14: the 1.14 output format breaks go test -json for parallel subtests, and backporting the fix to the output format is likely to be more maintainable than a one-off test2json fix that is only used for the 1.14 branch.
@gopherbot gopherbot added the CherryPickCandidate Used during the release process for point releases label May 28, 2020
@gopherbot gopherbot mentioned this issue May 28, 2020
Closed
@gopherbot gopherbot added this to the Go1.14.4 milestone May 28, 2020
@bcmills bcmills mentioned this issue May 28, 2020
Closed
@dmitshur dmitshur modified the milestones: Go1.14.4, Go1.14.5 Jun 1, 2020
@dmitshur
Copy link
Contributor

dmitshur commented Jun 4, 2020

@bcmills Do I understand correctly that backporting this to 1.14 will improve readability of the test output for humans, and fix a bug in go test -json (used by machines), so the behavior change from backporting it should be safe for a minor release?

Does the fix involve backporting both CL 229085 and CL 235997?

@bcmills
Copy link
Contributor

bcmills commented Jun 4, 2020 •
edited

Yes, your understanding of the behavior change is correct.

I would recommend backporting both changes, although only the first one is strictly necessary to fix the bug. You'll also want to pull in CL 234978 either way.

@dmitshur
Copy link
Contributor

dmitshur commented Jun 5, 2020

Thanks.

We've discussed this in a release meeting yesterday, and while it's unfortunate to be making a change, it is fixing a serious issue without a workaround. Approving for 1.14 (this issue doesn't affect 1.13).

@dmitshur dmitshur added CherryPickApproved Used during the release process for point releases and removed CherryPickCandidate Used during the release process for point releases labels Jun 5, 2020
@chizhg chizhg mentioned this issue Jun 9, 2020
Closed
@mdwhatcott
Copy link

@bcmills - Is this on track for 1.14.5?

@medyagh
Copy link

medyagh commented Jun 16, 2020

in minikube CI we have a bug (kubernetes/minikube#8349) that depends on fixing this issue
more info here
#33419 (comment)

@medyagh medyagh mentioned this issue Jun 16, 2020
Closed
@gopherbot
Copy link
Author

Change https://golang.org/cl/242058 mentions this issue: [release-branch.go1.14] testing: capture testname on --- PASS and --- FAIL lines

@gopherbot
Copy link
Author

Change https://golang.org/cl/242057 mentions this issue: [release-branch.go1.14] testing: reformat test chatty output

@gopherbot
Copy link
Author

Change https://golang.org/cl/242059 mentions this issue: [release-branch.go1.14] cmd/go: fix parallel chatty tests on solaris-amd64 builder

@andybons andybons modified the milestones: Go1.14.5, Go1.14.6 Jul 14, 2020
@gopherbot
Copy link
Author

Closed by merging 2ba9d45 to release-branch.go1.14.

gopherbot pushed a commit that referenced this issue Jul 16, 2020
@jeanbza @dmitshur
[release-branch.go1.14] testing: reformat test chatty output
2ba9d45 
In #24929, we decided to stream chatty test output. It looks like,

foo_test.go:138: TestFoo/sub-1: hello from subtest 1
foo_test.go:138: TestFoo/sub-2: hello from subtest 2

In this CL, we refactor the output to be grouped by === CONT lines, preserving
the old test-file-before-log-line behavior:

=== CONT TestFoo/sub-1
    foo_test.go:138 hello from subtest 1
=== CONT TestFoo/sub-2
    foo_test.go:138 hello from subtest 2

This should remove a layer of verbosity from tests, and make it easier to group
together related lines. It also returns to a more familiar format (the
pre-streaming format), whilst still preserving the streaming feature.

Updates #38458.
Fixes #39308.

Change-Id: Iaef94c580d69cdd541b2ef055aa004f50d72d078
Reviewed-on: https://go-review.googlesource.com/c/go/+/229085
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/242057
Reviewed-by: Jean de Klerk <deklerk@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Jean de Klerk <deklerk@google.com>
gopherbot pushed a commit that referenced this issue Jul 16, 2020
@tklauser @dmitshur
[release-branch.go1.14] cmd/go: fix parallel chatty tests on solaris-…
4298b46 
…amd64 builder

The parallel chatty tests added in CL 229085 fail on the
solaris-amd64-oraclerel builder, because a +NN:NN offset time zone is
used. Allow for the `+` character in the corresponding regex to fix
these tests. Also move the '-' to the end of the character class, so it
is not interpreted as the range 9-T.

Updates #38458.
For #39308.

Change-Id: Iec9ae82ba45d2490176f274f0dc6812666eae718
Reviewed-on: https://go-review.googlesource.com/c/go/+/234978
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/242059
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
gopherbot pushed a commit that referenced this issue Jul 16, 2020
@jeanbza @dmitshur
[release-branch.go1.14] testing: capture testname on --- PASS and ---…
3445c35 
… FAIL lines

This fixes an issue raised at #38458 (comment)
in which --- PASS and --- FAIL lines would not trigger --- CONT lines
of other tests.

Updates #38458.
For #39308.

Change-Id: I0d8cc54d682a370d0a6ea6816a11b2e462a92efe
Reviewed-on: https://go-review.googlesource.com/c/go/+/235997
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/242058
Reviewed-by: Jean de Klerk <deklerk@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
@dmitshur
Copy link
Contributor

Now that this change was submitted to the release branch, I'll remove release-blocker label to reduce visual noise when viewing the release history details.

dnephin added a commit to hashicorp/consul that referenced this issue Jul 17, 2020
@dnephin
ci: Use go1.14.6
b8a405b 
Includes the security patches from go14.5 and golang/go#39308
to fix our test logs.
@dnephin dnephin mentioned this issue Jul 17, 2020
Merged
@medyagh medyagh mentioned this issue Jul 20, 2020
Merged
This was referenced Jul 20, 2020
Closed
Merged
@davidvanlaatum davidvanlaatum mentioned this issue Aug 9, 2020
Closed
@bcmills bcmills mentioned this issue Aug 13, 2020
Closed
@golang golang locked and limited conversation to collaborators Jul 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
CherryPickApproved Used during the release process for point releases FrozenDueToAge
Projects
None yet
Milestone
Go1.14.6
Development

No branches or pull requests
6 participants
@mdwhatcott @andybons @dmitshur @medyagh @bcmills @gopherbot