Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

testing: Tests timeout with Go 1.20.x, but pass with Go 1.19 and 1.18 in ~20 seconds. #58867

Closed
MarvinJWendt opened this issue Mar 4, 2023 · 4 comments
Closed

testing: Tests timeout with Go 1.20.x, but pass with Go 1.19 and 1.18 in ~20 seconds. #58867

MarvinJWendt opened this issue Mar 4, 2023 · 4 comments
Labels
FrozenDueToAge

Comments

@MarvinJWendt
Copy link

What version of Go are you using (go version)?

go version go1.20.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
  GOARCH="amd64"
  GOBIN=""
  GOCACHE="/home/runner/.cache/go-build"
  GOENV="/home/runner/.config/go/env"
  GOEXE=""
  GOEXPERIMENT=""
  GOFLAGS=""
  GOHOSTARCH="amd64"
  GOHOSTOS="linux"
  GOINSECURE=""
  GOMODCACHE="/home/runner/go/pkg/mod"
  GONOPROXY=""
  GONOSUMDB=""
  GOOS="linux"
  GOPATH="/home/runner/go"
  GOPRIVATE=""
  GOPROXY="https://proxy.golang.org,direct"
  GOROOT="/opt/hostedtoolcache/go/1.[20](https://github.com/pterm/pterm/actions/runs/4332579616/jobs/7565164242#step:2:21).1/x64"
  GOSUMDB="sum.golang.org"
  GOTMPDIR=""
  GOTOOLDIR="/opt/hostedtoolcache/go/1.20.1/x64/pkg/tool/linux_amd64"
  GOVCS=""
  GOVERSION="go1.20.1"
  GCCGO="gccgo"
  GOAMD64="v1"
  AR="ar"
  CC="gcc"
  CXX="g++"
  CGO_ENABLED="1"
  GOMOD="/dev/null"
  GOWORK=""
  CGO_CFLAGS="-O2 -g"
  CGO_CPPFLAGS=""
  CGO_CXXFLAGS="-O2 -g"
  CGO_FFLAGS="-O2 -g"
  CGO_LDFLAGS="-O2 -g"
  PKG_CONFIG="pkg-config"
  GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build33[22](https://github.com/pterm/pterm/actions/runs/4332579616/jobs/7565164242#step:2:23)00[46](https://github.com/pterm/pterm/actions/runs/4332579616/jobs/7565164242#step:2:47)00=/tmp/go-build -gno-record-gcc-switches"

What did you do?

My package tests pass on go 1.19 and below, but not on Go 1.20.x.
When I run the tests with go 1.19 they pass in 19 seconds. When I do the same with Go > 1.20 they fail on a timeout after 11 minutes. I have verified this behavior on multiple systems and operating systems (pipeline matrix test and local).
I used -v -p 1 as tests flags, and it seems like random tests are hanging.

I have also verified that the tests pass on Go 1.18.

Pipeline runs

Both pipelines runs are on an ubuntu machine. No code has been modified between the runs, except for the workflows go version.

Passed tests (go 1.19): https://github.com/pterm/pterm/actions/runs/4332770035/jobs/7565480959
Failed tests (go 1.20.1): https://github.com/pterm/pterm/actions/runs/4332579616/jobs/7565164242

What did you expect to see?

Tests passing

What did you see instead?

Test failing
@MarvinJWendt MarvinJWendt changed the title testing: Tests fail with Go >= 1.20, but work with Go 1.19 and 1.18 testing: Tests timeout with Go >= 1.20, but pass with Go 1.19 and 1.18 in ~20 seconds. Mar 4, 2023
@MarvinJWendt MarvinJWendt changed the title testing: Tests timeout with Go >= 1.20, but pass with Go 1.19 and 1.18 in ~20 seconds. testing: Tests timeout with Go 1.20.x, but pass with Go 1.19 and 1.18 in ~20 seconds. Mar 4, 2023
@fzipp
Copy link
Contributor

fzipp commented Mar 4, 2023

The code seems to be riddled with data races, see go test -race .

@MarvinJWendt
Copy link
Author

MarvinJWendt commented Mar 5, 2023 •
edited

Hi @fzipp yes it looks like that, but that never failed the tests since go 1.15.
I am also running the tests with -p 1 to prevent multiple tests at the same time (that should help with the races, right?).
I am curious why go 1.20 constantly lets the tests fail.
Seems like this repo had a similar issue with go 1.20: concourse/concourse#8680
They also use Stdin inside their tests, maybe that's the reason?
Unfortunately, I use Stdin and Stdout in most tests, so it's not as easy to fix as they did.

I tried to find the commit that broke the tests by git bisecting the Go buildchain, but I am not experienced in building Go and sometimes the go build crashes (I guess my system is just not set up properly for building Go).

Fixing the race conditions is an open PR in my library, but also the biggest refactor we ever had (and most likely a breaking change). So we have to be very careful with it.

My current workaround is to use Go 1.19 in the pipeline, but that shouldn't be a requirement to run our tests.

Do you have any idea how I could instruct Go 1.20 to work with the tests? Or maybe a hint what change in Go broke this?

@qiulaidongfeng
Copy link
Contributor

I use go1.20.1 to run
Go test - face - test. cpu=1 -trimpath There is still data competition
race.txt

Go1.20 New fields for exec.Cmd
Cancel, if Cancel is not nil, then Cmd must be created with CommandContext
WaitDelay, the default value is 0, then I/O pipes will be read before EOF. I remember that the default behavior is the same before.
If you don't use the new fields in go1.20 exec.Cmd, you can fix the data competition first.

@ianlancetaylor
Copy link
Contributor

If the code has data races, then we aren't going to investigate why the behavior has changed. Sorry. If someone wants to take a look please go ahead, but we're unlikely to make any changes to the Go tools because of racy code.

@ianlancetaylor ianlancetaylor closed this as not planned Won't fix, can't repro, duplicate, stale Mar 5, 2023
@moshe-blox moshe-blox mentioned this issue Aug 28, 2023
Merged
@golang golang locked and limited conversation to collaborators Mar 4, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@fzipp @ianlancetaylor @gopherbot @MarvinJWendt @qiulaidongfeng