Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: go 1.22.0 inconsistent and variable code coverage output for the same folder #65570

Open
dev-gto opened this issue Feb 7, 2024 · 10 comments
Open

cmd/go: go 1.22.0 inconsistent and variable code coverage output for the same folder #65570

dev-gto opened this issue Feb 7, 2024 · 10 comments
Assignees
@thanm
Labels
GoCommand cmd/go
Milestone
Backlog

Comments

@dev-gto
Copy link

dev-gto commented Feb 7, 2024

Go version

go version go1.22.0 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/hamada/.cache/go-build'
GOENV='/home/hamada/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/hamada/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/hamada/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go1.22.0'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go1.22.0/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/hamada/go/src/gitlab.in.registro.br/dev/br/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3637209596=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Having a directory called, e.g., holiday with some go files and their corresponding test files, I called the go test -cover from the parent directory. Depending on directories arguments, I have different percentage values, which wasn't occurring on go < 1.22.0.

What did you see happen?

DEV=dev go test -tags dev -cover -buildvcs=false -ldflags '-s -w' ./holiday/
ok  	gitlab.in.registro.br/dev/br/core/holiday	(cached)	coverage: 97.6% of statements

DEV=dev go test -tags dev -cover -buildvcs=false -ldflags '-s -w' ./h*
...
ok  	gitlab.in.registro.br/dev/br/core/holiday	0.005s	coverage: 47.3% of statements

DEV=dev go test -tags dev -cover -buildvcs=false -ldflags '-s -w' ./...
...
ok  	gitlab.in.registro.br/dev/br/core/holiday	(cached)	coverage: 23.5% of statements
...

What did you expect to see?

A consistent percentage value (in this case, 97.6%) for all commands:

DEV=dev go test -tags dev -cover -buildvcs=false -ldflags '-s -w' ./holiday/
ok  	gitlab.in.registro.br/dev/br/core/holiday	(cached)	coverage: 97.6% of statements

DEV=dev go test -tags dev -cover -buildvcs=false -ldflags '-s -w' ./h*
...
ok  	gitlab.in.registro.br/dev/br/core/holiday	0.005s	coverage: 97.6% of statements

DEV=dev go test -tags dev -cover -buildvcs=false -ldflags '-s -w' ./...
...
ok  	gitlab.in.registro.br/dev/br/core/holiday	(cached)	coverage: 97.6% of statements
...
@dev-gto dev-gto changed the title go 1.22.0: inconsistent and variable code coverage output for the same folder cmd/go: go 1.22.0 inconsistent and variable code coverage output for the same folder Feb 7, 2024
@bcmills
Copy link
Contributor

bcmills commented Feb 7, 2024

This is almost certainly an effect of recent changes in -coverpkg:

See previously:

(attn @thanm)

@bcmills bcmills added the GoCommand cmd/go label Feb 7, 2024
@bcmills bcmills added this to the Backlog milestone Feb 7, 2024
@thanm thanm self-assigned this Feb 7, 2024
@thanm
Copy link
Contributor

thanm commented Feb 7, 2024

Thanks for the report. I'll take a look.

@thanm
Copy link
Contributor

thanm commented Feb 7, 2024

@dev-gto would you kindly post a pointer to the code that you are working with, so that I can reproduce the problem on my machine? Thanks.

@thanm thanm added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Feb 7, 2024
@dev-gto
Copy link
Author

dev-gto commented Feb 7, 2024 •
edited

@thanm , actually my project is a private and large one.

Despite some testing errors (the original project does not emit any errors), you can reproduce with
https://github.com/registrobr/zxcvbn-go

go test -cover ./...
ok  	github.com/registrobr/zxcvbn-go	(cached)	coverage: 60.6% of statements
ok  	github.com/registrobr/zxcvbn-go/adjacency	(cached)	coverage: 40.8% of statements
	github.com/registrobr/zxcvbn-go/data		coverage: 0.0% of statements
	github.com/registrobr/zxcvbn-go/data/ptbr		coverage: 0.0% of statements
ok  	github.com/registrobr/zxcvbn-go/entropy	(cached)	coverage: 35.9% of statements
...
go test -cover ./[ae]*
ok  	github.com/registrobr/zxcvbn-go/adjacency	(cached)	coverage: 83.3% of statements
ok  	github.com/registrobr/zxcvbn-go/entropy	(cached)	coverage: 50.0% of statements

@AlexanderYastrebov
Copy link
Contributor

#62212 might be related

@aaron42net
Copy link

GOEXPERIMENT=nocoverageredesign go test -cover ./... does appear to provide the expected coverage result, matching running go test -cover for each package separately. So this does seem related to the coverage redesign.

I've confirmed this with both @dev-gto's example above and a large private monorepo where I also encountered the issue.

@dev-gto
Copy link
Author

dev-gto commented Feb 9, 2024

Indeed, GOEXPERIMENT=nocoverageredesign returns the expected values on my original files

@alainstoffels alainstoffels mentioned this issue Feb 9, 2024
Open
@jhendrixMSFT jhendrixMSFT mentioned this issue Feb 16, 2024
Merged
@jasonphi jasonphi mentioned this issue Feb 16, 2024
Closed
@seankhliao seankhliao removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Feb 16, 2024
@AlexanderYastrebov AlexanderYastrebov mentioned this issue Feb 19, 2024
Merged
@xuezhaojun xuezhaojun mentioned this issue Mar 4, 2024
Merged
@dev-gto
Copy link
Author

dev-gto commented Apr 5, 2024

hi!
Any news on this? go 1.22.2 still has this issue.

@jehowell
Copy link

Code coverage has been broken for about a year now... Please can someone make this a priority to fix.

@c-pius c-pius mentioned this issue Apr 18, 2024
Closed
3 tasks
@sfc-gh-jlangefeld
Copy link

I came here because I just ran into this as well on 1.22.2

This was referenced Apr 27, 2024
Closed
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@thanm thanm

Labels
GoCommand cmd/go
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
8 participants
@AlexanderYastrebov @jehowell @bcmills @aaron42net @seankhliao @thanm @dev-gto @sfc-gh-jlangefeld