Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: coverpkg flag should be part of cache key #66504

Open
mitar opened this issue Mar 24, 2024 · 4 comments
Open

cmd/go: coverpkg flag should be part of cache key #66504

mitar opened this issue Mar 24, 2024 · 4 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.

Comments

@mitar
Copy link
Contributor

mitar commented Mar 24, 2024 •
edited

Go version

go version go1.22.1 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.1'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='0'
GOMOD='/code/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3544536393=/tmp/go-build -gno-record-gcc-switches'

What did you do?

On this example repository I ran:

go test -covermode=atomic -cover -count 1 -coverpkg=./... ./...

and:

go test -covermode=atomic -cover -coverpkg=./... ./...

What did you see happen?

I expected in both cases all coverage numbers to be the same.

(I have the problem with coverage numbers themselves, but I will report that separately.)

What did you expect to see?

Different numbers:

$ go test -covermode=atomic -cover -count 1 -coverpkg=./... ./...
ok  	github.com/mitar/accurate-test-coverage	0.002s	coverage: 0.0% of statements in ./... [no tests to run]
ok  	github.com/mitar/accurate-test-coverage/otherpkg	0.002s	coverage: 75.0% of statements in ./...
ok  	github.com/mitar/accurate-test-coverage/pkg	0.003s	coverage: 25.0% of statements in ./...

$ go test -covermode=atomic -cover -coverpkg=./... ./...
ok  	github.com/mitar/accurate-test-coverage	(cached)	coverage: 0.0% of statements in ./... [no tests to run]
ok  	github.com/mitar/accurate-test-coverage/otherpkg	(cached)	coverage: 100.0% of statements in ./...
ok  	github.com/mitar/accurate-test-coverage/pkg	(cached)	coverage: 33.3% of statements in ./...
@mitar
Copy link
Contributor Author

mitar commented Mar 24, 2024

(attn @thanm)

@mitar mitar mentioned this issue Mar 24, 2024
Open
@seankhliao
Copy link
Member

I can't reproduce your result for 100% and 33% ?
It consistently reports 75% and 25%

@mitar
Copy link
Contributor Author

mitar commented Mar 24, 2024 •
edited

Interesting. I did try before inside Docker container to make sure it is reproducible, but I was also playing a lot with contents of files. I think something got wrong with the cache. If I try now inside clean container with clean checkout without any prior cache, it works as expected.

This is now full reproduction:

$ docker run --rm -t -i golang:1.22.1-alpine3.18 sh
/go # apk add git
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz
(1/8) Installing brotli-libs (1.0.9-r14)
(2/8) Installing libunistring (1.1-r1)
(3/8) Installing libidn2 (2.3.4-r1)
(4/8) Installing nghttp2-libs (1.57.0-r0)
(5/8) Installing libcurl (8.5.0-r0)
(6/8) Installing libexpat (2.6.2-r0)
(7/8) Installing pcre2 (10.42-r1)
(8/8) Installing git (2.40.1-r0)
Executing busybox-1.36.1-r5.trigger
OK: 18 MiB in 24 packages
/go # cd /
/ # git clone https://github.com/mitar/accurate-test-coverage
Cloning into 'accurate-test-coverage'...
remote: Enumerating objects: 35, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 35 (delta 0), reused 6 (delta 0), pack-reused 28
Receiving objects: 100% (35/35), done.
Resolving deltas: 100% (4/4), done.
/ # cd accurate-test-coverage/
/accurate-test-coverage # rm main.go 
/accurate-test-coverage # go test -covermode=atomic -cover -coverpkg=./... ./...
ok  	github.com/mitar/accurate-test-coverage	0.002s	coverage: 0.0% of statements in ./... [no tests to run]
ok  	github.com/mitar/accurate-test-coverage/otherpkg	0.002s	coverage: 100.0% of statements in ./...
ok  	github.com/mitar/accurate-test-coverage/pkg	0.002s	coverage: 33.3% of statements in ./...
/accurate-test-coverage # git checkout .
Updated 1 path from the index
/accurate-test-coverage # go test -covermode=atomic -cover -coverpkg=./... ./...
ok  	github.com/mitar/accurate-test-coverage	0.003s	coverage: 0.0% of statements in ./... [no tests to run]
ok  	github.com/mitar/accurate-test-coverage/otherpkg	(cached)	coverage: 100.0% of statements in ./...
ok  	github.com/mitar/accurate-test-coverage/pkg	(cached)	coverage: 33.3% of statements in ./...
/accurate-test-coverage # go test -covermode=atomic -cover -count 1 -coverpkg=./... ./...
ok  	github.com/mitar/accurate-test-coverage	0.003s	coverage: 0.0% of statements in ./... [no tests to run]
ok  	github.com/mitar/accurate-test-coverage/otherpkg	0.003s	coverage: 75.0% of statements in ./...
ok  	github.com/mitar/accurate-test-coverage/pkg	0.003s	coverage: 25.0% of statements in ./...

@mitar
Copy link
Contributor Author

mitar commented Mar 24, 2024

I think -coverpkg should force all tests to re-run. Or cache should depend on -coverpkg.

@seankhliao seankhliao added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 24, 2024
@seankhliao seankhliao changed the title cmd/go: test coverage computed differently if count=1 is used cmd/go: coverpkg flag should be part of cache key Mar 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mitar @seankhliao