Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime/coverage: confusing go test -cover + GOCOVERDIR behavior #66225

Open
hugelgupf opened this issue Mar 10, 2024 · 2 comments
Open

runtime/coverage: confusing go test -cover + GOCOVERDIR behavior #66225

hugelgupf opened this issue Mar 10, 2024 · 2 comments
Assignees
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@hugelgupf
Copy link
Contributor

Go version

go1.22

Output of go env in your module/workspace:

-

What did you do?

I've created an example here: https://github.com/hugelgupf/gocoverdir/blob/main/main_test.go

The gist is: I'd like to be able to use GOCOVERDIR with go test -cover.

What did you see happen?

go test -cover seems to create a temporary GOCOVERDIR, but does not expose that collected coverage literally anywhere.

Clone that repo, and run the following commands:

$ go test -v
=== RUN   TestXxx
    main_test.go:18: gocoverdir: 
    main_test.go:22: out: warning: GOCOVERDIR not set, no coverage data emitted
        hello world
--- PASS: TestXxx (0.28s)
PASS
ok      github.com/hugelgupf/gocoverdir 0.307s

ok, that seems expected.

$ go test -coverprofile=cover.out -v
=== RUN   TestXxx
    main_test.go:18: gocoverdir: /tmp/go-build4220322233/b001/gocoverdir
    main_test.go:22: out: hello world
--- PASS: TestXxx (0.35s)
PASS
coverage: 0.0% of statements
ok      github.com/hugelgupf/gocoverdir 0.378s

$ go tool cover -func=cover.out
github.com/hugelgupf/gocoverdir/main.go:7:      main            0.0%
total:                                          (statements)    0.0%

blowing a raspberry here. Why does go test create a GOCOVERDIR but not give me the results?

$ mkdir cover
$ GOCOVERDIR=cover go test -coverprofile=cover.out -v
=== RUN   TestXxx
    main_test.go:18: gocoverdir: /tmp/go-build4220322233/b001/gocoverdir
    main_test.go:22: out: hello world
--- PASS: TestXxx (0.35s)
PASS
coverage: 0.0% of statements
ok      github.com/hugelgupf/gocoverdir 0.378s

$ go tool cover -func=cover.out
github.com/hugelgupf/gocoverdir/main.go:7:      main            0.0%
total:                                          (statements)    0.0%

$ ls -l cover
total 0
$ go tool covdata func -i=cover
warning: no applicable files found in input directories

If I supply my own GOCOVERDIR, why is it not available?

$ GOCOVERDIR=cover go test -v
=== RUN   TestXxx
    main_test.go:18: gocoverdir: cover
    main_test.go:22: out: hello world
--- PASS: TestXxx (0.29s)
PASS
ok      github.com/hugelgupf/gocoverdir 0.315s

$ go tool covdata func -i=cover
github.com/hugelgupf/gocoverdir/main.go:7:      main            100.0%
total                                           (statements)    100.0%

the only combo that works!

What did you expect to see?

The use case of GOCOVERDIR=x go test -coverprofile is that I'd like to be able to have (a) mixed tests, and (b) be able to have one go test -cover ./... at the root of the project. This makes that hard, as a test that wants GOCOVERDIR can't be run with -cover.

In order of priority, I'd request:

  1. Document this behavior somewhere, because I don't think either the blog post nor go cmd docs document any of this.
  2. If you supply your own GOCOVERDIR, don't make go test set one.
  3. If no GOCOVERDIR is set, I'd like go test -cover to set it, collect the coverage, and expose it through the old text format with a conversion.
  4. GOCOVERDIR=./relative-path go test ./... is likely a bit weird, because it'll expect a relative-path directory in each package directory that it is running in. May be worth printing a warning and pointing the user to the go test -coverprofile=foo behavior auto-collecting GOCOVERDIR coverage in that case, as in point 3.
@hugelgupf
Copy link
Contributor Author

cc @thanm

@cherrymui cherrymui added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 11, 2024
@cherrymui cherrymui added this to the Backlog milestone Mar 11, 2024
@cherrymui cherrymui added compiler/runtime Issues related to the Go compiler and/or runtime. GoCommand cmd/go labels Mar 11, 2024
@thanm
Copy link
Contributor

thanm commented Mar 14, 2024

Thanks for the report. This issue is somewhat similar in nature to the existing #60182.

When you say "I'd like to be able to use GOCOVERDIR with go test -cover" do you mean that you want all of the coverage data files generated during the test to be copied over into the directory that GOCOVERDIR points to when the test completes?

Or are you more interested in a use case like the one in #60182 where the object is to ensure when a Go test builds (with -cover) a binary and runs it, that the resulting profiles get included in the final coverage number reported by the top level test?

I agree that things are not in an ideal state, and the documentation could be better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
Development

No branches or pull requests

3 participants