Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: test -coverprofile with swig fails #64661

Closed
jackielii opened this issue Dec 11, 2023 · 4 comments
Closed

cmd/go: test -coverprofile with swig fails #64661

jackielii opened this issue Dec 11, 2023 · 4 comments
Assignees
@thanm
Labels
GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Go1.22

Comments

@jackielii
Copy link

jackielii commented Dec 11, 2023 •
edited

Go version

go version go1.21.2 darwin/arm64

What operating system and processor architecture are you using (go env)?

GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/jackieli/Library/Caches/go-build'
GOENV='/Users/jackieli/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/jackieli/go/pkg/mod'
GONOPROXY='tespkg.in,github.com/tespkg,gitlab.com/target-digital-transformation'
GONOSUMDB='tespkg.in,github.com/tespkg,gitlab.com/target-digital-transformation'
GOOS='darwin'
GOPATH='/Users/jackieli/go'
GOPRIVATE='tespkg.in,github.com/tespkg,gitlab.com/target-digital-transformation'
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/homebrew/Cellar/go/1.21.2/libexec'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/opt/homebrew/Cellar/go/1.21.2/libexec/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.21.2'
GCCGO='gccgo'
AR='ar'
CC='cc'
CXX='c++'
CGO_ENABLED='1'
GOMOD='/Users/jackieli/Downloads/simple/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/v7/nkqv5k_906s7gq6l4h30kq_w0000gn/T/go-build3418952473=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

Use a simple project with a swig file in it: simple-test.zip

Run go test works
Run go test -coverprofile .testCoverage.txt fails with message:

go build simple:
# cover simple
2023/12/11 22:43:50 cover: open $WORK/b102//var/folders/v7/nkqv5k_906s7gq6l4h30kq_w0000gn/T/go-build368947532/b102/_main_swig.cover.go: no such file or directory
FAIL	simple [build failed]

What did you expect to see?

go test -coverprofile works

What did you see instead?

go test -coverprofile .testCoverage.txt fails with message:

go build simple:
# cover simple
2023/12/11 22:43:50 cover: open $WORK/b102//var/folders/v7/nkqv5k_906s7gq6l4h30kq_w0000gn/T/go-build368947532/b102/_main_swig.cover.go: no such file or directory
FAIL	simple [build failed]

Additionally the output when with go test -x -coverprofile .testCoverage.txt

go test -x -coverprofile .testCoverage.txt 
  WORK=/var/folders/v7/nkqv5k_906s7gq6l4h30kq_w0000gn/T/go-build821224849
  mkdir -p $WORK/b001/
  mkdir -p $WORK/b020/
  mkdir -p $WORK/b013/
  mkdir -p $WORK/b016/
  mkdir -p $WORK/b014/
  mkdir -p $WORK/b009/
  mkdir -p $WORK/b007/
  mkdir -p $WORK/b035/
  mkdir -p $WORK/b025/
  mkdir -p $WORK/b037/
  mkdir -p $WORK/b015/
  mkdir -p $WORK/b041/
  mkdir -p $WORK/b018/
  mkdir -p $WORK/b045/
  mkdir -p $WORK/b019/
  mkdir -p $WORK/b071/
  mkdir -p $WORK/b089/
  mkdir -p $WORK/b090/
  mkdir -p $WORK/b096/
  mkdir -p $WORK/b027/
  mkdir -p $WORK/b012/
  mkdir -p $WORK/b006/
  mkdir -p $WORK/b017/
  mkdir -p $WORK/b011/
  mkdir -p $WORK/b039/
  mkdir -p $WORK/b010/
  mkdir -p $WORK/b005/
  mkdir -p $WORK/b026/
  mkdir -p $WORK/b032/
  mkdir -p $WORK/b004/
  mkdir -p $WORK/b049/
  mkdir -p $WORK/b036/
  mkdir -p $WORK/b052/
  mkdir -p $WORK/b024/
  mkdir -p $WORK/b030/
  mkdir -p $WORK/b034/
  mkdir -p $WORK/b051/
  mkdir -p $WORK/b028/
  mkdir -p $WORK/b062/
  mkdir -p $WORK/b086/
  mkdir -p $WORK/b043/
  mkdir -p $WORK/b050/
  mkdir -p $WORK/b040/
  mkdir -p $WORK/b048/
  mkdir -p $WORK/b061/
  mkdir -p $WORK/b058/
  mkdir -p $WORK/b038/
  mkdir -p $WORK/b081/
  mkdir -p $WORK/b047/
  mkdir -p $WORK/b023/
  mkdir -p $WORK/b067/
  mkdir -p $WORK/b080/
  mkdir -p $WORK/b031/
  mkdir -p $WORK/b070/
  mkdir -p $WORK/b088/
  mkdir -p $WORK/b029/
  mkdir -p $WORK/b022/
  mkdir -p $WORK/b056/
  mkdir -p $WORK/b033/
  mkdir -p $WORK/b021/
  mkdir -p $WORK/b003/
  mkdir -p $WORK/b053/
  mkdir -p $WORK/b046/
  mkdir -p $WORK/b044/
  mkdir -p $WORK/b055/
  mkdir -p $WORK/b054/
  mkdir -p $WORK/b094/
  mkdir -p $WORK/b092/
  mkdir -p $WORK/b099/
  mkdir -p $WORK/b069/
  mkdir -p $WORK/b095/
  mkdir -p $WORK/b091/
  mkdir -p $WORK/b101/
  mkdir -p $WORK/b093/
  mkdir -p $WORK/b042/
  mkdir -p $WORK/b097/
  mkdir -p $WORK/b098/
  mkdir -p $WORK/b100/
  mkdir -p $WORK/b087/
  mkdir -p $WORK/b103/
  cp /Users/jackieli/Library/Caches/go-build/9b/9b2bb0e3ac1b16efdbc8ca5a95d2ae084340b1f9d5e8300071e4d4f18e9a6227-d $WORK/b103/_cgo_gotypes.go
  cp /Users/jackieli/Library/Caches/go-build/ab/ab115b8041115aec8fccc5362216233eb3640d462533a86dbf27e17740c7cefc-d $WORK/b103/cgo.cgo1.go
  cp /Users/jackieli/Library/Caches/go-build/e7/e7322c53a39b536e027f8264f29ab9c8263c129048e4fd7c0381c646e5998571-d $WORK/b103/_cgo_import.go
  mkdir -p $WORK/b102/
  cd .
  swig -version
  cd /Users/jackieli/Downloads/simple
  /opt/homebrew/Cellar/go/1.21.2/libexec/pkg/tool/darwin_arm64/compile -o $WORK/b102/_go_.o -trimpath "$WORK/b102=>" -p main -complete -goversion go1.21.2 -c=4 -shared -nolocalimports $WORK/swig_intsize.go
  swig -go -cgo -intgosize 64 -module main -o $WORK/b102/main_wrap.c -outdir $WORK/b102/ main.swig
  mv $WORK/b102/main.go $WORK/b102/_main_swig.go
  cat >/var/folders/v7/nkqv5k_906s7gq6l4h30kq_w0000gn/T/go-build821224849/b102/pkgcfg.txt << 'EOF' # internal
  {"OutConfig":"/var/folders/v7/nkqv5k_906s7gq6l4h30kq_w0000gn/T/go-build821224849/b102/coveragecfg","PkgPath":"simple","PkgName":"main","Granularity":"perblock","ModulePath":"simple","Local":false}EOF
  cat >/var/folders/v7/nkqv5k_906s7gq6l4h30kq_w0000gn/T/go-build821224849/b102/coveroutfiles.txt << 'EOF' # internal
  /var/folders/v7/nkqv5k_906s7gq6l4h30kq_w0000gn/T/go-build821224849/b102/covervars.go
  /var/folders/v7/nkqv5k_906s7gq6l4h30kq_w0000gn/T/go-build821224849/b102/main.cover.go
  /var/folders/v7/nkqv5k_906s7gq6l4h30kq_w0000gn/T/go-build821224849/b102//var/folders/v7/nkqv5k_906s7gq6l4h30kq_w0000gn/T/go-build821224849/b102/_main_swig.cover.go
  EOF
  cd $WORK/b102/
  /opt/homebrew/Cellar/go/1.21.2/libexec/pkg/tool/darwin_arm64/cover -pkgcfg ./pkgcfg.txt -mode set -var goCover_a7a39b72f297_ -outfilelist ./coveroutfiles.txt /Users/jackieli/Downloads/simple/main.go /Users/jackieli/Downloads/simple$WORK/b102/_main_swig.go
  go build simple:
  # cover simple
  2023/12/11 22:45:14 cover: open $WORK/b102//var/folders/v7/nkqv5k_906s7gq6l4h30kq_w0000gn/T/go-build821224849/b102/_main_swig.cover.go: no such file or directory
  rm -r $WORK/b001/
  FAIL    simple [build failed]

Also note that last version I used for this project go1.18 works. I also tested go1.21 on Linux/amd64, same error.

/cc @ianlancetaylor
@prattmic prattmic changed the title go/test: test -coverprofile with swig fails cmd/go: test -coverprofile with swig fails Dec 12, 2023
@prattmic
Copy link
Member

cc @thanm

@prattmic prattmic added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Dec 12, 2023
@prattmic prattmic added this to the Backlog milestone Dec 12, 2023
@bcmills bcmills added the GoCommand cmd/go label Dec 12, 2023
@thanm
Copy link
Contributor

thanm commented Dec 12, 2023

Thanks for the report. I'll take a look.

@thanm thanm self-assigned this Dec 12, 2023
@thanm
Copy link
Contributor

thanm commented Dec 21, 2023

I spent a little time looking at this. When using Go 1.19 or with Go 1.20 plus the "new coverage experiment" disabled (GOEXPERIMENT=nocoverageredesign), the swig-generated source file (_main_swig.go) is never actually passed to the cover tool. This is due to the fact that only those files processed by DeclareCoverVars will be instrumented. The new coverage implementation gets rid of DeclareCoverVars, and as a result what happens is that _main_swig.go gets added to the package list of cgo files here and then processed along with everything else. This should be relatively straightforward to fix -- just run the cover tool on the original list of go and cgo files.

@gopherbot
Copy link

Change https://go.dev/cl/552095 mentions this issue: cmd/go: run cover tool before swig

@dmitshur dmitshur modified the milestones: Backlog, Go1.22 Jan 8, 2024
@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jan 8, 2024
ezz-no pushed a commit to ezz-no/go-ezzno that referenced this issue Feb 18, 2024
@thanm @ezz-no
cmd/go: run cover tool before swig
8a61ae9 
When building a package, run the cover tool on the collected go/cgo
source files before invoking swig (if swig files are present), as
opposed to running swig and then cover. Running swig adds new Go files
to the "cgo" list, and we want to avoid running those newly generated
files through the cover tool.

Fixes golang#64661.

Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Change-Id: I32b6dad5c39fcf5e656c40fb3b44220c69320889
Reviewed-on: https://go-review.googlesource.com/c/go/+/552095
Auto-Submit: Than McIntosh <thanm@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@thanm thanm

Labels
GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Go1.22
Development

No branches or pull requests
6 participants
@jackielii @prattmic @dmitshur @bcmills @gopherbot @thanm