Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/link: fingerprint mismatch: after adding default.pgo #61376

Closed
qiulaidongfeng opened this issue Jul 16, 2023 · 7 comments
Closed

cmd/link: fingerprint mismatch: after adding default.pgo #61376

qiulaidongfeng opened this issue Jul 16, 2023 · 7 comments
Assignees
@cherrymui
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Milestone
Go1.21

Comments

@qiulaidongfeng
Copy link
Contributor

What version of Go are you using (go version)?

$ go version go1.21rc3 linux/amd64

Does this issue reproduce with the latest release?

yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/qiu-lai-dong-feng/.cache/go-build'
GOENV='/home/qiu-lai-dong-feng/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/qiu-lai-dong-feng/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/qiu-lai-dong-feng/go/'
GOPRIVATE=''
GOPROXY='https://goproxy.cn,https://proxy.golang.org,direct'
GOROOT='/home/qiu-lai-dong-feng/.go/current/'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/home/qiu-lai-dong-feng/.go/current/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21rc3'
GCCGO='/usr/bin/gccgo'
GOAMD64='v1'
AR='ar'
CC='clang'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/qiu-lai-dong-feng/go/test/pkg/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2593822558=/tmp/go-build -gno-record-gcc-switches'

What did you do?

git clone https://gitee.com/u-language/u-language.git test
cd test
cd pkg
su
go test ./...

What did you expect to see?

Normal operation test.

What did you see instead?

output

# gitee.com/u-language/u-language/pkg.test

/home/qiu-lai-dong-feng/.go/current/pkg/tool/linux_amd64/link: fingerprint mismatch: os has d62759c2b920f031, import from gitee.com/u-language/u-language/pkg expecting 18ac6ddfd6f53c08
? gitee.com/u-language/u-language/pkg/astdata [no test files]
FAIL gitee.com/u-language/u-language/pkg [build failed]
ok gitee.com/u-language/u-language/pkg/ast 0.037s
ok gitee.com/u-language/u-language/pkg/cast 0.006s
? gitee.com/u-language/u-language/pkg/config [no test files]
ok gitee.com/u-language/u-language/pkg/check 0.014s
? gitee.com/u-language/u-language/pkg/enum [no test files]
? gitee.com/u-language/u-language/pkg/errcode [no test files]
ok gitee.com/u-language/u-language/pkg/data 0.029s
? gitee.com/u-language/u-language/pkg/format [no test files]
? gitee.com/u-language/u-language/pkg/internal/debug [no test files]
? gitee.com/u-language/u-language/pkg/internal/errutil [no test files]
ok gitee.com/u-language/u-language/pkg/fasm/amd64 0.007s
? gitee.com/u-language/u-language/pkg/internal/parser [no test files]
? gitee.com/u-language/u-language/pkg/internal/teco [no test files]
? gitee.com/u-language/u-language/pkg/internal/time [no test files]
? gitee.com/u-language/u-language/pkg/internal/utils [no test files]
ok gitee.com/u-language/u-language/pkg/internal/math/big 0.011s [no tests to run]
ok gitee.com/u-language/u-language/pkg/ir 0.008s
? gitee.com/u-language/u-language/pkg/parser [no test files]
ok gitee.com/u-language/u-language/pkg/lex 0.013s
@seankhliao seankhliao changed the title affected/package: Fingerprint mismatch after adding default.pgo cmd/link: fingerprint mismatch: after adding default.pgo Jul 16, 2023
@seankhliao seankhliao added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 16, 2023
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 16, 2023
@mauri870
Copy link
Member

mauri870 commented Jul 17, 2023 •
edited

I tried replicating this following the provided steps and it happens for me on tip as well. The fingerprint mismatch message relates to the symbol pkg.a -> _go.o -> main

@mauri870
Copy link
Member

Okay I was able to bisect this testing with -pgo=auto and found it started failing after this commit 74502e9bb4.

@seankhliao
Copy link
Member

cc @cherrymui

@prattmic
Copy link
Member

cc @golang/compiler

@cherrymui
Copy link
Member

Thanks for report. I can reproduce as well. Some observations: in my run of go test ./..., it compiles the os package without PGO as b003, os package with PGO as b141. When compiling the gitee.com/u-language/u-language/pkg package (with PGO automatically), the importcfg is

packagefile bufio=/tmp/go-build892594869/b059/_pkg_.a
packagefile context=/tmp/go-build892594869/b057/_pkg_.a
packagefile fmt=/tmp/go-build892594869/b047/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/ast=/tmp/go-build892594869/b089/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/cast=/tmp/go-build892594869/b099/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/check=/tmp/go-build892594869/b100/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/config=/tmp/go-build892594869/b101/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/data=/tmp/go-build892594869/b095/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/errcode=/tmp/go-build892594869/b096/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/internal/debug=/tmp/go-build892594869/b102/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/internal/parser=/tmp/go-build892594869/b105/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/internal/utils=/tmp/go-build892594869/b092/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/parser=/tmp/go-build892594869/b103/_pkg_.a
packagefile os=/tmp/go-build892594869/b003/_pkg_.a
packagefile os/exec=/tmp/go-build892594869/b079/_pkg_.a
packagefile path/filepath=/tmp/go-build892594869/b054/_pkg_.a
packagefile runtime/debug=/tmp/go-build892594869/b055/_pkg_.a
packagefile strings=/tmp/go-build892594869/b049/_pkg_.a
packagefile testing=/tmp/go-build892594869/b043/_pkg_.a
packagefile time=/tmp/go-build892594869/b030/_pkg_.a
packagefile context=/tmp/go-build892594869/b106/_pkg_.a
packagefile errors=/tmp/go-build892594869/b107/_pkg_.a
packagefile fmt=/tmp/go-build892594869/b131/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/ast=/tmp/go-build892594869/b149/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/cast=/tmp/go-build892594869/b167/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/config=/tmp/go-build892594869/b168/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/errcode=/tmp/go-build892594869/b163/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/format=/tmp/go-build892594869/b169/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/internal/debug=/tmp/go-build892594869/b170/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/internal/utils=/tmp/go-build892594869/b152/_pkg_.a
packagefile gitee.com/u-language/u-language/pkg/parser=/tmp/go-build892594869/b171/_pkg_.a
packagefile github.com/spf13/cobra=/tmp/go-build892594869/b174/_pkg_.a
packagefile io/fs=/tmp/go-build892594869/b147/_pkg_.a
packagefile os=/tmp/go-build892594869/b141/_pkg_.a
packagefile os/exec=/tmp/go-build892594869/b158/_pkg_.a
packagefile path/filepath=/tmp/go-build892594869/b161/_pkg_.a
packagefile runtime/debug=/tmp/go-build892594869/b164/_pkg_.a
packagefile strings=/tmp/go-build892594869/b155/_pkg_.a
packagefile time=/tmp/go-build892594869/b127/_pkg_.a
packagefile runtime=/tmp/go-build892594869/b113/_pkg_.a

This contains BOTH b003 and b141, along with many other duplicates, which can't be right...

If I do go list -test -json . ./ast, for gitee.com/u-language/u-language/pkg it has

        "Imports": [
                "context [gitee.com/u-language/u-language/pkg]",
                "errors [gitee.com/u-language/u-language/pkg]",
                "fmt [gitee.com/u-language/u-language/pkg]",
                "gitee.com/u-language/u-language/pkg/ast [gitee.com/u-language/u-language/pkg]",
                "gitee.com/u-language/u-language/pkg/cast [gitee.com/u-language/u-language/pkg]",
                "gitee.com/u-language/u-language/pkg/config [gitee.com/u-language/u-language/pkg]",
                "gitee.com/u-language/u-language/pkg/errcode [gitee.com/u-language/u-language/pkg]",
                "gitee.com/u-language/u-language/pkg/format [gitee.com/u-language/u-language/pkg]",
                "gitee.com/u-language/u-language/pkg/internal/debug [gitee.com/u-language/u-language/pkg]",
                "gitee.com/u-language/u-language/pkg/internal/utils [gitee.com/u-language/u-language/pkg]",
                "gitee.com/u-language/u-language/pkg/parser [gitee.com/u-language/u-language/pkg]",
                "github.com/spf13/cobra [gitee.com/u-language/u-language/pkg]",
                "io/fs [gitee.com/u-language/u-language/pkg]",
                "os [gitee.com/u-language/u-language/pkg]",
                "os/exec [gitee.com/u-language/u-language/pkg]",
                "path/filepath [gitee.com/u-language/u-language/pkg]",
                "runtime/debug [gitee.com/u-language/u-language/pkg]",
                "strings [gitee.com/u-language/u-language/pkg]",
                "time [gitee.com/u-language/u-language/pkg]"
        ],
        ...
        "TestImports": [
                "bufio",
                "context",
                "fmt",
                "gitee.com/u-language/u-language/pkg/ast",
                "gitee.com/u-language/u-language/pkg/cast",
                "gitee.com/u-language/u-language/pkg/check",
                "gitee.com/u-language/u-language/pkg/config",
                "gitee.com/u-language/u-language/pkg/data",
                "gitee.com/u-language/u-language/pkg/errcode",
                "gitee.com/u-language/u-language/pkg/internal/debug",
                "gitee.com/u-language/u-language/pkg/internal/parser",
                "gitee.com/u-language/u-language/pkg/internal/utils",
                "gitee.com/u-language/u-language/pkg/parser",
                "os",
                "os/exec",
                "path/filepath",
                "runtime/debug",
                "strings",
                "testing",
                "time",
                "unsafe"
        ]

The TestImports contains the non-PGO version of the packages. These probably should be fixed. I'll look into it.

@cherrymui cherrymui self-assigned this Jul 17, 2023
@cherrymui cherrymui added this to the Go1.21 milestone Jul 17, 2023
@gopherbot
Copy link

Change https://go.dev/cl/510835 mentions this issue: cmd/go: attach PGO profile for test dependencies

@gopherbot
Copy link

Change https://go.dev/cl/511698 mentions this issue: [release-branch.go1.21 cmd/go: attach PGO profile for test dependencies

gopherbot pushed a commit that referenced this issue Jul 20, 2023
@cherrymui
[release-branch.go1.21 cmd/go: attach PGO profile for test dependencies
c9f01f0 
When running "go test" including a main package which has a PGO
profile, we currently build the package being tested and its
dependencies with PGO, but we failed to attach the profile to
test-only dependencies. If a package is (transitively) imported
by both the package being tested and the test, the PGO version
and the non-PGO version of the package are both linked into the
binary, causing link-time error.

This CL fixes this by attaching the PGO profile to dependencies of
the test.

Fixes #61376.

Change-Id: I2559db9843c4cdab596b31e2025d8475ffbf58ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/510835
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit af0609b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/511698
bradfitz pushed a commit to tailscale/go that referenced this issue Aug 2, 2023
@cherrymui @bradfitz
[release-branch.go1.21 cmd/go: attach PGO profile for test dependencies
01af70f 
When running "go test" including a main package which has a PGO
profile, we currently build the package being tested and its
dependencies with PGO, but we failed to attach the profile to
test-only dependencies. If a package is (transitively) imported
by both the package being tested and the test, the PGO version
and the non-PGO version of the package are both linked into the
binary, causing link-time error.

This CL fixes this by attaching the PGO profile to dependencies of
the test.

Fixes golang#61376.

Change-Id: I2559db9843c4cdab596b31e2025d8475ffbf58ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/510835
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit af0609b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/511698
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cherrymui cherrymui

Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Projects
None yet
Milestone
Go1.21
Development

No branches or pull requests
6 participants
@prattmic @gopherbot @mauri870 @seankhliao @cherrymui @qiulaidongfeng