Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime: TestCgoPprofThread failures with invalid traceback origin #52726

Closed
bcmills opened this issue May 5, 2022 · 3 comments
Closed

runtime: TestCgoPprofThread failures with invalid traceback origin #52726

bcmills opened this issue May 5, 2022 · 3 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Linux release-blocker
Milestone
Go1.19

Comments

@bcmills
Copy link
Contributor

bcmills commented May 5, 2022 •
edited 

--- FAIL: TestCgoPprofThread (0.86s)
    crash_cgo_test.go:345: [/tmp/workdir-host-linux-amd64-wsl/go/bin/go tool pprof -traces /tmp/workdir-host-linux-amd64-wsl/tmp/go-build1132426278/testprogcgo_.exe /tmp/workdir-host-linux-amd64-wsl/tmp/prof944139058]:
        File: testprogcgo_.exe
        Build ID: 261d318cd4ab866e2474b8a08356fc426efe8d5a
        Type: cpu
        Time: May 5, 2022 at 8:35am (CST)
        Duration: 301.74ms, Total samples = 110ms (36.45%)
        -----------+-------------------------------------------------------
              10ms   cpuHogThread
                     cpuHogThread2
                     runtime.cgocall
                     main._Cfunc_runCPUHogThread
                     main.pprofThread
                     main.CgoPprofThread
                     main.main
                     runtime.main
        -----------+-------------------------------------------------------
             100ms   cpuHogThread
                     cpuHogThread2
        -----------+-------------------------------------------------------
    crash_cgo_test.go:357: invalid traceback origin: got=[cpuHogThread cpuHogThread2 runtime.cgocall main._Cfunc_runCPUHogThread main.pprofThread main.CgoPprofThread main.main runtime.main]; want=[cpuHogThread ... cpuHogThread2]
    crash_cgo_test.go:345: [/tmp/workdir-host-linux-amd64-wsl/go/bin/go tool pprof -traces /tmp/workdir-host-linux-amd64-wsl/tmp/prof944139058]:
        File: testprogcgo_.exe
        Build ID: 261d318cd4ab866e2474b8a08356fc426efe8d5a
        Type: cpu
        Time: May 5, 2022 at 8:35am (CST)
        Duration: 301.74ms, Total samples = 110ms (36.45%)
        -----------+-------------------------------------------------------
              10ms   cpuHogThread
                     cpuHogThread2
                     runtime.cgocall
                     main._Cfunc_runCPUHogThread
                     main.pprofThread
                     main.CgoPprofThread
                     main.main
                     runtime.main
        -----------+-------------------------------------------------------
             100ms   cpuHogThread
                     cpuHogThread2
        -----------+-------------------------------------------------------
    crash_cgo_test.go:357: invalid traceback origin: got=[cpuHogThread cpuHogThread2 runtime.cgocall main._Cfunc_runCPUHogThread main.pprofThread main.CgoPprofThread main.main runtime.main]; want=[cpuHogThread ... cpuHogThread2]
FAIL
FAIL	runtime	26.366s

greplogs -l -e 'FAIL: TestCgoPprofThread .*(?:\n[ ]{4}.*)* invalid traceback origin'
2022-05-04T15:38:55-9511f6d/linux-amd64-wsl
2021-12-09T12:28:48-65633fe/linux-amd64-wsl
2021-11-29T00:57:09-78af02e/linux-amd64-longtest
2021-11-16T17:13:33-29ec902/linux-amd64-jessie
2021-11-11T11:00:33-a01a6d6/linux-amd64
2021-10-28T22:38:00-af05d8b/linux-amd64-fedora
2021-10-11T15:28:50-702e337/linux-amd64-wsl
2021-10-09T01:04:29-e1c294a/linux-amd64-fedora
2021-10-06T19:45:35-e38ec96/linux-amd64-longtest
2021-10-05T19:15:03-ae83301/linux-amd64-staticlockranking
2021-09-15T03:29:39-9fc2889/linux-amd64-wsl
2021-09-13T18:02:42-e74e363/linux-amd64-wsl
2021-09-04T19:33:54-28dae3d/linux-amd64-wsl
2021-09-02T14:02:42-b8420ba/linux-amd64-wsl
2021-09-02T01:06:17-df4c625/linux-amd64-wsl
2021-09-01T23:41:53-2872496/linux-amd64-wsl
2021-08-16T14:27:24-b9f135d/linux-amd64-wsl
2021-02-24T16:38:32-ae1fa08/linux-amd64-wsl
2020-12-11T17:26:14-1341a3d/linux-amd64-wsl

(CC @golang/runtime)
@bcmills
Copy link
Contributor Author

bcmills commented May 5, 2022 •
edited

Marking as release-blocker for Go 1.19 because linux/amd64 is a first class port (and several of these failures are on regular builders rather than the -wsl builder).

This issue at least needs to be triaged for Go 1.19; if the runtime team decides that it isn't a priority, this test (or this failure mode) should call testenv.SkipFlaky to reduce noise from failures.

@bcmills bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker OS-Linux labels May 5, 2022
@bcmills bcmills added this to the Go1.19 milestone May 5, 2022
@cherrymui
Copy link
Member

The cgo traceback function ( https://cs.opensource.google/go/go/+/master:src/runtime/testdata/testprogcgo/threadpprof.go;l=45 ) pretends all C samples are in cpuHogThread. But if a profiling signal lands outside of the C thread, e.g. before/when the thread is created, we will get a sample like above, which looks like Go calls into that C thread. Perhaps the cgo traceback function should only return those PCs when a signal lands on that thread.

@gopherbot
Copy link

Change https://go.dev/cl/406934 mentions this issue: runtime: deflake TestCgoPprofThread

@golang golang locked and limited conversation to collaborators May 17, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Linux release-blocker
Projects
None yet
Milestone
Go1.19
Development

No branches or pull requests
3 participants
@bcmills @gopherbot @cherrymui