Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime/trace: expose API to parse traces #62461

Closed
cdvr1993 opened this issue Sep 5, 2023 · 4 comments
Closed

runtime/trace: expose API to parse traces #62461

cdvr1993 opened this issue Sep 5, 2023 · 4 comments
Assignees
@mknyszek
Labels
compiler/runtime Issues related to the Go compiler and/or runtime.
Milestone
Backlog

Comments

@cdvr1993
Copy link

cdvr1993 commented Sep 5, 2023

Go pprof allows a way to see profiles in a text based format. This is very useful when you have hundreds of profiles to analyze and you want to find the specific instance that has the most code in a specific function.

On the other hand, trace only allows to do UI analysis. Then if I have hundreds of traces I need to go one by one and traces can take several seconds if not minutes to load. To put a "real" example, I recently had to find why we were having long GC pauses, but these GC pauses were only happening in 1-3 instances once in a while (3-5 minutes). I was able to find the pattern to run a complete horizontal tracing of it, but after that I needed to analyze ~64 90 seconds traces.

What I ended up doing was to copy the source code from internal/trace and manually create a function that only returned the data that I wanted. Once I got the instance with the highest GC pause I fallback to the normal way of loading a trace in the UI and I was able to find the source of the problem.

The proposal/ask is if the community has considered exposing some of the APIs under internal/trace? Or provide a text based command line approach for the trace command?
@gopherbot gopherbot added this to the Proposal milestone Sep 5, 2023
@mknyszek mknyszek self-assigned this Sep 5, 2023
@rhysh
Copy link
Contributor

rhysh commented Sep 5, 2023

CC @mknyszek (and @golang/runtime ?)

The design doc in #60773 includes plans for a parsing API, sometime in the future. You might also be interested in following the discussion on #57175.

Today, you might be able to use the parser in https://pkg.go.dev/honnef.co/go/gotraceui@v0.3.0/trace#Parse

Or, if the information you're trying to extract is something relatively easy to calculate like "the time difference between STW begin and end events", you might be able to use go tool trace -d to dump the text format of the execution trace for processing with a tool like awk.

@mknyszek
Copy link
Contributor

mknyszek commented Sep 5, 2023

FWIW, I'm already working on this as part of #57175. I plan to land a trace format reader in x/exp for the Go 1.22 release, which will get vendored or copied into std as an internal package. Just to set expectations, to begin with, it may only support Go 1.22+ traces depending on how much time I have. But long-term I expect it to support older traces as well (with worse decoding performance, since the old trace format requires decoding the entire trace up-front).

@mknyszek mknyszek changed the title proposal: Expose API to parse traces affected/package: runtime proposal: runtime/trace: expose API to parse traces Sep 5, 2023
@mknyszek mknyszek added the compiler/runtime Issues related to the Go compiler and/or runtime. label Sep 5, 2023
@ianlancetaylor
Copy link
Contributor

Taking this issue out of the proposal process.

@ianlancetaylor ianlancetaylor changed the title proposal: runtime/trace: expose API to parse traces runtime/trace: expose API to parse traces Sep 5, 2023
@mknyszek mknyszek mentioned this issue Sep 13, 2023
Open
@mknyszek
Copy link
Contributor

mknyszek commented Dec 21, 2023 •
edited

See #62627, which I think this issue is now a duplicate of. That issue has a link to an experimental public API for this, which is now available at golang.org/x/exp/trace. It only works for Go 1.22 traces, but it'll work for older traces soon, sometime early next year. If you're inclined, please give it a try! We're collecting feedback at this point.

@mknyszek mknyszek closed this as not planned Won't fix, can't repro, duplicate, stale Dec 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mknyszek mknyszek

Labels
compiler/runtime Issues related to the Go compiler and/or runtime.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
5 participants
@rhysh @mknyszek @ianlancetaylor @cdvr1993 @gopherbot