Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: runtime/trace: function entry/exit profiling #17988

Closed
FiloSottile opened this issue Nov 19, 2016 · 7 comments
Closed

proposal: runtime/trace: function entry/exit profiling #17988

FiloSottile opened this issue Nov 19, 2016 · 7 comments
Labels
FrozenDueToAge NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. Proposal
Milestone

Comments

@FiloSottile
Copy link
Contributor

This might be patently stupid, but I can't see why right now.

When both CPU profiling and tracing is enabled, could the SIGPROF handler write the pc and the goroutine id to the trace?

That way we would get an approximate idea of what the goroutine is doing during the scheduled time in the trace. Which would get us a bundled performance overview with just the trace.

The big, big difference from profiles however is that we would be able to generate histograms of how much CPU a function takes over the individual goroutines, as opposed to a total average.

(Maybe it's stupid because samples are not granular enough for this to be useful?)

@quentinmit quentinmit added NeedsFix The path to resolution is known, but the work has not been done. NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. and removed NeedsFix The path to resolution is known, but the work has not been done. labels Nov 21, 2016
@quentinmit quentinmit added this to the Go1.9Maybe milestone Nov 21, 2016
@quentinmit
Copy link
Contributor

I've wanted this as well.

/cc @aclements

@titpetric
Copy link

I've just tried discovering some performance data with tracing, also noting that go tool profile has a -pprof option, which however doesn't give out some kind of cpuprofile like output that would give us a bundled performance overview, or data suitable for a flame graph or something like the call graph options from pprof. This would be a great thing to have. Maybe I'm just not connecting the dots here, if it's possible to get this overview somehow differently?

@aclements
Copy link
Member

I assume you mean go tool trace, not go tool profile. I'm not sure I quite understand what you're asking, but the execution trace currently doesn't have enough information in it to produce any sort of call graph profile. We could add profiling samples to the trace, which could then be extracted as a profile, though that wouldn't be any different from simply taking a pprof CPU profile (unless you filtered it or something first, as suggested by @FiloSottile).

@titpetric
Copy link

titpetric commented Jun 21, 2017 via email

@aclements
Copy link
Member

I think there's some difference with pprof as the trace is complete and pprof is sampled, no?

While that's true, the differences are far more drastic. The execution trace records specific runtime events, like goroutine scheduling, network events, and GC.

Basically I'd want a full pprof profile (without sampling)

It sounds like what you want is a function call trace, which is not what runtime/trace is (or anything else built in to Go). Function call traces are very expensive to collect, which makes them basically useless for performance optimization. There has been recent work on this topic in general (e.g., https://research.google.com/pubs/pub45287.html), but even that has 20–40% overhead when enabled.

@aclements aclements modified the milestones: Go1.10, Go1.9Maybe Jul 20, 2017
@aclements aclements changed the title runtime/trace: record CPU profiling samples proposal: runtime/trace: function entry/exit profiling Nov 9, 2017
@aclements aclements modified the milestones: Go1.10, Proposal Nov 9, 2017
@aclements
Copy link
Member

I've re-titled this to be what I understand the request to be, which is not actually CPU profiling, but function entry/exit tracing. That's a reasonable feature request, and one could imagine implementing something like XRay for Go, though I don't think anyone is going to work on this unless there's significantly more demand for it. It may be that in the long run something like Intel Processor Trace (PT) is a better answer.

The other issue of adding CPU profiling samples to the execution trace is #16895.

@rsc
Copy link
Contributor

rsc commented Nov 13, 2017

Given that doing this efficiently is basically an active research topic, we're not going to lead here. The "profiles during SIGPROF" is #16895, which is still open, but I'm going to close this issue.

(There's also #16619 for user-defined events, which you might use (if we had it) in this situation.)

@rsc rsc closed this as completed Nov 13, 2017
@golang golang locked and limited conversation to collaborators Nov 13, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. Proposal
Projects
None yet
Development

No branches or pull requests

7 participants