Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: net/http/pprof: support finer-grained delta computation #57765

Open
nsrip-dd opened this issue Jan 12, 2023 · 4 comments
Open

proposal: net/http/pprof: support finer-grained delta computation #57765

nsrip-dd opened this issue Jan 12, 2023 · 4 comments
Labels
FeatureRequest NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Proposal
Milestone

Comments

@nsrip-dd
Copy link
Contributor

nsrip-dd commented Jan 12, 2023

The /debug/pprof endpoint supports a "seconds" parameter. When a non-CPU profile is requested with that parameter, the endpoint returns a profile representing the difference ("delta") between two of those profiles taken "seconds" apart. The current delta implementation computes that difference for every value in a sample. In particular, the heap profile will contain differences for alloc_space and alloc_objects, as well as the inuse_space and inuse_objects counts.

On the one hand, I can see how seeing the change in inuse_space or inuse_objects could aid memory leak debugging. On the other hand, there's another use case for getting the delta profiles: aggregation. The Datadog and Google Cloud Profiler client libraries compute deltas, but only for the alloc_space and alloc_objects values in a heap profile. For the inuse values, the most recent value is used rather than the difference. Those values are preferable for computing averages across several profiles.

Both of those libraries use runtime/pprof and manipulate the profiles locally to get the desired result. But for profiling services by pulling from the /debug/pprof endpoints, you'd need to do something like pull two profiles and compute the delta later. Perhaps there can be an option to request delta profiles where some of the sample values have deltas computed, but others don't? Admittedly, this would primarily be an optimization to avoid having to request multiple profiles. See this example for how it might be implemented, by adding another query parameter delta_value to specifies which values in the profile to compute deltas for.

@seankhliao seankhliao added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. FeatureRequest labels Jan 12, 2023
@seankhliao
Copy link
Member

just to be clear, what would actually change (or be added) to support this

cc @cherrymui @rsc

@nsrip-dd
Copy link
Contributor Author

nsrip-dd commented Jan 12, 2023

@seankhliao I've updated the issue to clarify my suggested change, adding a query parameter to control which values in a sample have deltas computed when the "seconds" parameter is also provided.

@seankhliao seankhliao changed the title net/http/pprof: support finer-grained delta computation proposal: net/http/pprof: support finer-grained delta computation Jan 20, 2023
@gopherbot gopherbot added this to the Proposal milestone Jan 20, 2023
@ianlancetaylor
Copy link
Contributor

CC @neild @bradfitz

@gkampitakis
Copy link

I got also confused by this detail when looking at inuse_space and was noticing negative values on my profile, not realising this was a delta profile.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
FeatureRequest NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Proposal
Projects
Status: Incoming
Development

No branches or pull requests

5 participants