Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: Allow profiling channels usage/utilization via pprof #27347

Closed
ingojaeckel opened this issue Aug 29, 2018 · 3 comments
Closed

proposal: Allow profiling channels usage/utilization via pprof #27347

ingojaeckel opened this issue Aug 29, 2018 · 3 comments
Labels
FrozenDueToAge Proposal WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone

Comments

@ingojaeckel
Copy link

(this is a continuation of a thread that started on go-nuts)

Goal

I am proposing to extend pprof.Profile with information about active channels.

This is inspired by pprof.SetGoroutineLabels and other information already available in pprof about go routines (list of go routines, stack trace for each go routine). Making this data available via pprof would simplify the development of publisher/subscriber applications. Providing information about the the count, memory use, and utilization (fill rate) of all existing channels could speed up development. Especially IO-heavy operations that use channels for async IO would benefit from embedding this kind of information into pprof.

Examples for additional data

For each channel c of type chan T, being able to gather the following data via pprof may be useful:

(Immutable)

  • type T
  • Information about where the channel was created (via stacktrace/package name/etc)
  • Capacity of the channel, type of channel
  • Creation time (aka age of the channel)

(Mutable)

  • List of all active channels
  • Length according to len(c)
  • Bytes allocated due to the channel itself (metadata) and it's contents
  • Enqueue/Dequeue rate
  • Number of publishers/subscribers and/or which package/func is reading from/writing to c

I realize that some of those are more generic than others and that the implementation & runtime cost will vary between them.

@gopherbot gopherbot added this to the Proposal milestone Aug 29, 2018
@ianlancetaylor
Copy link
Contributor

CC @hyangah

@rsc
Copy link
Contributor

rsc commented Sep 19, 2018

This level of detail is not something pprof is really designed for. It is designed for aggregating a single number across call stacks (like cpu time spent, bytes allocated, etc).
Anything would have to be sampled to avoid runtime slowdowns.

Bytes allocated due to the channel should already show up as allocations.

You could imagine trying to record bytes transferred over a channel as an interesting thing to count. I'm not sure what I'd do with that number though, so I can't say why it would be worth the cost to collect.

Have you had any success with "manual" collection of data like this for understanding programs or debugging them?

@rsc rsc added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Oct 3, 2018
@rsc
Copy link
Contributor

rsc commented Oct 10, 2018

It sounds like basic memory allocation profile will already capture the things that can be captured in the pprof profile format. The others don't really fit, and we don't have evidence that they're useful for real-world analysis.

Closing.

@rsc rsc closed this as completed Oct 10, 2018
@golang golang locked and limited conversation to collaborators Oct 10, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge Proposal WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

4 participants