Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime: threadcreate profile is broken #6104

Open
dvyukov opened this issue Aug 11, 2013 · 14 comments
Open

runtime: threadcreate profile is broken #6104

dvyukov opened this issue Aug 11, 2013 · 14 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime.
Milestone
Unplanned

Comments

@dvyukov
Copy link
Member

dvyukov commented Aug 11, 2013 

With the new scheduler threadcreate profile is broken. Most of the threads are created
by sysmon thread, which stack is not very informative.
We need to either remove the profile, or capture stack of the blocking cgo/syscall that
caused new thread creation.
@rsc
Copy link
Contributor

rsc commented Aug 13, 2013

Comment 1:

It would be nice to capture a useful stack but it may be okay to remove it
too. I can't remember ever using it.

@dvyukov
Copy link
Member Author

dvyukov commented Aug 13, 2013

Comment 2:

What is a "useful stack"?
I've attached threadcreate profile from HTTP server FTR.
Lots of threads are created by sysmon (not useful).
Timerproc created 3 threads (not useful).
Creation of random goroutines caused thread creation up to GOMAXPROCS (not useful).
I can imagine an end user trying to figure out what threadcreate profile is trying to
say...
It must be centered around blocking syscalls and locked threads. But which exactly?
Probably all, because they all can trigger thread creation. It all needs to be rethought
and probably even renamed.
Probably now it's better to just remove it.

Attachments:

  1. threadcreate.txt (5407 bytes)

@rsc
Copy link
Contributor

rsc commented Aug 13, 2013

Comment 3:

The idea was that if, for example, your program was creating 1000s of
threads because they kept getting blocked on cgo DNS lookups, you'd see the
cgo DNS call in the profile sample, because it is what blocked and
triggered the creation of a new thread.

@robpike
Copy link
Contributor

robpike commented Aug 30, 2013

Comment 4:

Not for 1.2.

Labels changed: removed go1.2maybe.

@rsc
Copy link
Contributor

rsc commented Nov 27, 2013

Comment 5:

Labels changed: added go1.3maybe.

@rsc
Copy link
Contributor

rsc commented Nov 27, 2013

Comment 6:

Labels changed: removed feature.

@rsc
Copy link
Contributor

rsc commented Dec 4, 2013

Comment 7:

Labels changed: added release-none, removed go1.3maybe.

@rsc
Copy link
Contributor

rsc commented Dec 4, 2013

Comment 8:

Labels changed: added repo-main.

@rsc rsc added this to the Unplanned milestone Apr 10, 2015
@pradeepsawlani

This comment was marked as off-topic.

Sign in to view
@ianlancetaylor

This comment was marked as off-topic.

Sign in to view
@gopherbot
Copy link

Change https://golang.org/cl/189318 mentions this issue: runtime/pprof: Plumb labels for goroutine profiles

gopherbot pushed a commit that referenced this issue Apr 22, 2020
@dfinkel @hyangah
runtime/pprof: plumb labels for goroutine profiles
38c2c12 
Goroutines are directly associated with labels. It's relatively easy to
plumb those through without creating goroutine-locals in the wild.

This is accomplished by splitting out most of the code from the public
`runtime.GoroutineProfile` into a new unexported
`runtime.goroutineProfileWithLabels`, which then has a thin wrapper
linked into the `runtime/pprof` package as
`runtime_goroutineProfileWithLabels`. (mirroring the way labels get
associated with the `g` for a goroutine in the first place)

Per-#6104, OS-thread creation profiles are a bit useless, as `M`s tend
to be created be created by a background goroutine. As such, I decided
not to add support for capturing the labels at `M`-creation-time, since
the stack-traces seem to always come out `nil` for my simple test
binaries.

This change currently provides labels for debug=0 and debug=1, as
debug=2 is currently entirely generated by the runtime package and I
don't see a clean way of getting the `labelMap` type handled properly
within the `runtime` package.

Update the comment added in cl/131275 to mention goroutine support for
labels.

Updates #23458

Change-Id: Ia4b558893d7d10156b77121cd9b70c4ccd9e1889
Reviewed-on: https://go-review.googlesource.com/c/go/+/189318
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
@hyangah hyangah mentioned this issue Apr 27, 2020
Open
@brancz brancz mentioned this issue Feb 24, 2022
Closed
@arp242 arp242 mentioned this issue Jul 24, 2022
Closed
SaveTheRbtz added a commit to onflow/flow-go that referenced this issue Jul 29, 2022
@SaveTheRbtz
[Profile] disable threadcreate profiler
ef2fb8e 
It is broken since 2013[1]. Verified on localnet.

[1] golang/go#6104
@SaveTheRbtz SaveTheRbtz mentioned this issue Jul 29, 2022
Merged
bors bot added a commit to onflow/flow-go that referenced this issue Jul 29, 2022
@bors @SaveTheRbtz
Merge #2898
03634c1 
2898: [Profile] disable threadcreate profiler r=SaveTheRbtz a=SaveTheRbtz

[It has been broken since 2013](golang/go#6104). Verified on localnet and removed.

Co-authored-by: Alexey Ivanov <rbtz@dapperlabs.com>
@JaskaranSM JaskaranSM mentioned this issue Aug 2, 2022
Closed
@gopherbot
Copy link

Change https://go.dev/cl/423116 mentions this issue: runtime: always collect M creation stack

@prattmic
Copy link
Member

I've sent https://go.dev/cl/423116 as a "fix" for this. tl;dr, the CL just stops hiding stacks from within the scheduler. The result of this profile probably isn't very meaningful to the typical user, but it can be quite handy for someone investigating strange behavior in the runtime.

@prattmic prattmic added the compiler/runtime Issues related to the Go compiler and/or runtime. label Aug 16, 2022
@prattmic
Copy link
Member

e.g., here is an example profile from the package os tests:

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
compiler/runtime Issues related to the Go compiler and/or runtime.
Projects
Go Compiler / Runtime
Status: In Progress
Milestone
Unplanned
Development

No branches or pull requests
7 participants
@rsc @dvyukov @pradeepsawlani @prattmic @ianlancetaylor @robpike @gopherbot