code review 5639059: runtime, pprof: add profiling of thread creation

runtime, pprof: add profiling of thread creation

Same idea as heap profile: how did each thread get created?
Low memory (256 bytes per OS thread), high reward for
programs that suddenly have many threads running.

Fixes issue 1477.

[rsc, 2012-02-08 06:40:35]

Hello golang-dev@googlegroups.com,

I'd like you to review this change to
https://code.google.com/p/go/

[r, 2012-02-08 09:10:53]

LGTM
nice
maybe in another CL couple it to the testing package.

[dvyukov, 2012-02-08 09:20:34]

nitpicks

http://codereview.appspot.com/5639059/diff/2010/src/pkg/runtime/debug.go
File src/pkg/runtime/debug.go (right):

http://codereview.appspot.com/5639059/diff/2010/src/pkg/runtime/debug.go#newc...
src/pkg/runtime/debug.go:107: Stack0 [32]uintptr // stack trace for this record;
ends at first 0 entry
does it need to be exported?

http://codereview.appspot.com/5639059/diff/2010/src/pkg/runtime/mprof.goc
File src/pkg/runtime/mprof.goc (right):

http://codereview.appspot.com/5639059/diff/2010/src/pkg/runtime/mprof.goc#new...
src/pkg/runtime/mprof.goc:285: M *first, *m;
s/m/mp/

http://codereview.appspot.com/5639059/diff/2010/src/pkg/runtime/pprof/pprof.go
File src/pkg/runtime/pprof/pprof.go (right):

http://codereview.appspot.com/5639059/diff/2010/src/pkg/runtime/pprof/pprof.g...
src/pkg/runtime/pprof/pprof.go:124: n, ok := runtime.ThreadProfile(nil)
s/ok/_/ ?

[dvyukov, 2012-02-08 09:35:34]

On 2012/02/08 06:40:35, rsc wrote:
> Hello mailto:golang-dev@googlegroups.com,
> 
> I'd like you to review this change to
> https://code.google.com/p/go/

I am trying to test, but I can't make it work.

First, go tool pprof behave strangely:

dvyukov$ go tool pprof http://localhost:6060/debug/pprof/heap
Read http://localhost:6060/debug/pprof/symbol
Fetching /pprof/heap profile from localhost:6060 to
  /home/dvyukov/pprof/rpc.test.1328693433.localhost-port6060.pprof.heap
Wrote profile to
/home/dvyukov/pprof/rpc.test.1328693433.localhost-port6060.pprof.heap
Adjusting heap profiles for 1-in-524288 sampling rate
Use of uninitialized value $_ in string at
/usr/local/google/home/dvyukov/go_select/bin/tool/pprof line 3078, <PROFILE>
line 70.
Welcome to pprof!  For help, type 'help'.
(pprof) dvyukov$ 

That is, it instantly exits and does not enters interactive mode.

Then, when I run

dvyukov$ .../bin/tool/pprof http://localhost:6060/debug/pprof/thread

It says:

FATAL ERROR: Did not specify profile file

What am I doing wrong?

[rsc, 2012-02-08 14:01:43]

On Wed, Feb 8, 2012 at 04:35,  <dvyukov@google.com> wrote:
> /usr/local/google/home/dvyukov/go_select/bin/tool/pprof line 3078,

You are running an old copy of pprof.
Line 3078 used to read
   my $post_data = join("+", sort((map {"0x" . "$_"} @thisround)));
and it did in fact exhibit the bug you are seeing.
But not after this CL.

Russ

[dvyukov, 2012-02-08 14:10:12]

On Wed, Feb 8, 2012 at 6:01 PM, Russ Cox <rsc@golang.org> wrote:

> On Wed, Feb 8, 2012 at 04:35,  <dvyukov@google.com> wrote:
> > /usr/local/google/home/dvyukov/go_select/bin/tool/pprof line 3078,
>
> You are running an old copy of pprof.
> Line 3078 used to read
>   my $post_data = join("+", sort((map {"0x" . "$_"} @thisround)));
> and it did in fact exhibit the bug you are seeing.
> But not after this CL.
>
>
I manually type 'make install' in src/cmd/pprof, and the warning gone. But
all other problems remain - go tool pprof http://localhost:6060/debug/**
pprof/heap <http://localhost:6060/debug/pprof/heap> instantly exists and
thread says the following:

dvyukov$ go tool pprof http://localhost:6060/debug/pprof/thread
Did not specify profile file

Usage:
[... lengthy pprof usage stripped]

FATAL ERROR: Did not specify profile file
go tool pprof: exit status 1
dvyukov$

[rsc, 2012-02-08 15:26:02]

On Wed, Feb 8, 2012 at 09:10, Dmitry Vyukov <dvyukov@google.com> wrote:
> I manually type 'make install' in src/cmd/pprof, and the warning gone. But
> all other problems remain - go tool
> pprof http://localhost:6060/debug/pprof/heap instantly exists and thread
> says the following:
>
> dvyukov$ go tool pprof http://localhost:6060/debug/pprof/thread
> Did not specify profile file

What if you run ./pprof from the cmd/prof directory?

[rsc, 2012-02-08 15:33:57]

*** Submitted as 3b53f76946b4 ***

runtime, pprof: add profiling of thread creation

Same idea as heap profile: how did each thread get created?
Low memory (256 bytes per OS thread), high reward for
programs that suddenly have many threads running.

Fixes issue 1477.

R=golang-dev, r, dvyukov
CC=golang-dev
http://codereview.appspot.com/5639059

[dvyukov, 2012-02-08 15:39:13]

On Wed, Feb 8, 2012 at 7:26 PM, Russ Cox <rsc@golang.org> wrote:

> On Wed, Feb 8, 2012 at 09:10, Dmitry Vyukov <dvyukov@google.com> wrote:
> > I manually type 'make install' in src/cmd/pprof, and the warning gone.
> But
> > all other problems remain - go tool
> > pprof http://localhost:6060/debug/pprof/heap instantly exists and thread
> > says the following:
> >
> > dvyukov$ go tool pprof http://localhost:6060/debug/pprof/thread
> > Did not specify profile file
>
> What if you run ./pprof from the cmd/prof directory?
>

Yes, it works. I think I see the problem:

dvyukov.../go/src/cmd/prof$ make install
go tool dist install -vv
cp .../go/*bin/tool/pprof* .../go/*bin/tool/pprof*
...

[rsc, 2012-02-08 15:40:50]

My copy of cmd/dist has other changes in it.  Can you verify
that this fixes the bug and send a CL?


	if(streq(dir, "cmd/prof")) {
		copy(bpathf(&b, "%s/bin/tool/pprof", goroot),
>>>			bpathf(&b, "%s/src/cmd/prof/pprof", goroot));
	}

change the &b on that line to &b1.

Russ

[dvyukov, 2012-02-08 15:57:51]

On Wed, Feb 8, 2012 at 7:40 PM, Russ Cox <rsc@golang.org> wrote:

> My copy of cmd/dist has other changes in it.  Can you verify
> that this fixes the bug and send a CL?
>
>
>        if(streq(dir, "cmd/prof")) {
>                copy(bpathf(&b, "%s/bin/tool/pprof", goroot),
> >>>                     bpathf(&b, "%s/src/cmd/prof/pprof", goroot));
>        }
>
> change the &b on that line to &b1.
>
>
Yes, it fixes. I've sent the CL.

'go tool pprof' still fails to start pprof in interactive mode. Should I
file a bug?

[rsc, 2012-02-08 16:31:47]

On Wed, Feb 8, 2012 at 10:57, Dmitry Vyukov <dvyukov@google.com> wrote:
> 'go tool pprof' still fails to start pprof in interactive mode. Should I
> file a bug?

Sure.  It's working for me though.

Russ