code review 6086043: cmd/cgo, cmd/go: detect dynamic linker automatically

cmd/cgo, cmd/cc, cmd/ld: detect dynamic linker automatically
Some newer Linux distributions (Ubuntu ARM at least) use a new multiarch
directory organization, where dynamic linker is no longer in the hardcoded
path in our linker.
For example, Ubuntu 12.04 ARM hardfloat places its dynamic linker at
/lib/arm-linux-gnueabihf/ld-linux.so.3

Ref: http://lackof.org/taggart/hacking/multiarch/

Also, to support Debian GNU/kFreeBSD as a FreeBSD variant, we need this capability, so it's part of issue 3533.

This CL add a new pragma (#pragma dynlinker "path") to cc.

[minux1, 2012-04-20 11:56:49]

Hello golang-dev@googlegroups.com (cc: golang-dev@googlegroups.com),

I'd like you to review this change to
https://code.google.com/p/go/

[iant, 2012-04-20 15:49:36]

LGTM but wait for rsc.

[rsc, 2012-04-20 16:46:27]

This seems okay but how do we get from here to 6l using the right
dynamic linker path? I do not want to have to invoke cgo during the
link phase of every program.

For example, go build cmd/godoc links a cgo-enabled program (because
godoc uses package net) but assuming package net is already compiled,
I do not want the link of godoc to have to invoke gcc+cgo to determine
the interp path.  What is the big picture here?

Russ

[minux1, 2012-04-20 16:52:40]

On Sat, Apr 21, 2012 at 12:46 AM, Russ Cox <rsc@golang.org> wrote:

> This seems okay but how do we get from here to 6l using the right
> dynamic linker path? I do not want to have to invoke cgo during the
> link phase of every program.
>
Oh, I overlooked this scenario.

>
> For example, go build cmd/godoc links a cgo-enabled program (because
> godoc uses package net) but assuming package net is already compiled,
> I do not want the link of godoc to have to invoke gcc+cgo to determine
> the interp path.  What is the big picture here?

It seems we have to do it the hard way, a new #pragma and change both
cc and ld.

[minux1, 2012-04-20 17:27:01]

On 2012/04/20 16:52:40, minux wrote:
> On Sat, Apr 21, 2012 at 12:46 AM, Russ Cox <mailto:rsc@golang.org> wrote:
> > For example, go build cmd/godoc links a cgo-enabled program (because
> > godoc uses package net) but assuming package net is already compiled,
> > I do not want the link of godoc to have to invoke gcc+cgo to determine
> > the interp path.  What is the big picture here?
> It seems we have to do it the hard way, a new #pragma and change both
> cc and ld.
Wait a minute, this change is too big for this small enhancement.
How abou this:
when linking programs like godoc who uses cgo *indirectly* we look for dynamic
linker used by the running cmd/go itself?

That is, we move the dynamic linker detection code from cmd/cgo to cmd/go,
and always pass '-I /path/to/dynld.so' to our ld if the program uses cgo?
(during bootstrapping, pkg/net and cmd/go are compiled in one run, and the
current approach will automagically set correct dynamic linker for cmd/go)

[minux1, 2012-04-20 17:34:40]

On Sat, Apr 21, 2012 at 1:27 AM, <minux.ma@gmail.com> wrote:
>
> when linking programs like godoc who uses cgo *indirectly* we look for
> dynamic linker used by the running cmd/go itself?
>
If we assume dynamic linker is fixed after cmd/go is bootstrapped, we can
simplify this further: we use 'ld -X' to embed the correct dynamic linker
into
cmd/go during bootstrap.

This assumption is fair, except when cross compiling. What do you think?

[rsc, 2012-04-25 03:16:40]

On Fri, Apr 20, 2012 at 13:27,  <minux.ma@gmail.com> wrote:
> That is, we move the dynamic linker detection code from cmd/cgo to
> cmd/go,
> and always pass '-I /path/to/dynld.so' to our ld if the program uses
> cgo?
> (during bootstrapping, pkg/net and cmd/go are compiled in one run, and
> the
> current approach will automagically set correct dynamic linker for
> cmd/go)

I strongly prefer not to do that.  Right now it is the case that as long
as you build package net's net.a on a Linux machine, you can copy
the package tree elsewhere and still cross-compile.  If you make the
link phase invoke gcc or inspect its own go binary, that's the end of
any hope of cross-compiling.

Adding a new #pragma here seems like the right choice.

[minux1, 2012-04-25 08:48:07]

On Wed, Apr 25, 2012 at 11:16 AM, Russ Cox <rsc@golang.org> wrote:

> On Fri, Apr 20, 2012 at 13:27,  <minux.ma@gmail.com> wrote:
> > That is, we move the dynamic linker detection code from cmd/cgo to
> > cmd/go,
> > and always pass '-I /path/to/dynld.so' to our ld if the program uses
> > cgo?
> > (during bootstrapping, pkg/net and cmd/go are compiled in one run, and
> > the
> > current approach will automagically set correct dynamic linker for
> > cmd/go)
>
> I strongly prefer not to do that.  Right now it is the case that as long
> as you build package net's net.a on a Linux machine, you can copy
> the package tree elsewhere and still cross-compile.  If you make the
> link phase invoke gcc or inspect its own go binary, that's the end of
> any hope of cross-compiling.
>
> Adding a new #pragma here seems like the right choice.
>
OK, this will be a big change.

BTW, do you have plan to support cross compiling with cgo?
(for example, 'go build -gcc "arm-none-gnueabi-gcc" cgo.go'; and if this
approach
is acceptable, I will submit a CL for it.)

[rsc, 2012-05-02 01:46:39]

On Wed, Apr 25, 2012 at 4:47 AM, minux <minux.ma@gmail.com> wrote:
> BTW, do you have plan to support cross compiling with cgo?
> (for example, 'go build -gcc "arm-none-gnueabi-gcc" cgo.go'; and if this
> approach
> is acceptable, I will submit a CL for it.)

I believe that you can use
    CC="arm-none-gnueabi-gcc" go build
already.

Russ

[minux1, 2012-05-03 09:07:06]

On Wed, May 2, 2012 at 9:46 AM, Russ Cox <rsc@golang.org> wrote:

> On Wed, Apr 25, 2012 at 4:47 AM, minux <minux.ma@gmail.com> wrote:
> > BTW, do you have plan to support cross compiling with cgo?
> > (for example, 'go build -gcc "arm-none-gnueabi-gcc" cgo.go'; and if this
> > approach
> > is acceptable, I will submit a CL for it.)
>
> I believe that you can use
>    CC="arm-none-gnueabi-gcc" go build
> already.

http://tip.golang.org/src/cmd/go/build.go#L1380

Do you want to add $CC to address that TODO?
(we don't need to constrain cgo to native builds after this change)

[minux1, 2012-05-03 10:57:18]

On 2012/04/25 03:16:40, rsc wrote:
> Adding a new #pragma here seems like the right choice.
Done. PTAL.

[rsc, 2012-05-03 21:30:38]

LGTM

Thanks!
The $$ note applies to all three compilers.

http://codereview.appspot.com/6086043/diff/14001/src/cmd/8c/swt.c
File src/cmd/8c/swt.c (right):

http://codereview.appspot.com/6086043/diff/14001/src/cmd/8c/swt.c#newcode267
src/cmd/8c/swt.c:267: Bprint(&outbuf, "\n$$  // dynlinker\n");
Please move this statement above the if, so that the format of the file does not
vary depending on whether this directive is present.  I realize that the linker
looks for the comments but I do intend them as just comments, so that the $$ are
the truly important part.

http://codereview.appspot.com/6086043/diff/14001/src/cmd/ld/go.c
File src/cmd/ld/go.c (right):

http://codereview.appspot.com/6086043/diff/14001/src/cmd/ld/go.c#newcode603
src/cmd/ld/go.c:603: interpreter = strdup(dynlinker);
free(interpreter);
interpreter = strdup(dynlinker);
?

[minux1, 2012-05-04 17:54:38]

*** Submitted as http://code.google.com/p/go/source/detail?r=d744d86bd5ea ***

cmd/cgo, cmd/cc, cmd/ld: detect dynamic linker automatically
Some newer Linux distributions (Ubuntu ARM at least) use a new multiarch
directory organization, where dynamic linker is no longer in the hardcoded
path in our linker.
For example, Ubuntu 12.04 ARM hardfloat places its dynamic linker at
/lib/arm-linux-gnueabihf/ld-linux.so.3

Ref: http://lackof.org/taggart/hacking/multiarch/

Also, to support Debian GNU/kFreeBSD as a FreeBSD variant, we need this
capability, so it's part of issue 3533.

This CL add a new pragma (#pragma dynlinker "path") to cc.

R=iant, rsc
CC=golang-dev
http://codereview.appspot.com/6086043