Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: bootstrapping gc with gccgo skips standard library #10258

Closed
h4ck3rm1k3 opened this issue Mar 26, 2015 · 23 comments
Closed

cmd/go: bootstrapping gc with gccgo skips standard library #10258

h4ck3rm1k3 opened this issue Mar 26, 2015 · 23 comments

Comments

@h4ck3rm1k3
Copy link

With gcc go, bootstrapping the compiler itself

./go build -v -x -compiler gccgo -work -n -a fmt/

The sources are skipped on this line, In order to get it to compile with gccgo, need to comment out the line that skips the compilation.

https://github.com/golang/go/blob/master/src/cmd/go/build.go#L623

Originally written in:
https://codereview.appspot.com/5580051
d717208

touched here as well:
6d888f1

h4ck3rm1k3 referenced this issue Mar 26, 2015
The previous logic was mainly non-working. It only needs to
ensure that the go tool doesn't try to build the standard
library with gccgo.

R=golang-dev, rsc
CC=golang-dev, remy
https://golang.org/cl/5580051
@mikioh mikioh changed the title GCCGO bootstrap build action skips stdlib/ was fix handling of gccgo standard library gccgo: bootstrap build action skips stdlib/ was fix handling of gccgo standard library Mar 26, 2015
@ianlancetaylor
Copy link
Contributor

What do you suggest?

@ianlancetaylor ianlancetaylor changed the title gccgo: bootstrap build action skips stdlib/ was fix handling of gccgo standard library cmd/go: bootstrapping gc with gccgo skips standard library Mar 26, 2015
@jmdcal
Copy link

jmdcal commented Mar 26, 2015

It looks like a bug to me. I dont even know if it works on other code either.

@ianlancetaylor
Copy link
Contributor

This code is not a bug for ordinary usage. People using gccgo do not rebuild the standard library, because it is built as part of building gccgo itself. There is a special case here: using gccgo to bootstrap the gc compiler. Without looking into the details, I imagine that something about the setting of GOROOT is causing the go tool to decide that it is looking at the standard library sources, and deciding that since it is using gccgo, it does not need to build them.

So somebody needs to investigate this issue, figure out what the real problem is (whether it is my speculation or something else), and figure out what the right fix is.

@davecheney
Copy link
Contributor

We had to work around this for gccgo-4.9 shipped in Ubuntu. We had to
provide a modified version of the go tool which knew the list of packages
which made up gccgo's version of $GOROOT and always reported that they were
up to date in the dependency graph,

On Fri, Mar 27, 2015 at 10:03 AM, Ian Lance Taylor <notifications@github.com

wrote:

This code is not a bug for ordinary usage. People using gccgo do not
rebuild the standard library, because it is built as part of building gccgo
itself. There is a special case here: using gccgo to bootstrap the gc
compiler. Without looking into the details, I imagine that something about
the setting of GOROOT is causing the go tool to decide that it is looking
at the standard library sources, and deciding that since it is using gccgo,
it does not need to build them.

So somebody needs to investigate this issue, figure out what the real
problem is (whether it is my speculation or something else), and figure out
what the right fix is.


Reply to this email directly or view it on GitHub
#10258 (comment).

@h4ck3rm1k3
Copy link
Author

Ok, I have tested this on normal code. It tries to build only the userland code. If you comment out those lines it will try and build the core lib again as well. I will see what happens if I just copy the code someplace else.

@h4ck3rm1k3
Copy link
Author

if I copy the code to a different directory structure it will build fine.

export GOPATH=`pwd`
cd src/h4ck3rm1k3/gocore/fmt
cp ~/go/src/fmt/* .

So I guess this is an OK workaround.

@davecheney
Copy link
Contributor

I write about this about a year ago,
http://go-talks.appspot.com/github.com/davecheney/gosyd/gccgo.slide#8. I
had a look in the issue tracker but cannot find anything relevant. Perhaps
I never logged a bug as we fixed the issue in our downstream fork

On Fri, Mar 27, 2015 at 10:13 AM, James Michael DuPont <
notifications@github.com> wrote:

Closed #10258 #10258.


Reply to this email directly or view it on GitHub
#10258 (comment).

@h4ck3rm1k3 h4ck3rm1k3 reopened this Mar 26, 2015
@h4ck3rm1k3
Copy link
Author

I have thought about this. First of all. we need the go sources to compile go? The idea that occurs to me is that we need a way to sign/signify that a source file has been compiled and the object file is OK. I don't know how this all works but maybe some form of tracking could be used?

@davecheney
Copy link
Contributor

I have thought about this. First of all. we need the go sources to
compile go? The idea that occurs to me is that we need a way to
sign/signify that a source file has been compiled and the object file is
OK. I don't know how this all works but maybe some form of tracking could
be used?

I'm not following you. What does this have to do with using gccgo to build
the bootstrap compilers and the dist tool ?

On Fri, Mar 27, 2015 at 10:33 AM, James Michael DuPont <
notifications@github.com> wrote:

I have thought about this. First of all. we need the go sources to compile
go? The idea that occurs to me is that we need a way to sign/signify that a
source file has been compiled and the object file is OK. I don't know how
this all works but maybe some form of tracking could be used?


Reply to this email directly or view it on GitHub
#10258 (comment).

@h4ck3rm1k3
Copy link
Author

Well the idea is that you would create some manifest file that describes the compilation that you consider valid, what inputs files and object files are good and what compiler what used. Some signature hash of those things would be stored. That manifest file would be shipped with the compiler and it would say that the runtime lib has been compiled with a certain compiler.

If you use a different compiler then that manifest would be invalid because it includes what compiler is used and the files would be recompiled.

You could tell the compiler to ignore that file like make -b and it would recompile the objects, otherwise it would reuse the object files if it noticed that compiler to the source changed.

@davecheney
Copy link
Contributor

I think that is unnecessary, go programs already include all their dependencies expressed as import statements.

On 27 Mar 2015, at 12:38, James Michael DuPont notifications@github.com wrote:

Well the idea is that you would create some manifest file that describes the compilation that you consider valid, what inputs files and object files are good and what compiler what used. Some signature hash of those things would be stored. That manifest file would be shipped with the compiler and it would say that the runtime lib has been compiled with a certain compiler.

If you use a different compiler then that manifest would be invalid because it includes what compiler is used and the files would be recompiled.

You could tell the compiler to ignore that file like make -b and it would recompile the objects, otherwise it would reuse the object files if it noticed that compiler to the source changed.


Reply to this email directly or view it on GitHub.

@h4ck3rm1k3
Copy link
Author

The dependencies are expressed, but we are talking about what object files are to be reused.

@davecheney
Copy link
Contributor

I'm very sorry but I'm just not following. For gccgo $GOROOT is entirely captured by libgo.so. All that I think is necessary is to make a small adjustment to the go tool to understand that things which lib in the standard library are always up to date

On 27 Mar 2015, at 13:06, James Michael DuPont notifications@github.com wrote:

The dependencies are expressed, but we are talking about what object files are to be reused.


Reply to this email directly or view it on GitHub.

@h4ck3rm1k3
Copy link
Author

Up to date is the question of what compiler was used and what version of the source was compiled, if any of those change you need to recompile.
If libgo.so is the results of compiling the runtime dir, we would save that in a file with the compiler used. If you use a new compiler that would invalidate that and cause the recompilation.

@davecheney
Copy link
Contributor

libgo is for the version of gccgo that is being used to compile the
bootstrap go 1.5 compiler.

Maybe we're talking at cross purposes.

On Fri, Mar 27, 2015 at 1:57 PM, James Michael DuPont <
notifications@github.com> wrote:

Up to date is the question of what compiler was used and what version of
the source was compiled, if any of those change you need to recompile.
If libgo.so is the results of compiling the runtime dir, we would save
that in a file with the compiler used. If you use a new compiler that would
invalidate that and cause the recompilation.


Reply to this email directly or view it on GitHub
#10258 (comment).

@h4ck3rm1k3
Copy link
Author

I have built a new version of the gccgo compiler and would like to compile the go/src again from scratch with that new compiler. It should not skip any of the files.

@davecheney
Copy link
Contributor

In that case you are using gccgo to build a different version of gc go. I'm
pretty certain that gccgo, via the go tool, is not capable of rebuilding its
own libgo.a file.

In any case I believe this is tangental to this issue. The goal is to be
able to use gccgo to build cmd/dist and the first stage bootstrap gc
compilers. After that, the rest is done by the gc toolchain itself.

On Fri, Mar 27, 2015 at 2:02 PM, James Michael DuPont <
notifications@github.com> wrote:

I have built a new version of the gccgo compiler and would like to compile
the go/src again from scratch with that new compiler. It should not skip
any of the files.


Reply to this email directly or view it on GitHub
#10258 (comment).

@h4ck3rm1k3
Copy link
Author

Actually, my goal is to continue to build with gccgo not gc.

@minux
Copy link
Member

minux commented Mar 27, 2015 via email

@davecheney
Copy link
Contributor

Indeed. If you want to work with gccgo, then use it via its autotools build
chain. You can't use cmd/go to circumvent this.

On Fri, 27 Mar 2015 14:33 Minux Ma notifications@github.com wrote:

You can't continue to build with gccgo, because starting
from the runtime, the code is only meant for the gc toolchain,
not gccgo.


Reply to this email directly or view it on GitHub
#10258 (comment).

@h4ck3rm1k3
Copy link
Author

Ok thanks for the feedback. I am going to have to look into this in more detail before responding.

@davecheney
Copy link
Contributor

Can I ask you to take this idea to a new issue.

The resolution of this issue will be when ./all.bash can be completed with
gccgo as the bootstrap compiler.

On Fri, Mar 27, 2015 at 12:38 PM, James Michael DuPont <
notifications@github.com> wrote:

Well the idea is that you would create some manifest file that describes
the compilation that you consider valid, what inputs files and object files
are good and what compiler what used. Some signature hash of those things
would be stored. That manifest file would be shipped with the compiler and
it would say that the runtime lib has been compiled with a certain compiler.

If you use a different compiler then that manifest would be invalid
because it includes what compiler is used and the files would be recompiled.

You could tell the compiler to ignore that file like make -b and it would
recompile the objects, otherwise it would reuse the object files if it
noticed that compiler to the source changed.


Reply to this email directly or view it on GitHub
#10258 (comment).

@h4ck3rm1k3
Copy link
Author

see #10092. this ticket was specific to this line I need to comment out, which is not needed. I have a workaround to copy the code to a different location. Will close for now.

@golang golang locked and limited conversation to collaborators Jun 25, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

6 participants