Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: parse environment variables correctly on Windows #5042

Closed
gopherbot opened this issue Mar 13, 2013 · 16 comments
Closed

cmd/go: parse environment variables correctly on Windows #5042

gopherbot opened this issue Mar 13, 2013 · 16 comments
Milestone

Comments

@gopherbot
Copy link

by jgodfrey@thelostsite.co.uk:

This is a Windows specific issue. GOTOOLDIR is being incorrectly constructed.   

 Reproduction:   
1. Set GOROOT using the SET command (BAT file), enclosing it in quotes (E.g.
"C:\Program Files\go"). As in:
SET GOROOT="C:\Program Files\go"     
2. Leave GOTOOLDIR undefined   
3. Run "go env" or attempt to build anything which requires GOTOOLDIR 


What is the expected output?

"go env" should output: 
set GOTOOLDIR="C:\Program Files\go\pkg\tool\windows_386"

Or: 
set GOTOOLDIR=C:\Program Files\go\pkg\tool\windows_386     

With or without the quotes should be equivalent. 

What do you see instead?

set GOTOOLDIR="C:\Program Files\go"\pkg\tool\windows_386

It appears to take the literal variable, including quotation marks, and add
"\pkg\tool\windows_386" to the end. This both shows up incorrectly in go env
and also stops using from compiling since no packages will ever be found.    

Which operating system are you using?

Windows 7    

Which version are you using?  (run 'go version')

go version go1.0.3 

Please provide any additional information below.

This is arguably Windows' own fault for the way it handles SET values. However it is
fairly common to quote paths with spaces in them (e.g. "Program Files" etc) so
I can foresee this being a problem in the future if quotes aren't "considered."
@rsc
Copy link
Contributor

rsc commented Mar 13, 2013

Comment 1:

Labels changed: added priority-later, go1.1, removed priority-triage.

Status changed to Accepted.

@rsc
Copy link
Contributor

rsc commented Mar 13, 2013

Comment 2:

This is ridiculous (Windows's fault, not yours).
I guess we should call filepath.SplitPath to get the unquoted form.
Russ

@alexbrainman
Copy link
Member

Comment 3:

I am not convinced that there is an issue here. None of my environment variables have
quotes around them, some of them are paths with space in it.
Putting quotes around GOROOT environment variable is not needed - it achieves nothing. I
think it is a user mistake. I would keep things simple and do nothing.
Alternatively, if we want to be nice, and try and support this case, then, I think, the
change should be done in runtime.GOROOT, not in cmd/go. Just to be consistent. But, I am
concerned to introduce "unquoted" runtime.GOROOT that will not equal to
os.Getenv("GOROOT"). Also what are the rules of GOPATH "unquoting"? Should we follow
filepath.SplitPath? But it is not documented anywhere.
Just makes everything complicated.
Perhaps, we could just os.Stat(runtime.GOROOT) at cmd/go start, and fail with
appropriate message.
Alex

@gopherbot
Copy link
Author

Comment 4 by jgodfrey@thelostsite.co.uk:

If you have a path with a '&' character in it you're essentially required to put ""
around it or it won't get passed at all (in BAT). 
I actually found another related bug while researching this one. If you put whitespace
after the GOROOT path with or without quotes it will blindly put that into GOTOOLDIR
e.g. 
REM The below line has an extra space at the end   
set GOTOOLDIR=C:\go 
Then if we run "go env":   
set GOTOOLDIR=C:\go \pkg\tool\windows_386
So seems like the problem is "larger" than just quotation marks. We aren't even trimming
GOROOT.   
Now I would accept that ALL of these are the user's fault. But the way BAT is designed
it makes these kind of mistakes fairly easy to do and fairly hard to detect once they're
done.    
I mean even if we ignore the quote issue (edge case?) then it is hard to ignore the
whitespace issue.

@gopherbot
Copy link
Author

Comment 5 by jgodfrey@thelostsite.co.uk:

If you have a path with a '&' character in it you're essentially required to put ""
around it or it won't get passed at all (in BAT). 
I actually found another related bug while researching this one. If you put whitespace
after the GOROOT path with or without quotes it will blindly put that into GOTOOLDIR
e.g. 
REM The below line has an extra space at the end   
set GOROOT=C:\go 
Then if we run "go env":   
set GOTOOLDIR=C:\go \pkg\tool\windows_386
So seems like the problem is "larger" than just quotation marks. We aren't even trimming
GOROOT.   
Now I would accept that ALL of these are the user's fault. But the way BAT is designed
it makes these kind of mistakes fairly easy to do and fairly hard to detect once they're
done.   
All users get told is "fmt" and "runetime" packages not found (when building). Instead
of something like "WARNING: GOTOOLDIR not found."   
I mean even if we ignore the quote issue (edge case?) then it is hard to ignore the
whitespace issue.

@alexbrainman
Copy link
Member

Comment 6:

jgodfrey,
Your last discovery just proves my point - we should treat all these cases as user
mistake. Otherwise we'll be chasing our tail trying to do what cmd.exe does.
We should just display good error message, if os.Stat(runtime.GOROOT) fails.
Alex

@gopherbot
Copy link
Author

Comment 7 by jgodfrey@thelostsite.co.uk:

While I would really like to see a clearer error message, I am struggling with the
concept that an extra space character can be a "user mistake."    
Also, GOROOT with an extra space is a valid path since the space is stripped out at a
later point. The issue is with the construction of GOTOOLDIR where the space is used
verbatim as in C:\go \pkg\tool\windows_386 and the path will never be valid.    
I guess it depends on how "forgiving" you want go to be. You're essentially arguing that
go should be so unforgiving that it doesn't even understand the quirks of the underlying
OS/environment it runs on.   
So we get into a situation of shifting the responsibility of understanding Windows and
its quirks from the go team onto the end users themselves. It really isn't that rare for
users to set environment variables in BAT or even using the dialog, both of which
support quotes and extra spaces/whitespace.   
Only Powershell is likely not going to introduce these issues...

@gopherbot
Copy link
Author

Comment 8 by visan.ovidiu:

Tested on both windows 64 and 32 bit and the GOTOOLDIR it's correctly contructed if you
run the SET command without any quotes in path.
eg. SET GOROOT=C:\Program Files\go

@gopherbot
Copy link
Author

Comment 9 by eric.hill:

Quotes in Windows were intended for the command interpreter to understand where to break
command line arguments to an application:
app.exe "First Argument" "Second Argument"
Placing them into environment variables is not generally a good idea, including the
system PATH.  Applications that use PATH will split it on a semicolon and then use the
strings as prefixes to the file name they're looking for.  Environment variables with a
single path should be just that, the path:
SOMEPATH=C:\Program Files\Long Path\bin
Inside of a program that reads the variable, that expands to a single string that should
be treated as a path segment.  Command scripts work the same way:
@echo off
"%SOMEPATH%\app.exe" "First Argument" "Second Argument"
I think the parsing or stripping quotes from environment variables may cause unintended
side effects.  For example, an application that computes folder size:
@echo off
set APPPATH=C:\Program Files\Folder Sizer
set APPARGS=folder name
rem Use case 1
"%APPPATH%\app.exe" %APPARGS%
rem Use case 2
"%APPPATH%\app.exe" "%APPARGS%"
Use case 1 is very different from use case 2.  In case 1, the application should compute
the space for two folders, one named "folder", and one named "name".  In the second
case, the application should compute the space for a single folder called "folder name".
 If the application performed quote stripping, it may not work properly for case 2.

@rsc
Copy link
Contributor

rsc commented Mar 19, 2013

Comment 10:

I agree with the many people who have said that we should not interpret quotes here.

Status changed to Unfortunate.

@gopherbot
Copy link
Author

Comment 11 by JGodfrey16:

That's fine. But maybe an error then? You don't even need to do anything with the
quotes, just check if GOROOT exists. That way you catch quotes, whitespace (see above),
and simply invalid paths.    
Nobody has addressed the whitespace bug I posted above. Should I re-submit that one as a
new bug report?

@gopherbot
Copy link
Author

Comment 12 by jgodfrey@thelostsite.co.uk:

That's fine. But maybe an error then? You don't even need to do anything with the
quotes, just check if GOROOT exists. That way you catch quotes, whitespace (see above),
and simply invalid paths.    
Nobody has addressed the whitespace bug I posted above. Should I re-submit that one as a
new bug report?

@rsc
Copy link
Contributor

rsc commented Mar 19, 2013

Comment 13:

I don't know enough about Windows to know what to reject and what to accept. My
inclination is to require users to set environment variables correctly.
Note that the GOROOT environment variable need not be set at all as long as the Go
installation does not move after it is built (or is installed into the same directory
where it was built on another machine). 
So maybe the real fix is to stop telling people to set %GOROOT%.

@gopherbot
Copy link
Author

Comment 14 by jgodfrey@thelostsite.co.uk:

But a single space character after the environmental variable is "correct" from the
perspective of most users. Such a path can even be used by most system APIs to do
things. You just cannot concatenate it into the middle of a longer path with the space
still attached. Ditto with the quotes.   
Checking if the directory exists would be hugely helpful not just on Windows but also on
Linux and OSX. There is never a situation where invalid environmental variables won't
cause go to fail and fail in rather obscure ways.    
Plus you won't even have to handle every single edge case... Just one case - "is this
path valid? Yes|No" and almost every OS has functionality to check that for you.

@alexbrainman
Copy link
Member

Comment 15:

I suggest we check GOROOT as in https://golang.org/cl/7786047/.
Alex

@alexbrainman
Copy link
Member

Comment 16:

This issue was closed by revision d67e300.

Status changed to Fixed.

@rsc rsc added this to the Go1.1 milestone Apr 14, 2015
@rsc rsc removed the go1.1 label Apr 14, 2015
@golang golang locked and limited conversation to collaborators Jun 24, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants