cmd/go/internal/envcmd: GOCACHE=off behavior difficult to debug #29243

kevinburke · 2018-12-14T00:55:41Z

Please answer these questions before submitting your issue. Thanks!

What did you do?

I have a test that calls go build with the following code:

	env := []string{
		"GOROOT=" + os.Getenv("GOROOT"),
		"GOPATH=" + filepath.Join(root, "src", "tool", "_gopath"),
	}
        cmdArgs := []string{"build", "-o", binPath}
	cmd := exec.Command("go", cmdArgs...)
	cmd.Env = append(env, "GOOS="+p.OS.String(), "GOARCH="+p.Arch.String())

Compiling with tip this failed first with the error "exit status 1", then when I printed out the stdout/stderr I got:

build cache is disabled by GOCACHE=off, but required as of Go 1.12

This was very confusing because I double and triple checked the env vars and the source code and I never referred to GOCACHE.

Finally, by editing the cmd/go source code and recompiling I was able to determine the actual problem was that I was not passing through the $HOME variable, so os.UserCacheDir() was failing, and cmd/go/internal/cache/default.go:DefaultDir was returning an error. $HOME is not a requirement to use the build cache on Go 1.11, or, perhaps this test program was failing to hit the build cache in Go 1.11 and it was not an error.

This is concerning because we should not expect the average Go user to recompile the Go command to determine what is wrong with their program.

It would be good to provide an error message that lets the user better debug what is wrong with their program and how/why GOCACHE is being set to off in the build environment.

Does this issue reproduce with the latest release (go1.11.2)?

No

System details

go version devel +a6293995c5 Thu Dec 13 23:58:06 2018 +0000 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/kevin/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/kevin"
GOPROXY=""
GORACE=""
GOROOT="/Users/kevin/go"
GOTMPDIR=""
GOTOOLDIR="/Users/kevin/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
GOROOT/bin/go version: cache default dir /Users/kevin/Library/Caches/go-build
go version devel +a6293995c5 Thu Dec 13 23:58:06 2018 +0000 darwin/amd64
GOROOT/bin/go tool compile -V: cache default dir /Users/kevin/Library/Caches/go-build
compile version devel +a6293995c5 Thu Dec 13 23:58:06 2018 +0000
uname -v: Darwin Kernel Version 18.2.0: Fri Oct  5 19:41:49 PDT 2018; root:xnu-4903.221.2~2/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.14.1
BuildVersion:	18B75
lldb --version: lldb-1000.0.38.2
  Swift-4.2

The text was updated successfully, but these errors were encountered:

kevinburke · 2018-12-14T12:24:12Z

see also #29251, it's concerning that this error message has appeared twice, which increases the likelihood (to me anyway) that a significant portion of the user base will hit it.

bcmills · 2018-12-14T16:55:39Z

$HOME is not a requirement to use the build cache on Go 1.11

On Darwin, the build cache requires either GOCACHE or os.UserCacheDir(), and os.UserCacheDir() requires HOME.

or, perhaps this test program was failing to hit the build cache in Go 1.11 and it was not an error.

That seems more likely. But we should certainly fix the diagnostics.

bcmills · 2018-12-14T16:59:22Z

FWIW, the diagnostic in this case is intended to read build cache is required, but could not be located: $HOME is not defined.

The logic for that is here, and seemed so trivial that I only tested one mode:

go/src/cmd/go/internal/cache/default.go

Lines 74 to 83 in 47fb1fb

    
           // die calls base.Fatalf with a message explaining why DefaultDir was "off". 
        
           func die() { 
        
           	if os.Getenv("GOCACHE") == "off" { 
        
           		base.Fatalf("build cache is disabled by GOCACHE=off, but required as of Go 1.12") 
        
           	} 
        
           	if _, err := os.UserCacheDir(); err != nil { 
        
           		base.Fatalf("build cache is required, but could not be located: %v", err) 
        
           	} 
        
           	panic(fmt.Sprintf("cache.die called unexpectedly with cache.DefaultDir() = %s", DefaultDir())) 
        
           }

Obviously I should have tested more thoroughly. 🙂

FiloSottile · 2018-12-14T18:11:20Z

Are we committed to not letting users build code unless they set $HOME or $GOCACHE? I'd expect this to break quite a few Docker images, which probably didn't want to keep the cache around anyway. Also, I run some machines with a read-only $HOME, which I guess won't be able to build Go code anymore.

It already broke tip.golang.org and stuff inside Google, too.

bcmills · 2018-12-14T18:42:40Z

Are we committed to not letting users build code unless they set $HOME or $GOCACHE?

It's what we announced in the 1.11 release notes, and nobody complained about it then.

We could perhaps put the cache in os.TempDir() if we can't find it some other place, though.

bcmills · 2018-12-14T18:42:54Z

At any rate, that seems like a separate issue from the broken diagnostic message.

kevinburke · 2018-12-14T18:45:03Z

Apologies for not complaining, I guess I didn’t realize the implications. I compile from tip daily and just noticed it now

On Fri, Dec 14, 2018 at 18:44 Bryan C. Mills ***@***.***> wrote: At any rate, that seems like a separate issue from the broken diagnostic message. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#29243 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAOSI9PaxaLmLPhn3XD-j4eg_CsdVHKeks5u4_F3gaJpZM4ZSsld> .

-- Kevin Burke phone: 925.271.7005 | kev.inburke.com

FiloSottile · 2018-12-14T19:49:25Z

At any rate, that seems like a separate issue from the broken diagnostic message.

Opened #29267.

gopherbot · 2018-12-14T22:07:47Z

Change https://golang.org/cl/154309 mentions this issue: cmd/go/internal/cache: save more data from DefaultDir

purpleidea · 2018-12-21T12:57:37Z

build cache is disabled by GOCACHE=off, but required as of Go 1.12

Just started seeing this in all my travis builds. Which broke everything! I set this variable in my .envrc because I NEVER want the tests cached! How else can I disable test caching now??

ianlancetaylor · 2018-12-21T14:21:16Z

You can always disable test caching by using the -test.count=1 option, as documented at https://golang.org/cmd/go/#hdr-Test_packages.

gopherbot · 2019-02-26T08:04:52Z

Change https://golang.org/cl/163798 mentions this issue: playground: use Go 1.12

Go 1.12 has been released¹ and should be used, so that it's possible to play with Go programs that rely on Go 1.12-only features. In Go 1.12, build cache is required. Either GOCACHE must be set, or HOME must be set so that GOCACHE can be implicitly deduced. Set HOME and pass it on to the go build invocation inside the compileAndRun function. This fixes the following error, detected by the playground test: 2019/02/26 06:28:44 compile error: build cache is required, but could not be located: GOCACHE is not defined and neither $XDG_CACHE_HOME nor $HOME are defined The command '/bin/sh -c /app/playground test' returned a non-zero code: 1 This is related to issues golang/go#29243 and golang/go#29251, and the fix in CL 154181. ¹ https://groups.google.com/d/msg/golang-announce/Y1RZqnCKUKY/N9yK5c8iBgAJ Fixes golang/go#30397 Change-Id: I44c5c8928060732f31fd935b114568258c7298c7 Reviewed-on: https://go-review.googlesource.com/c/163798 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

bcmills added the GoCommand cmd/go label Dec 14, 2018

bcmills self-assigned this Dec 14, 2018

bcmills added this to the Go1.12 milestone Dec 14, 2018

bcmills added the release-blocker label Dec 14, 2018

FiloSottile mentioned this issue Dec 14, 2018

cmd/go: builds fail if neither $HOME nor $GOCACHE are set #29267

Closed

gopherbot closed this as completed in fd323a8 Dec 15, 2018

golang locked and limited conversation to collaborators Feb 26, 2020

gopherbot added the FrozenDueToAge label Feb 26, 2020

rsc unassigned bcmills Jun 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go/internal/envcmd: GOCACHE=off behavior difficult to debug #29243

cmd/go/internal/envcmd: GOCACHE=off behavior difficult to debug #29243

kevinburke commented Dec 14, 2018

kevinburke commented Dec 14, 2018

bcmills commented Dec 14, 2018

bcmills commented Dec 14, 2018

FiloSottile commented Dec 14, 2018

bcmills commented Dec 14, 2018

bcmills commented Dec 14, 2018

kevinburke commented Dec 14, 2018 via email

FiloSottile commented Dec 14, 2018

gopherbot commented Dec 14, 2018

purpleidea commented Dec 21, 2018

ianlancetaylor commented Dec 21, 2018

gopherbot commented Feb 26, 2019

cmd/go/internal/envcmd: GOCACHE=off behavior difficult to debug #29243

cmd/go/internal/envcmd: GOCACHE=off behavior difficult to debug #29243

Comments

kevinburke commented Dec 14, 2018

What did you do?

Does this issue reproduce with the latest release (go1.11.2)?

System details

kevinburke commented Dec 14, 2018

bcmills commented Dec 14, 2018

bcmills commented Dec 14, 2018

FiloSottile commented Dec 14, 2018

bcmills commented Dec 14, 2018

bcmills commented Dec 14, 2018

kevinburke commented Dec 14, 2018 via email

FiloSottile commented Dec 14, 2018

gopherbot commented Dec 14, 2018

purpleidea commented Dec 21, 2018

ianlancetaylor commented Dec 21, 2018

gopherbot commented Feb 26, 2019