Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: "go build" fails if go.sum is not writable #30185

Closed
atombender opened this issue Feb 12, 2019 · 11 comments
Closed

cmd/go: "go build" fails if go.sum is not writable #30185

atombender opened this issue Feb 12, 2019 · 11 comments
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done. Tools This label describes issues relating to any tools in the x/tools repository.
Milestone
Go1.14

Comments

@atombender
Copy link

What version of Go are you using (go version)?

$ go version
1.11.5

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/alex/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/alex/.go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/opt/go/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/opt/go/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/alex/Trash/gotest/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/qs/wpmg19r12_9_nz7pvvs2_82r0000gn/T/go-build642111807=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I have a Dockerfile that runs go mod download followed by go build. Both the download and the build are run as an unprivileged user against a read-only codebase whose files are owned by root and not writable by anyone else.

A rudimentary dockerfile:

FROM golang:1.11-alpine AS build
RUN apk add --no-cache curl git ca-certificates \
  && addgroup -S build && adduser -S -G build build
WORKDIR /srv
COPY . ./
USER build
RUN go mod download
RUN go mod verify
RUN CGO_ENABLED=0 go build -mod=readonly -o ./server ./cmd/server

Possibly related to #29452?

What did you expect to see?

I expect go build to not attempt to write go.sum, especially if -mod=readonly is provided. I can obviously chmod the files to be writable, but this still shouldn't fail.

What did you see instead?

go: writing go.sum: open /srv/go.sum: permission denied
@agnivade agnivade added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Feb 12, 2019
@agnivade
Copy link
Contributor

/cc @bcmills

@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Feb 12, 2019
@bcmills bcmills added this to the Go1.13 milestone Feb 12, 2019
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Feb 12, 2019
@bcmills bcmills self-assigned this Feb 12, 2019
@bcmills bcmills mentioned this issue Feb 14, 2019
Closed
@pmer pmer mentioned this issue Feb 15, 2019
Open
@odeke-em odeke-em self-assigned this Feb 17, 2019
@saibing saibing mentioned this issue Jun 29, 2019
Open
@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019
@vaind vaind mentioned this issue Aug 8, 2019
Closed
@dsymonds
Copy link
Contributor

This was biting me just now. I believe the source of the difference is that my local dev machine has a newer version of Go than the continuous build I'm using, so when it runs go get -d ./... it gets a slightly different set of dependencies, then fails because it can't write go.sum.

I don't think failing to write go.sum should cause the go tool to exit unsuccessfully.

@dsymonds
Copy link
Contributor

It is also made worse in my situation by the go tool writing its temporary files to the package directory (where the go.mod and go.sum would end up) rather than using $TMPDIR.

@bcmills
Copy link
Contributor

bcmills commented Aug 19, 2019 •
edited

It is also made worse in my situation by the go tool writing its temporary files to the package directory (where the go.mod and go.sum would end up) rather than using $TMPDIR.

That part is necessary in general:

  • To ensure that we don't lose existing information if the go command is killed partway through a write, we write to a temporary file and then rename it over the existing file.
  • That rename operation fails if the temporary file is not on the same filesystem as the destination.
  • The most portable way to ensure that both files are on the same filesystem is to construct them in the same directory.

@dsymonds
Copy link
Contributor

I'm not sure I see why the go tool is so different to the multitude of tools that have existed for decades that quite happily use $TMPDIR for the same effect.

@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@bcmills bcmills modified the milestones: Backlog, Go1.14 Oct 10, 2019
@griesemer griesemer added the Tools This label describes issues relating to any tools in the x/tools repository. label Oct 16, 2019
@griesemer
Copy link
Contributor

Related to #34860?

@bcmills
Copy link
Contributor

bcmills commented Oct 16, 2019

Somewhat? They're mostly orthogonal, I think, but this issue can make the failure mode for #34860 worse (or less explicable).

@dowgird
Copy link

dowgird commented Oct 23, 2019

This makes guru fail on (some) projects fetched from Github. Reproduction & error message pasted below to make discovery easier.

go get github.com/GoogleCloudPlatform/osconfig
guru "-scope" "github.com/GoogleCloudPlatform/osconfig" "-tags=" "callstack" "$GOPATH/src/github.com/GoogleCloudPlatform/osconfig/inventory/inventory.go:#1629"

Result (snippet):

 [redacted]/go/pkg/mod/github.com/golang/protobuf@v1.3.2/ptypes/duration/duration.pb.go:8:8: could not import github.com/golang/protobuf/proto (go/build: importGo github.com/golang/protobuf/proto:      exit status 1
    error writing go.mod: open [redacted]/go/pkg/mod/github.com/golang/protobuf@v1.3.2/go.mod298498081.tmp: permission denied

@gopherbot
Copy link

Change https://golang.org/cl/204521 mentions this issue: cmd/go: default to mod=readonly when the go.mod file is read-only

gopherbot pushed a commit that referenced this issue Nov 1, 2019
cmd/go: default to mod=readonly when the go.mod file is read-only
2c8529c 
Updates #30185
Updates #33326
Updates #34822

Change-Id: Ie13651585898d1bbbf4f779b97ee50b6c7e7ad50
Reviewed-on: https://go-review.googlesource.com/c/go/+/204521
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
@bcmills
Copy link
Contributor

bcmills commented Nov 6, 2019

The use of $TMPDIR will be addressed incidentally by the fix for #34634.

The remainder is working as designed per #30667.

@bcmills
Copy link
Contributor

bcmills commented Nov 6, 2019

Duplicate of #34054

@bcmills bcmills marked this as a duplicate of #34054 Nov 6, 2019
@bcmills bcmills closed this as completed Nov 6, 2019
@honnix honnix mentioned this issue Nov 29, 2019
Merged
@golang golang locked and limited conversation to collaborators Nov 5, 2020
@rsc rsc unassigned odeke-em and bcmills Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done. Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Milestone
Go1.14
Development

No branches or pull requests
10 participants
@dsymonds @atombender @rsc @dowgird @andybons @agnivade @odeke-em @bcmills @griesemer @gopherbot