Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: go mod vendor has started to report error when encountering permission errors #40723

Closed
Pawda opened this issue Aug 12, 2020 · 3 comments
Closed

cmd/go: go mod vendor has started to report error when encountering permission errors #40723

Pawda opened this issue Aug 12, 2020 · 3 comments
Labels
FrozenDueToAge GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Vendoring
Milestone
Backlog

Comments

@Pawda
Copy link

Pawda commented Aug 12, 2020

What version of Go are you using (go version)?

$ go version
go version go1.15 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

From Docker golang:1.15-alpine

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/localuser/.cache/go-build"
GOENV="/home/localuser/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/src/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build160550825=/tmp/go-build -gno-record-gcc-switches"

What did you do?

The same happens on Ubuntu but also on alpine so here is a reproducible Dockerfile

ARG GO_VERSION=1.15

FROM golang:${GO_VERSION}-alpine
ENV GO111MODULE=on

RUN adduser -D localuser
RUN mkdir /src && chown localuser /src
WORKDIR /src
USER localuser

RUN go mod init gomodisbroken
RUN echo $'\
package main \n\
\n\
import (\n\
    "fmt"\n\
    "github.com/keltia/leftpad"\n\
)\n\
\n\
func main() { \n\
    fmt.Println(leftpad.Pad("this should work", 42))\n\
}\n' | tee main.go

RUN go env

RUN mkdir -p utils/protected && chmod 000 utils/protected

RUN go mod vendor && go mod tidy

RUN CGO_ENABLED=0 go build -o a.out .

ENTRYPOINT [ "/src/a.out" ]

Then run for different go version:
go 1.15 breaking: docker build --build-arg GO_VERSION=1.15 -t gomodisbroken . && docker run --rm gomodisbroken
go 1.14 working: docker build --build-arg GO_VERSION=1.14 -t gomodisbroken . && docker run --rm gomodisbroken

What did you expect to see?

Expected go mod vendor to ignore silently folders it cannot read and exit with a 0 code.
Same as go1.14:

Step 12/14 : RUN go mod vendor && go mod tidy
 ---> Running in 56076b11eabe
go: finding module for package github.com/keltia/leftpad
go: downloading github.com/keltia/leftpad v0.1.0
go: found github.com/keltia/leftpad in github.com/keltia/leftpad v0.1.0
Removing intermediate container 56076b11eabe
 ---> 5a218931e08a
Step 13/14 : RUN CGO_ENABLED=0 go build -o a.out .
 ---> Running in 6fa88481a2ed
Removing intermediate container 6fa88481a2ed
 ---> e58b469d8dfa
Step 14/14 : ENTRYPOINT [ "/src/a.out" ]
 ---> Running in 41f90efe6d1c
Removing intermediate container 41f90efe6d1c
 ---> a35dbd80b4da
Successfully built a35dbd80b4da
Successfully tagged gomodisbroken:latest
                          this should work <nil>

What did you see instead?

with go 1.15: go mod vendor exited with code 1 due to permission denied on the folder protected.

Step 12/14 : RUN go mod vendor && go mod tidy
 ---> Running in 8a3fa42265cd
go: finding module for package github.com/keltia/leftpad
go: downloading github.com/keltia/leftpad v0.1.0
go: found github.com/keltia/leftpad in github.com/keltia/leftpad v0.1.0
pattern ...: open /src/utils/protected: permission denied
The command '/bin/sh -c go mod vendor && go mod tidy' returned a non-zero code: 1
@Pawda
Copy link
Author

Pawda commented Aug 12, 2020

Seems to have be introduced on purpose by https://golang.org/cl/232579

As found here:

go/src/cmd/go/internal/modload/search.go

Line 72 in 5c7748d

if strings.HasPrefix(elem, ".") || strings.HasPrefix(elem, "_") || elem == "testdata" {

A workaround is to have the protected folder named testdata or prefix it with "." or "_"

@dmitshur
Copy link
Contributor

dmitshur commented Aug 12, 2020 •
edited

I agree this appears to be working as intended.

/cc @bcmills @matloob @jayconrod FYI.

@dmitshur dmitshur added GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Vendoring labels Aug 12, 2020
@dmitshur dmitshur changed the title Breaking change introduced in go1.15 with "go mod vendor" cmd/go: go mod vendor has started to report error when encountering permission errors Aug 12, 2020
@dmitshur dmitshur added this to the Backlog milestone Aug 12, 2020
@bcmills
Copy link
Contributor

bcmills commented Aug 12, 2020

Yes, this is an intentional bug-fix. The go command cannot distinguish between an accidental permission error (such as when a directory is accidentally created as root) and an intentional one.

Note that you can also use a go.mod file to indicate that parts of the filesystem tree are intentionally excluded from the main module. So you could either add /src/utils/go.mod to prune out /src/utils and its subdirectories, or rename /src/utils/protected to /src/utils/_protected.

@bcmills bcmills closed this as completed Aug 12, 2020
ShoshinNikita added a commit to ShoshinNikita/budget-manager that referenced this issue Sep 15, 2020
@ShoshinNikita
mount directory '_var' instead of 'var'
646ed5c 
Directory 'var/pg_data' is created by root, and go 1.15 started to report permission errors. So, we exclude 'var' directory from the module by adding prefix '_'

Source of the fix: golang/go#40723 (comment)
@golang golang locked and limited conversation to collaborators Aug 12, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Vendoring
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
4 participants
@dmitshur @Pawda @bcmills @gopherbot