Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Go mod vendor fails where go mod download succeeds #38820

Closed
c00w opened this issue May 3, 2020 · 5 comments
Closed

Go mod vendor fails where go mod download succeeds #38820

c00w opened this issue May 3, 2020 · 5 comments

Comments

@c00w
Copy link

c00w commented May 3, 2020

What version of Go are you using (go version)?

go version go1.14.1 linux/amd64

Does this issue reproduce with the latest release?

Yes - 14.1 is the latest release

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
 % go env                                         ~/Downloads/tiny2 
GO111MODULE=""                                                      
GOARCH="amd64"                                                      
GOBIN=""                                                            
GOCACHE="/home/colin/.cache/go-build"                               
GOENV="/home/colin/.config/go/env"                                  
GOEXE=""                                                            
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/colin"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/nix/store/gvw1mfpdrk7i82884yhxf9lf5j3c12zm-go-1.14.1/share/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/nix/store/gvw1mfpdrk7i82884yhxf9lf5j3c12zm-go-1.14.1/share/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="/nix/store/k930l9lckxjf4zmafv4aapvdbd2sibmj-gcc-wrapper-9.3.0/bin/cc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/colin/Downloads/tiny2/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build181210972=/tmp/go-build -gno-record-gcc-switches"

What did you do?

git clone https://github.com/tinygo-org/tinygo.git
cd tinygo
git checkout v0.13.1
go mod download
go mod vendor

What did you expect to see?

I expect that if go mod download succeeds, go mode vendor should as well

What did you see instead?

% go mod vendor ~/Downloads/tinygo
github.com/tinygo-org/tinygo/src/device/arm imports
runtime/volatile: package runtime/volatile is not in GOROOT
(/nix/store/gvw1mfpdrk7i82884yhxf9lf5j3c12zm-go-1.14.1/share/go/src/
runtime/volatile)
github.com/tinygo-org/tinygo/src/examples/adc imports
machine: package machine is not in GOROOT (/nix/store/gvw1mf
pdrk7i82884yhxf9lf5j3c12zm-go-1.14.1/share/go/src/machine)
github.com/tinygo-org/tinygo/src/examples/systick imports
device/arm: package device/arm is not in GOROOT (/nix/store/
gvw1mfpdrk7i82884yhxf9lf5j3c12zm-go-1.14.1/share/go/src/device/arm)
github.com/tinygo-org/tinygo/src/machine imports
device/avr: package device/avr is not in GOROOT (/nix/store/
gvw1mfpdrk7i82884yhxf9lf5j3c12zm-go-1.14.1/share/go/src/device/avr)
github.com/tinygo-org/tinygo/src/machine imports
device/nrf: package device/nrf is not in GOROOT (/nix/store/
gvw1mfpdrk7i82884yhxf9lf5j3c12zm-go-1.14.1/share/go/src/device/nrf)
github.com/tinygo-org/tinygo/src/machine imports
device/sam: package device/sam is not in GOROOT (/nix/store/
gvw1mfpdrk7i82884yhxf9lf5j3c12zm-go-1.14.1/share/go/src/device/sam)
github.com/tinygo-org/tinygo/src/machine imports
device/sifive: package device/sifive is not in GOROOT (/nix/
store/gvw1mfpdrk7i82884yhxf9lf5j3c12zm-go-1.14.1/share/go/src/device/sifive)
github.com/tinygo-org/tinygo/src/machine imports
device/stm32: package device/stm32 is not in GOROOT (/nix/store/gvw1mfpdrk7i82884yhxf9lf5j3c12zm-go-1.14.1/share/go/src/device/stm32)
github.com/tinygo-org/tinygo/src/machine imports
runtime/interrupt: package runtime/interrupt is not in GOROOT (/nix/store/gvw1mfpdrk7i82884yhxf9lf5j3c12zm-go-1.14.1/share/go/src/runtime/interrupt)
github.com/tinygo-org/tinygo/src/runtime imports
device/riscv: package device/riscv is not in GOROOT (/nix/store/gvw1mfpdrk7i82884yhxf9lf5j3c12zm-go-1.14.1/share/go/src/device/riscv)
github.com/tinygo-org/tinygo/src/runtime imports
internal/task: package internal/task is not in GOROOT (/nix/store/gvw1mfpdrk7i82884yhxf9lf5j3c12zm-go-1.14.1/share/go/src/internal/task)

Context

I'm working on migrating how the nixos operating system downloads go modules. Currently we use the files produced by go mod download (which includes some non deterministic content). I'd like to use go mod vendor (and I've gotten it working almost everywhere), however this specific failure is confusing me. In general it appears that go mod vendor, go -mod=vendor install does not work everywhere go mod download, then go mod install works.

NixOS/nixpkgs#86376

@nigelis
Copy link

nigelis commented May 3, 2020

This issue has nothing to do with go modules.

The missing packages are located under https://github.com/tinygo-org/tinygo/tree/master/src, including runtime/volatile, machine, device/arm, device/avr, etc..
You should follow tinygo's guide to build from source, https://github.com/tinygo-org/tinygo/blob/master/BUILDING.md.

@bcmills
Copy link
Contributor

bcmills commented May 4, 2020

go mod download operates on module dependencies. go mod vendor operates on packages within those modules.

If you have an error at the package level that does not correspond to an error at the module level, then it is expected that go mod vendor would detect that error where go mod download does not.

(go mod tidy, in contrast, also operates on packages. Please do let us know if you see a discrepancy between go mod tidy and go mod vendor that cannot be explained by the former following test dependencies more aggressively.)

@bcmills bcmills closed this as completed May 4, 2020
@c00w
Copy link
Author

c00w commented May 5, 2020

Hmmm - That's definitely confusing. The UX of having two different modes is pretty frustrating, since It doesn't seem to make sense that you need to do more work for go mod vendor than go mod download.

It's also not very intuitive that go install can succeed, even though go mod tidy and go vendor fail :(

@bcmills
Copy link
Contributor

bcmills commented May 5, 2020

It doesn't seem to make sense that you need to do more work for go mod vendor than go mod download.

It isn't more work — it is different work. (go mod download downloads the module dependencies of the main module. go mod vendor copies over the package dependencies of the main module. They walk different-but-related graphs.)

@c00w
Copy link
Author

c00w commented May 8, 2020

Sure - but go mod vendor appears to have nothing to do with modules - go mod download downloads modules, go mod tidy reads the package graph and updates the go.mod file, go mod vendor reads the package graph and makes a vendor file (apparently mostly ignoring the contents of go.mod). I guess modules are involved since it reads the go module to figure out versions?

Would a flag to go vendor to make it just look at the contents of the go.mod file be a useful patch? (Otherwise I'll probably write a tool for this externally, but it seems like missing functionality from go).

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants