Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: relative imports do not work with vendoring #15478

Closed
titpetric opened this issue Apr 28, 2016 · 17 comments
Closed

cmd/go: relative imports do not work with vendoring #15478

titpetric opened this issue Apr 28, 2016 · 17 comments
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone

Comments

@titpetric
Copy link

  1. What version of Go are you using (go version)?

go version go1.6.1 linux/amd64

  1. What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GO15VENDOREXPERIMENT="1"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="g++"
CGO_ENABLED="1"

Linux serenity 4.3.0-1-amd64 #1 SMP Debian 4.3.5-1 (2016-02-06) x86_64 GNU/Linux
Running golang with Docker image, official golang build, latest image from docker hub.
Full reproduction script available with ./run

Source files: https://cdn.si/assets/go-gh-issue-vendoring.tgz

  1. What did you do?

I created a small program, using a local subpackage.
main.go (using a subpackage + vendor) = experienced problem.
main2.go (using vendored import from main) = works as expected.

Subpackage is imported with: import "./apiservice". Import of vendored package from this subpackage results in a run/build error.

Also included is a test-runner using the official golang docker image. It's the same environment, that I'm running the example in. The folder vendor was created with gvt fetch github.com/namsral/flag, full source provided.

  1. What did you expect to see?

Expected output with main.go:

# go run main2.go
Hello world! Network port: 8080
  1. What did you see instead?
# go run main.go
apiservice/apiservice.go:4:8: cannot find package "_/go/src/app/vendor/github.com/namsral/flag" in any of:
        /usr/local/go/src/_/go/src/app/vendor/github.com/namsral/flag (from $GOROOT)
        /go/src/_/go/src/app/vendor/github.com/namsral/flag (from $GOPATH)

Problem:

Import path for local subpackage is not honored. The dot gets converted to underscore, and as a consequence, it's looking for the vendor directory in the wrong path:

Expected path:

        /usr/local/go/src/app/vendor/github.com/namsral/flag (from $GOROOT)
@davecheney
Copy link
Contributor

davecheney commented Apr 28, 2016

Your need to put your source in GOPATH. From the example you posted (sorry
I haven't run your script) you have not done this.

@titpetric
Copy link
Author

@davecheney I've been putting it in GOROOT, I updated now to use GOPATH, same issue persists.

#!/bin/bash
printf "Expected:\n\n"
docker run --rm -it -v `pwd`:/go/src/app -w /go/src/app golang go run main2.go
printf "\nActual:\n\n"
docker run --rm -it -v `pwd`:/go/src/app -w /go/src/app golang go run main.go
# ./run_gopath
Expected:

Hello world! Network port: 8080

Actual:

apiservice/apiservice.go:4:8: cannot find package "_/go/src/app/vendor/github.com/namsral/flag" in any of:
        /usr/local/go/src/_/go/src/app/vendor/github.com/namsral/flag (from $GOROOT)
        /go/src/_/go/src/app/vendor/github.com/namsral/flag (from $GOPATH)

So, sorry, no. I also tried putting everything in /go instead of /go/src/app but that breaks both examples.

@davecheney
Copy link
Contributor

davecheney commented Apr 28, 2016

I belive GOPATH is not set in your enviroment correctly. I believe this
because of the leading _/ in the error report. This is the path that the go
tool uses when source is outside GOPATH or GOPATH is not set correctly.
If GOPATH is not set correctly, vendoring will not be used.

Is it possible to isolate your issue by removing docker?

@titpetric
Copy link
Author

I don't have an environment outside of docker where I can run the tests. If I get one, I'll post the results, and if you have one, please just stick the sources from the tgz into $GOPATH/src/app and run go run main.go to see if the error occurs with you.

But, my sanity check here tells me that you shouldn't be so suspicious, go env reports /go (same as $GOPATH in bash), sources don't reference some /other or ../other locations. I might be quite new at this, but I'm pretty sure that nowhere in about 10-20 lines of code I'm going outside of GOPATH, and I'm not touching it in any way. The obvious question is "what's the correct way to set up GOPATH?" if the official docker image doesn't do this. Source Dockerfile is here, if you can spot the issue: https://github.com/docker-library/golang/blob/master/1.6/Dockerfile

I'll attach my results if I can get my hands on on a local install of go somewhere.

@davecheney
Copy link
Contributor

davecheney commented Apr 28, 2016

You shouldn't need docker to compile your tests -- they may not pass, but
you'll at least confirm that you can compile your code.

@davecheney
Copy link
Contributor

davecheney commented Apr 28, 2016

I just checked out your sample. Sorry I should have done that before.

The root cause is you are using go run, which does not support vendoring.

I recommend you lay out your code in the prescribed package format as
documented here https://golang.org/doc/code.html

Then compile and run your program.

@titpetric
Copy link
Author

I'm having the same result with go build, but I'll check the layout extensively. I followed the recommended layout from glide - in the main README, which should be this:

$GOPATH(/go)/src/your-project
  |__ main.go (Where your main code lives)
  |__ subpackage/
  |     |__ subpackage.go (You can create subpackages)
  |     \__ ...
  |__ ... (Other files or subpackages)
  \__ vendor/
        |__ github.com/namsral/flag
        |__ github.com/gorilla/mux
        \__ ...

So if I understand you, instead of using "subpackage" I should create src/subpackage and use import "subpackage" as a replacement? I'm just about to go AFK into a meeting, and I'll try this asap. But it really seems that I shouldn't. Importing subpackages which reside in the same level is done with "./package", which is a documented thing somewhere. It actually works... at least some packages use a slightly different approach to structure the sources in the same way, with the double-slash import option:

package main // import "github.com/janeczku/go-dnsmasq"
import "github.com/janeczku/go-dnsmasq/hostsfile"

Is there a way to achieve something similar for local development without a recognized VCS location? I believed until now that: import "./package" was it.

@davecheney
Copy link
Contributor

davecheney commented Apr 28, 2016

Is there a way to achieve something similar for local development without
a recognized VCS location? I believed until now that: import "./package"
was it.

Yes, packages are just directories on disk. They follow the url based
convention to avoid namespace collision and to enable go get. You should
layout your code using the url based pattern, but there is no requirement
(unless you want someone else to go get your code) to push it to a remove
vcs location

@kostya-sh
Copy link
Contributor

I think that currently local (relative) imports do not work with vendoring. I am not sure if it is even possible to make these features work together. Local imports do not work in some other cases as well (see for example http://stackoverflow.com/questions/10687627/relative-import-from-parent-directory).

@titpetric
Copy link
Author

@davecheney It is just slightly awkward if you're using go get to mix remote packages from github, etc, with whatever you have locally. main.go and package/package.go with import "./package" would seem like a valid usage pattern.

@kostya-sh That's unfortunate. I guess I'd mark this issue as a feature request then, I would very much like local imports to work with vendoring.

@titpetric
Copy link
Author

titpetric commented Apr 28, 2016

If anybody is coming here for a fix, this worked for me:

  1. My app name is 'app' (/go/src/app),
  2. instead of importing ./submodule, import app/submodule
  3. Vendoring starts to work, everything works with go run.

I will not close this issue, just because I expect that this should be fixed in a future version. I hope someone agrees and tags/assigns the issue properly. Relative imports should work with vendoring, but I understand the motivations for FQDN imports. Considering that i'm using them from the main package, which is a reasonable "root" of the project, and if I'll ever reuse it from another package i would use the FQDN import from a VCS/other.

In the current state I would have to rewrite './' to $(basename $(dirname $0)) before doing go build/go run, which would require some external tooling like gb. Don't want external tools for something that simple.

@ianlancetaylor ianlancetaylor changed the title Subpackage vendoring issue cmd/go: relative imports do not work with vendoring Apr 28, 2016
@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Apr 28, 2016
@bcmills
Copy link
Contributor

bcmills commented Jan 23, 2019

It looks like there were several different issues at play in this thread.

@titpetric, could you summarize the remaining problem that you want to be fixed, ideally as a list of commands that we can run in isolation to observe the problem? (A .tgz file is not a suitable format for a bug report, sorry.)

I suspect that this is another instance of #14566 / #18007, but since I don't really understand the problem statement it's hard to be sure.

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jan 23, 2019
@titpetric
Copy link
Author

titpetric commented Jan 24, 2019

There's only one issue, vendoring doesn't play well with relative imports.

/main.go

package main

import (
        "fmt"
        "./errors"
)

func main() {
        fmt.Println(errors.Error404())
}

/errors/errors.go

package errors

import (
        e "github.com/pkg/errors"
)

func Error404() error {
        return e.New("Not found")
}

Expected result: stdout Not found

Actual result:

errors/errors.go:4:2: cannot find package "_/go/src/app/vendor/github.com/pkg/errors" in any of:
        /usr/local/go/src/_/go/src/app/vendor/github.com/pkg/errors (from $GOROOT)
        /go/src/_/go/src/app/vendor/github.com/pkg/errors (from $GOPATH)

I verified the above with Go 1.6-1.11:

docker run --rm -it -v `pwd`:/go/src/app -w /go/src/app golang:1.6-alpine go run main.go
docker run --rm -it -v `pwd`:/go/src/app -w /go/src/app golang:1.7-alpine go run main.go
docker run --rm -it -v `pwd`:/go/src/app -w /go/src/app golang:1.8-alpine go run main.go
docker run --rm -it -v `pwd`:/go/src/app -w /go/src/app golang:1.9-alpine go run main.go
docker run --rm -it -v `pwd`:/go/src/app -w /go/src/app golang:1.10-alpine go run main.go
docker run --rm -it -v `pwd`:/go/src/app -w /go/src/app golang:1.11-alpine go run main.go

@bcmills
Copy link
Contributor

bcmills commented Jan 24, 2019

Neither of the paths listed in that comment includes a vendor component. Is the whole thing in the vendor directory? If so, where are you running the go command?

Please provide a list of specific commands that we can run to reproduce the problem, starting from an empty GOPATH. The details are important.

@bcmills bcmills added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Jan 24, 2019
@titpetric
Copy link
Author

titpetric commented Jan 24, 2019 via email

@titpetric
Copy link
Author

Here you have it:

git clone git@github.com:titpetric/go-issues-15478.git
cd go-issues-15378
./test.sh

@bcmills
Copy link
Contributor

bcmills commented Jan 25, 2019

Ok, this is definitely #18007.

When you use a relative path from within main.go by using go run with a filename, the imported package does not have an ordinary GOPATH-based import path; instead, it has a synthetic path derived from the relative file path. vendor support uses the import path to resolve the locations of vendor directories, so it cannot find the directory using the synthetic path.

@bcmills bcmills closed this as completed Jan 25, 2019
@golang golang locked and limited conversation to collaborators Jan 25, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

6 participants