Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: vendored x/y cannot import unvendored x/internal/z #16622

Closed
seanhagen opened this issue Aug 5, 2016 · 10 comments
Closed

cmd/go: vendored x/y cannot import unvendored x/internal/z #16622

seanhagen opened this issue Aug 5, 2016 · 10 comments
Labels
FrozenDueToAge NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made.
Milestone

Comments

@seanhagen
Copy link

Please answer these questions before submitting your issue. Thanks!

  1. What version of Go are you using (go version)?

1.6.3

  1. What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/sean/Code/Go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GO15VENDOREXPERIMENT="1"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="g++"
CGO_ENABLED="1"
  1. What did you do?
    If possible, provide a recipe for reproducing the error.
    A complete runnable program is good.
    A link on play.golang.org is best.

Created a repository named 'internal', to hold organization-level packages for a project of microservices and api clients ( "github.com/seanhagen/internal", example package "github.com/seanhagen/internal/tokens" )

Inside of a project using vendored dependencies ( via godep ), attempt to import a package from internal repo.

  1. What did you expect to see?

Binary to compile without issue.

  1. What did you see instead?
package github.com/seanhagen/test
        imports github.com/seanhagen/internal/tokens: use of internal package not allowed

However, when the vendor folder is removed, go build works fine. It seems that because the internal package is now located at vendor/github.com/seanhagen/internal/session, go build thinks that the files I'm trying to build are outside that tree/scope, and blocks the import of the internal package.

It also works when just the github.com/seanhagen/internal folder is removed from the vendor folder.

@davecheney
Copy link
Contributor

I'm sorry, you cannot use "internal" as a name for a repository. Sorry this
is not well documented, you will have to rename your repo.

On Sat, Aug 6, 2016 at 5:51 AM, Sean Hagen notifications@github.com wrote:

Please answer these questions before submitting your issue. Thanks!

  1. What version of Go are you using (go version)?

1.6.3

  1. What operating system and processor architecture are you using (go
    env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/sean/Code/Go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GO15VENDOREXPERIMENT="1"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="g++"
CGO_ENABLED="1"

  1. What did you do? If possible, provide a recipe for reproducing the
    error. A complete runnable program is good. A link on play.golang.org
    is best.

Created a repository named 'internal', to hold organization-level packages
for a project of microservices and api clients ( "
github.com/seanhagen/internal", example package "github.com/seanhagen/
internal/tokens" )

Inside of a project using vendored dependencies ( via godep ), attempt to
import a package from internal repo.

  1. What did you expect to see?

Binary to compile without issue.

  1. What did you see instead?

package github.com/seanhagen/test
imports github.com/seanhagen/internal/tokens: use of internal package not allowed

However, when the vendor folder is removed, go build works fine. It seems
that because the internal package is now located at vendor/
github.com/seanhagen/internal/session, go build thinks that the files I'm
trying to build are outside that tree/scope, and blocks the import of the
internal package.

It also works when just the github.com/seanhagen/internal folder is
removed from the vendor folder.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#16622, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAcA5_94cdR5e_z6lMYP1ZLWHuoi2Qnks5qc5ROgaJpZM4JeAcP
.

@seanhagen
Copy link
Author

@davecheney is there a reason for this? I thought based on the way the doc I found was written, it didn't matter where 'internal' was in the chain of the package name.

Is it just a weird edge case where vendoring and internal packages don't play nice that's not easy to fix? Cause it works fine without vendoring.

If this isn't ever going to be fixed, that's okay, I'm just trying to get a better understanding of what's going on that's causing the issue.

@davecheney
Copy link
Contributor

I think it's just an update expected side effect of the combination of the
two features.

I'm sorry I cannot advise you if or when it would be fixed.

On Sat, 6 Aug 2016, 09:45 Sean Hagen notifications@github.com wrote:

@davecheney https://github.com/davecheney is there a reason for this? I
thought based on the way the doc I found
https://docs.google.com/document/d/1e8kOo3r51b2BWtTs_1uADIA5djfXhPT36s6eHVRIvaU/edit
was written, it didn't matter where 'internal' was in the chain of the
package name.

Is it just a weird edge case where vendoring and internal packages don't
play nice that's not easy to fix? Cause it works fine without vendoring.

If this isn't ever going to be fixed, that's okay, I'm just trying to get
a better understanding of what's going on that's causing the issue.


You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
#16622 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAAcA28WGZ9xJo_uxQH5PJhVcHvRsN_5ks5qc8sXgaJpZM4JeAcP
.

@kostya-sh
Copy link
Contributor

kostya-sh commented Aug 6, 2016

On Aug 6, 2016 00:06, "Dave Cheney" notifications@github.com wrote:

I'm sorry, you cannot use "internal" as a name for a repository. Sorry this
is not well documented, you will have to rename your repo.

Is it true though? Go compiler doesn't even know that it's a repository.
Also gonum uses repository named internal: github.com/gonum/internal

@josharian josharian changed the title Vendor folder breaks internal package cmd/go: cannot vendor package named "internal" Aug 8, 2016
@kostya-sh
Copy link
Contributor

I have tried to reproduce this issue with Go 1.6.2 and Go 1.7 (tip) but couldn't: https://github.com/kostya-sh/sandbox/tree/master/go/16622

@seanhagen
Copy link
Author

@kostya-sh You have pkgA importing the internal package, and then calling pkga.A from main.

The problem I ran into was if pkga is the main entry point, then the compilation fails.

For example, if I change pkga.go to look like this:

package main

import "github.com/org1/internal/test"

// A prints "A"
func main() {
    test.A()
}

And the internal repo is in vendor, then I get this error:

package github.com/kostya-sh/sandbox/go/16622/vendor/github.com/org1/pkga
        imports github.com/org1/internal/test: use of internal package not allowed

Dir structure for clarity

$ tree
.
├── pkga.go
└── vendor
    └── github.com
        └── org1
            └── internal
                └── test
                    └── test.go

5 directories, 3 files

@kostya-sh
Copy link
Contributor

@seanhagen, so what you are saying is that github.com/org1/pkga should have access to github.com/org1/pkga/vendor/github.com/org1/internal/test because it already has access to github.com/org1/internal/test. This is probably reasonable.

I have updated https://github.com/kostya-sh/sandbox/tree/master/go/16622

@quentinmit quentinmit added this to the Go1.8 milestone Aug 8, 2016
@seanhagen
Copy link
Author

@kostya-sh yeah, pretty much.

@quentinmit quentinmit added the NeedsFix The path to resolution is known, but the work has not been done. label Oct 6, 2016
@quentinmit
Copy link
Contributor

/cc @rsc to make sure we want to do this.

@rsc rsc added NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. and removed NeedsFix The path to resolution is known, but the work has not been done. labels Oct 6, 2016
@rsc rsc self-assigned this Oct 6, 2016
@rsc
Copy link
Contributor

rsc commented Oct 21, 2016

Thanks @kostya-sh for helping to clarify this and for the repo.

To restate the problem, the issue is:

  1. We have code x/y that imports x/y/internal/z. This works.
  2. We add x/vendor/x/y/internal/z.
  3. Now x/y's import of x/y/internal/z is interpreted as really meaning x/vendor/x/y/internal/z, but the rules for "internal" visibility disallow that.

This follows from the definitions of internal and vendor.

The definition of internal is intentionally about source directory location and not about comparing just import paths. If you have $GOPATH1/src/internal/foo then $GOPATH2/src/bar is not allowed to import "internal/foo" even though $GOPATH1/src/quux is. It is also intentional that internal's definition does not mention vendor, and vendor's definition does not mention internal. They are independent concepts.

I understand that this case shows they interact in an unexpected way, but if we make an exception then they will interact in different unexpected ways. I don't think there's an obvious "everything is simple now" solution, so I would prefer to avoid introducing new special cases.

I am not sure that internal is doing much work in the example. It might be best to rename the repo to something like base, core, common, lib.

If this keeps coming up in other contexts we can revisit. But for now I'd like to be conservative and leave things alone.

@rsc rsc changed the title cmd/go: cannot vendor package named "internal" cmd/go: vendored x/y cannot import unvendored x/internal/z Oct 21, 2016
@rsc rsc closed this as completed Oct 21, 2016
@golang golang locked and limited conversation to collaborators Oct 21, 2017
@rsc rsc removed their assignment Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made.
Projects
None yet
Development

No branches or pull requests

6 participants