Related Issues

• mod/vendor: broken support for git-submodules in vendoring mode #56920 (closed)
• cmd/go: can use invalid sub-dependency version #69437 (closed)
• cmd/go: bad error message when using `vendor` directory while specifying too old Go version #67587 (closed)
• cmd/go: go mod vendor ignore workspace #54452 (closed)
• cmd/go: Inconsistent vendoring detected. Please re-run "go mod vendor" #45103 (closed)
• cmd/go: error out in 'go mod tidy' if the result uses a module for multiple paths #34650 (closed)
• cmd/go: import of vendored package error "cannot find package" from testdata package #50143

Related Code Changes

• cmd/go: import vendored packages only listed in modules.txt
• cmd/go: show an error when a package in a module conflicts with one in std

Related Discussions

• About the change for default "-mod=vendor" in Go 1.14

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

Issue #37397 was about some packages whose import path ends with "/vendor" being treated as if they're vendored, and being left out of module zips.

It looks like what you're running into isn't that issue, since the import paths in the error message do not end with "/vendor", but instead they contain "/vendor/". See https://pkg.go.dev/cmd/go#hdr-Vendor_Directories for documentation about vendor directories, which applies:

Code below a directory named "vendor" is importable only by code in the directory tree rooted at the parent of "vendor", and only using an import path that omits the prefix up to and including the vendor element.

Does the problem go away if you rename the directory vendor in your repo to something else (e.g., "copy") and update the import paths in dummy.go accordingly?

@dmitshur I created issue #37397 -- and while you're right about the failing example being an import path that "ends with /vendor", the intention of the issue was certainly to allow "vendor" folders in all but the root of a module, and that's how I understood that the fix in "x/mod: fix handling of vendored packages with '/vendor' in non-top-level paths" tried to accomplish, but I might have read it wrong.

As the issue starter I can confirm that the issue I reported isn't fixed, as in: It didn't fix MY issue.

Does the problem go away if you rename the directory vendor in your repo to something else (e.g., "copy") and update the import paths in dummy.go accordingly?

It probably would go away after renaming this, however its pretty difficult to rename it as my code is auto-generated and this vendor directory was called by C dependency's maintainer, not by me.
Also I think that as long as vendor sub directories in the root of the module should deffinitly be ommited, the same behaviour should not apply to vendor's supdirectories in non-top-level packages (this doesn't make sense for me tbh).

This documentation quote imo applies only to the "real" vendor directories (these in top-level module directory) not such a "fake" vendors like mine.

I'm coming here from #71978, it looks to be similar but specific to the embed.FS confusion. For my issue renaming the vendor breaks the usecase for the test type I make use of the embed.FS. Was able to reproduce this is a tiny pair of modules listed in the issue.

@dmitshur, I'm curious if the link you shared re: vendor directories should apply to directories that do not have *.go files within? Take my testdata/foo/vendor/bar/msg.txt file for example. My example doesn't involve importing any go code, just creating a file at the correct path that represents real constraints. Its also deeply nested in that filepath. It makes sense the embed.FS would contain everything within the testdata dir developing within the module that contains it. However, its confusing that go.work doesn't respect the same rules as go get might in this situation. That confusion cost me a fair bit of time.

@jsteenb2 The intersection of directories "testdata" and "vendor" is an interesting edge case. (It's possible it already came up and was discussed in a prior issue.) It may be reasonable to expect that testdata directory should be able to contain arbitrary data for testing, possibly including directories named vendor. In practice, there will still be some constraints, such as go.mod files inside testdata still having the effect of carving out that directory from the parent module. And it seems directories named vendor currently have the same behavior - today's implementation of isVendoredPackage only looks for "/vendor/" path elements regardless of whether it's inside a "testdata" directory.

Edit: Also note that embed package documentation hints at this limitation, albeit indirectly given its intersection with this issue: "Patterns must not match files outside the package's module, such as ‘.git/*’, symbolic links, 'vendor/', or any directories containing go.mod (these are separate modules)." Emphasis mine.

Given that the /vendor/ directory started out as having effect anywhere back in GOPATH mode, but has been refined to be primarily at the module root in module mode, maybe this could use reconsideration. It would need a proposal. That's something for cmd/go owners to consider, if they haven't already.

CC @matloob, @samthanawalla via owners.

@dmitshur that makes a ton of sense. As I was debugging this I read up on the embed pkg docs and what it states makes perfect sense if the vendor/ is at the root alongside the go.mod file. However, it still included the testdata/**/vendor/** paths locally and via go.work. It only fell apart when the go get of the module was used.

It would be great if we could include vendor in the testdata dir. I fully expect any rouge go.mod files within the testdata dirs to have the existing behavior applied. I've run into similar with a function as as service offering, having to change the go.mod to a go.mod.tmpl to include it in the embed.FS for bootstrapping new functions for our users.

Would love to see testdata/**/vendor/** supported for the embed usecase. This would make it possible to test that part of our system.

I've made a proposal of fixing this in golang/mod#24. I think that disallowing packages like package/name/vendor/foo makes no point as it is not considered vendor direcotry by maincc go command anyway.

I think we may have conflated your bug with an issue that we had wanted to fix earlier ('vendor' being the last element of the path).

We should have a proposal for this. I think we'd want to have sufficient evidence that the benefits gained from doing this are worth the additional complexity that comes from a second change in zipfile behavior.

I think we'd want to have sufficient evidence that the benefits gained from doing [...]

I can literally copy-paste content of #37397 - this is the same issue, but about the full issue - not it special case (vendor being at the end of module path).

additional complexity that comes from a second change in zipfile behavior.

In my proposal (I'm aware it isn't perfect, but this is only an approximation) it is +18 -2, so this "complexity" is not SO large.

I see it this way:

Edit 1:
Well, according to the other docs snippet (mentioned above):

Code below a directory named "vendor" is importable only by code in the directory tree rooted at the parent of "vendor", and only using an import path that omits the prefix up to and including the vendor element.

Two notes could be made:

1. I understend an idea behind this, however, in my opinion taking a bit more "global" approach (in the sense of treating vendor like e.g. internal) would be reasonable to not sabotage use-cases like mine.
2. This will not break compatibility promise anyway.

https://pkg.go.dev/cmd/go#hdr-Vendor_Directories

In my proposal (I'm aware it isn't perfect, but this is only an approximation) it is +18 -2, so this "complexity" is not SO large.

The amount of code introduced isn't a big issue, the issue is that that there would be more Go versions that need to be kept in mind to know what zip file is created.