What package imports github.com/influxdb/influxdb/influxql ?

@adg it's pulled in from vendor/github.com/rcrowley/go-metrics/influxdb/influxdb.go, which is not imported by the application. Technically I guess I could remove the file from vendor/, but I still think the behavior of ./... should not consider packages in vendor/ unless they are transitively imported by packages outside.

That would require special-casing vendor in package pattern matching, which is something we decided not to do (at this point anyway).

When I run go test ./..., it runs the tests for all of the packages in vendor/ as well as my project. Are you suggesting that this is the desired behavior (for now), @adg?

@technosophos +1 on that problem.

@adg IMO that makes the vendor experiment nearly useless (for me anyway). I'll either need to make godep much more intelligent wrt package handling and/or try to discern what to build through additional package introspection. It's also just a bad user experience IMO.

@technosophos yes, that is the correct behavior as of now.

@freeformz there is an easy work around. Instead of using ./... you can do:

go install $(go list ./... | grep -v /vendor/)

or use go test if you want to run tests.

On the testing front, I think it is valuable to know that your dependencies' tests pass.

Yeah, I don't find any of this particularly disconcerting. I just wanted to verify that it was the intended behavior.

@adg Heh, thanks. I should have thought of that. :-(

Are there any plans to special case vendor/ at all, say in the 1.6 timeframe?

Possibly. That's what this issue is now tracking. We're going to gather people's experience with the 1.5 vendoring experiment before making such plans.

@freeformz After looking at this issue carefully, I would say the current behavior would be my desired behavior. While I understand Godep copies over the entire repository to vendor, I don't think that is what I would encourage. I think copying over each package as needed by the vendor tool is where the analysis work needs to be. At least that is what vendor will do.

@kardianos The issue really has nothing to do with Godep, even though Godep's current behavior triggers it.

A couple reasons I would prefer go test ./... not include things in vendor:

When I choose to take on a dependency, I've already verified it serves my need. That includes verifying automated tests are run by that dependency's overall project, running them myself, or similar. With things in vendor likely changing infrequently relative to my project it doesn't feel right to run the same tests for unchanged things every time. This is the same logic I use for feeling comfortable using, say, net/http in my project without running its tests every time.

Second, It's possible a dependency's tests will never build, work, or be relevant in my particular environment. This could be due to the way the tests are written, differences in developing vs deploying platform, etc. I develop projects on my OS X laptop that use libcontainer that would probably be good examples for this.

@dpiddy Tests that are specific to an environment shouldn't run on a different environment anyway. If they try to run file a bug against the package. I would also argue when copying packages over the vendor tool user should decide if they want to bring over the test files or not

The case where my reasoning falls down is where people choose to use a vcs features like sub-modules.

(Not directly trying to argue here, just state other sides.)

@kardianos good point, tools could do that.

Re the original issue of go install ./... installing things from vendor, that sounds really confusing and surprising. "Why did $GOPATH/bin/x change unexpectedly? Oh, I ran go install ./... in some completely unrelated project that vendors an old version of x three dependencies deep."

Vendoring tools could also skip main packages I suppose.

@dpiddy Of note, I've had at least one person explicitly want to vendor "main" packages. What the kardianos/vendor tool does it to categorize packages by a "status":

Missing (packages referenced but not found)
Standard (in standard library)
Local (package is under "project" and not rewritten or in vendor dir)
External (package is outside of "project" but referenced)
Internal (package is rewritten under "internal" but still a vendor package)
Unused (package is rewritten under "internal" or in vendor folder but unused)
Program (package is a "main" package but rewritten or in vendor folder)
Vendor (package is in vendor folder)

These are two overlapping issues I hope to address soon (location and status), but that's the general idea. Then my tool can work on package status rather then just paths. For instance, if vendor added an install command, you could run "vendor install -status local".

I'm not convinced that adding a "status" concept to the go tool would ever be a desirable thing. "go" already supports "std". The syntax "go install ./...!local" could be interesting I think.

This has come up repeatedly, but it seems like a bad idea.
Wildcards are wildcards.
Vendoring is vendoring.
They're completely separate things.
One should not influence the other.
If there is junk in a vendor/ subtree,
I think that's an argument for deleting, not for ignoring.

I missed Andrew's comment earlier about using this issue to track feedback. That's fine, but if so it should not be marked as milestone Go 1.5. Reopening and reclassifying.

@rsc Thanks for re-opening. I hope other's provide additional feedback ... My general thoughts on your last comments are: I "technically" agree with you, but it makes for a poor user experience.

Having go test ./... run dependencies' tests seems attractive to me in theory, but I just tried it in practice and it was a pain in the ass. Mostly because it means you need to pull in the transitive closure of their dependencies, which is large and which godep doesn't seem to support well.

@jacobsa Sure, but don't confuse the issue of "go test ./..." with a tool, godep. There (will be) many tools. There will be some that analyze dependencies well.

This isn't just about testing. Here are my thoughts to date....

When vendor/ is the name of the vendoring directory and a package spec of ./... matches everything in that directory as well...

The issues I have with this are:

1. Most people I've talked to don't expect the package spec ./... to match stuff in vendor/. This indicates to me that it's counter intuitive. I will likely have to do a lot more education about packages / paths / etc. This isn't really a bad thing, but most people I talk to are already confused and/or frustrated by the way $GOPATH works. I don't want to add to that confusion.
2. It's impossible to test only the packages that I'm writing with a repo level ./.... I care about testing the packages I'm writing more regularly than testing the stuff I've vendored. I'll test the stuff I vendor when I originally vendor it and/or when I upgrade the vendored deps. IMO/E more people follow this workflow than the test everything all the time workflow. I'm worried that this will end up leading people to encode test Makefile or similar targets to avoid having to remember / type the go test $(go list ./... | grep -v vendor/) stuff. I thought we wanted to avoid that?
3. Additionally some package's tests require additional setup which I may not have on my local machine / environment.
4. I will require additional meta data about what to install when someone pushes code to me. Right now I can assume that go install ./... will install all of the packages in the pushed repo. Godeps handles this case just fine by recording the package spec(s) used during godep save, but I can tell you most people just use ./...
5. Similarly for projects with multiple main packages for commands, it will require additional documentation where it's now possible to simply say go get github.com/my/proj/.... This is probably pretty minor though.
6. As per @davecheney's comment: Not everyone is using Go 1.5. vendor/ will cause issues for users of Go versions < 1.5.

Some, but not all of the above stuff can be handled by vendoring tools. IMO if we're going to cause this much disruption then Go itself should provide a vendoring tool such as govendor/godeps/something new/whatever as part of the standard tooling (i.e. go vendor).

I second @freeformz's points FWIW..

@minux

And If you just want to run all the tests, I think knowing that the vendored packages also works correctly is also desirable.

If I add dependency X that is only a sub-package of a larger package Y, the whole thing ends up in the vendored folder, even if I only need the subpackage and its dependencies. If I test the whole project with go test ./..., it tests the whole package Y. I don't need to test all of Y, and worse, it will fail if I did not add its recursive dependencies, which I don't need for package X.

You may ask why I add the all of Y if I only need X. The reason is that I use git submodules to add the repo, and go get behaves the same way.

I share all of @freeformz's concerns. I think it is telling that at least godep has always used an ignored _workspace path, even when using godep save -r which made saved dependencies valid import paths as the vendoring experiment does.

@bradfitz that is what I figured. I fear if it takes a major release to change the experiment's behavior enough momentum (whether because it's good or not) will be built with the current behavior that it will be unlikely to ever change.

My main concern, before I can start to think about others, is that of the import paths. As I understand example.com/foo/bar/vendor/example.com/foo/baz is not a valid import path. example.com/foo/baz is a valid import path of a Go package that happens to be vendored inside example.com/foo/bar.

Are other people okay with the idea that go list all will print out a bunch of invalid import paths (in addition to valid ones)? How should other tools deal with these invalid import paths?

I thought the point of vendoring was to put a copy of the original package (e.g., example.com/foo/baz) inside your VCS tree, but not change it in any way. No import path rewriting required - the vendored package still has the same original import path. You should be able to vendor/not vendor any package as you wish. It seems to do that but also change import paths of the packages being vendored.

I've been dealing with the repercussions of the vendoring experiment for a few weeks now. I like having fine-grained control over my dependencies, and it’s awesome to be able to commit the vendor source along with my project source without having to change all the vendored import paths.

I think I voice the same concern as most of the non-googlers here, which is that I didn’t expect to have golint, go vet, and go test blow up in my face when I moved everything to the vendor directory. It’s frustrating to have to contort my workflow to accommodate external code that didn’t previously require it.

However, I wouldn’t have known about many of these problems with the third-party libraries had I not moved them into the vendor directory. It’s been a pain to get everything to play nice, and I understand that not everyone has the luxury of being able to fix other-people’s-code, but the new methodology incentivizes quality code… which is one reason I believe Go is as successful as it is today.

If I compare the two solutions from a long-term perspective, I see two very different outcomes. Allowing people to ignore bad code results in a complex solution whereby some code is whitelisted, some is blacklisted, the lists require regular maintenance, and you have no guarantee of a minimum level of quality. The other solution is to develop good code, fork bad code, and take the penalty of due diligence up front.

I’m not saying either solution is right or wrong, but since this issue is still open for feedback, I’m of the opinion that the short-term pain of broken libraries will eventually subside as the authors feel the pain and/or new authors emerge. I believe that the long-term result will be a better foundation from which to build future solutions.

I was asked to provide my opinion here, so here it is.

I agree with @rsc's comment in #11659 (comment).

Further, it seems renaming the directory to _vendor would make it harder to operate on the vendored packages. For example, it's useful (at least to me) to be able to run go list ./vendor/... and see everything that's been vendored.

Personally, I think the current behavior is ok. It hasn't really been a problem for me (though I understand why some find it inconvenient). For example, when I run interactive commands to test or vet packages, I type go test or go test ./foo ./bar or maybe go test ./foo/... since I generally want to test only one or two packages at a time. Our automated testing system tests all our packages, using a command like go test $(go list ./...|grep -v vendor). This would be inconvenient to type by hand, but there is more than one way to make it convenient. Complicating the behavior of the go command probably isn't the best way.

Thanks for all of your input. I think everything has been covered here that could be covered.

I'm satisfied with the outcome, although I still think it ignores usability a bit, so I'm going to close this issue (since I opened it).

That doesn't help with the issue where go list all outputs invalid import paths (sorry, I'm repeating myself).

But I said go list all specifically, not just go list.

I have multiple tools that use the output of go list all to navigate and open Go packages. For example, if I want to go to github.com/russross/blackfriday, I expect there to be just one. With vendoring as is, there may be 10, 20 or 30+ matches for blackfriday in various vendor folders.

All tools that prefer to preserve previous behavior of not including vendored package copies (e.g., like they did not include the vendored packages in Godeps/_workspace subfolders) need to be updated to perform go list all and then try to filter out vendored copies. Even if GO15VENDOREXPERIMENT is not enabled.

@shurcooL In hopes of being helpful, you might try to migrate your tools to use something else. For instance you could try http://godoc.org/github.com/kardianos/govendor/context create a new context then call ctx.Status(), then filter all the returned items by their "status" which includes if they are in a vendor directory or not. If that doesn't quite fit your need, but is close, feel free to open an issue.

I have been doing research and setting up a continuous integration & testing pipeline for Go during most of the day, on different tools, and this is an issue that really bothers me.

I love Go for it's minimalist and clean approach. In line with that, I use the clean architecture approach (or something close to it) for most of my projects. This means I will usually have four subpackages, with the main package in the root folder. Sometimes, there are some small additional tools in the cmd folder as well. I was under the impression that this is a configuration that most projects use.

I now have to use a dirty workaround to get CI to build and test properly, all due to the fact that ./.. includes folders in vendor. This is counter-intuitive and not well thought-out at all. Which is contrary to what I've come to expect and appreciate about Go. The default case should not require any workaround.

There should be a coherent approach where all go tools exclude vendor when using ./... if the GO15VENDOREXPERIMENT environment variable is set to 1. That would be consistent. As someone mentioned before, you can still do "cd vendor && go test ./..." if you want to include the subpackages. But you will have to be explicit about it, which is fine since it constitutes a non-default workflow imo.

If the go tool deals with vendoring using the environment flag (and by default in Go 1.6), then it should also deal with the resulting consequences and solve them in a coherent way. Let's keep it simple, folks.

For now, I'm forced to switch back to godep to keep a clean non-hacky script in my repository.

I think there are two cases:

1. A package in a vendor tree that is actually used (directly or
   indirectly) by the code outside the vendor tree. In this case I think you
   would want to test the package, just as you test all your other
   dependencies. If code you depend on is broken, you want to find out in the
   most precise test possible, not wait until a test of the main program fails
   mysteriously.
2. A package in a vendor tree that is not actually used (directly or
   indirectly) by the code outside the vendor tree. In this case, obviously no
   one cares and the test need not run. But in this case I don't understand
   why tools are bringing that package into the vendor tree in the first
   place. I would argue that, for many reasons, dead (unused) code should not
   be copied into a vendor tree. Code that is in your vendor tree has to be
   maintained, updated, and so on. If it's not needed, it should just be
   omitted.

I think it's a mistake to skip the test in (1). And I think (2) indicates a
problem in the vendoring process.

The counter-argument to this is that you can't omit individual packages
when you're using git submodules. But when you're using git submodules I
think you're tying yourself to the target repository quite a bit more and
so it may still be appropriate to test the entire thing.

@rsc I disagree even with your first point. If you vendor your dependencies, you will usually pin a specific version, so it is nonsense to run the tests each and every time you push a commit and your CI kicks in. You should test your dependencies once, manually, each time you update them. That's completely besides the point of automatic testing of your own commits and pull requests.

Agree with @awishformore, similar to what I said in #11659 (comment).

How many projects that use net/http run those tests every time? Would that be reasonable? I don't think so, because presumably you've verified that the Go release you're using is good and you know net/http isn't changing out from under you.

I feel things in vendor/ can be treated similarly: on add/change, verify it works for you, then be sure the vendored code doesn't change.

As others have noted, there is an easy workaround if you just want to test
everything but vendor:

go test $(go list ./... | grep -v /vendor/)

Depending on your configuration, it may be appropriate or worthwhile to use
that in your CI system. My point is that it may also be appropriate or
worthwhile to include those tests, so it is not appropriate for ./... to
categorically exclude certain directories. Different people use vendor
differently. Some people do make local changes, or do want to include it in
"go vet ./..." or other commands.

I also still think that for the specific case of testing, if we could solve
test caching then that would help considerably.

@rsc Please don't qualify it as an "easy" workaround. It's a dirty workaround for something that should work out-of-the-box. Why? Because someone decided that the Go tool should deal with vendoring, and since it does, the default way should correspond to the most intuitive and common approach. The workaround should be required if you do something out of the ordinary, like testing dependencies on each build or, like you mention, customizing dependencies. In that case, a workaround (i.e. cd vendor && go test ./...) seems perfectly fine. I don't understand why you argue to make the default more annoying in favour of some edge cases. One of the most wonderful things about Go is that defaults usually work in intuitive ways and hardly ever annoy me.

You should test your dependencies once, manually, each time you update them.

It's more convenient and simpler (for both the user and for the go tool's implementation) to test them every time you run your own tests. I'd love to know, concretely, why some folks are so resistant to this idea. Is it because running more tests is slower? (If so, see @rsc's last comment above.)

It's a dirty workaround for something that should work out-of-the-box.

From my point of view, it does work out of the box. That's because I like to run the tests for my dependencies. Of course, anyone is free to prefer otherwise, but that preference doesn't necessarily imply that the go tool's design is wrong.

That's because I like to run the tests for my dependencies.

In case it's helpful, here's an easy workaround if you want to test dependencies:

go test $(go list -f '{{join .Deps "\n"}}' ./...)

@kr build time is certainly a factor with current Go versions. I also gave my reasons in #11659 (comment) and other comments.

Perhaps my mental model of what vendor/ is is just wrong. I consider it to be an extension of the stdlib for my project. If that is true, should I be running tests for everything I've used from the stdlib every time I test my project?

Of course I'm forgetting that an easy way to exclude vendored tests is to do just that: exclude them when vendoring things. At least godep and govendor already support this. Same for unwanted main packages.

Curious to see how things like go install github.com/foo/bar/... play out once 1.6 is released.

The fact that everyone is proposing workarounds just points to the fact that this should be fixed. Renaming it to _vendor just works. If you want to test or install stuff from your vendor directory, you can by just cd'ing into that directory.... but if you don't want to, then you can go whatever ./... just like you do today, and it'll work just like it does today. Making it "vendor" breaks things. I can't imagine anyone wanting that to be the default.

Keep go simple. Make the vendor directory _vendor so that tools skip it. I shouldn't have to use some annoyingly arcane command to just build and test my code.

"vendor" is not being renamed. That ship has long sailed.

I think everything that can be said on this issue has been said in the many comments above. I think it's time we move this to other forums (for general discussion) or new bugs (for specific new bugs that aren't just a dup of this one). Ultimately this bug is a policy/behavior question and not actually a bug report.