I think the actual sort is pretty far from cmd/go; I think it's the sort done by filepath.Walk.

But I'm not sure I believe that the sort takes 4 seconds. As you suggest, the sort time is likely negligible. It's more likely the fact that the go tool collects all the packages together, and then lists them.

just grepping around I found a bunch of calls to sort in internal/load which the list subcommand uses, but didn't think about filepath.Walk. It appears to use that as well.

I'm sure it would be a large change for a niche use case but figured it's better to file than not.

@jimmyfrasche it would be useful if you could get some metrics for large package sets (such as std cmd). For example, time all the sort.Sort calls - how much is that compared to the overall run-time of go list?

I also don't know if it would be possible for go list to print its output incrementally. Surely, for something as trivial as just printing the package paths, it's technically possible. But what about {{ .Deps }}, which should print all the transitive dependencies? It might just be impossible to add a sane "incremental" mode to go list.

/cc @rsc who has designed much of this, and who I think wants to work on go list in 1.11.

@mvdan I'll try to gather some metrics later.

A fast path for just printing import paths would be helpful. That's the most common use case, at least for me, though when used as such it's usually for smaller package sets, such as to get the import path of the current working directory or something like github.com/jimmyfrasche/project/sub/....

The particular use case inspiring the bug involves invoking go list -e -json all to load a snapshot of the entire workspace, but I don't need it in any particular order. That's always going to be somewhat slow and it's questionable if it's worth optimizing.

As far as incremental output, the dependency graph is acyclic so theoretically an incremental mode could output leaves immediately and then any packages that only depend on leaf packages and so on up the tree.

Maybe it would suffice to be smarter about caching the results (and invalidating the cache)? Warm up would still be slow but it would still tend to be quite fast in most cases.

It seems that a lot of go subcommands make use of the internal/load package so optimizing it might make a lot of things faster whenever large package sets are involved:

$ # as of go1.9.4 linux/amd64, 
$ # since rdeps doesn't like working on my copy of tip
$ rdeps cmd/go/internal/load
cmd/go/internal/generate
cmd/go/internal/envcmd
cmd/go/internal/fix
cmd/go/internal/fmtcmd
cmd/go/internal/run
cmd/go/internal/list
cmd/go/internal/clean
cmd/go/internal/get
cmd/go/internal/vet
cmd/go/internal/test
cmd/go/internal/work

Sorry but I don't think we are going to attempt to stream the results into go list.
Go list is complex enough as it is.