Previously we used os.SameFile, but that is needlessly expensive when
we expect the original (user-provided) paths to be preserved during
the walk: the user-provided list of ignored directories is relative.

Because 'go list' does not traverse symlinks, we do not expect a
user's source roots to contain symlinks for Go package source either,
and in the rare even that the tree contains a directory that resides
within a non-ignored subdirectory symlink, the user can explicitly
include all of the relative paths by which the directory may be
encountered.

This reduces benchmark latency by ~7.5% compared to CL 508506, bringing
the overall latency to +~6% over the previous approach using
internal/fastwalk.

	~/x/tools$ benchstat cl508506.txt cl508507.txt
	goos: linux
	goarch: amd64
	pkg: golang.org/x/tools/internal/imports
	cpu: AMD EPYC 7B12
	                │ cl508506.txt │            cl508507.txt            │
	                │    sec/op    │   sec/op     vs base               │
	ScanModCache-24    264.4m ± 1%   244.6m ± 1%  -7.49% (p=0.000 n=10)

	~/x/tools$ benchstat cl508505.txt cl508507.txt
	goos: linux
	goarch: amd64
	pkg: golang.org/x/tools/internal/imports
	cpu: AMD EPYC 7B12
	                │ cl508505.txt │            cl508507.txt            │
	                │    sec/op    │   sec/op     vs base               │
	ScanModCache-24    231.0m ± 1%   244.6m ± 1%  +5.90% (p=0.000 n=10)

For golang/go#58035.

Change-Id: I937faf7793b8fad10a88b2fdc21fa4e4001c7246
Reviewed-on: https://go-review.googlesource.com/c/tools/+/508507
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

….walk

filepath.WalkDir may invoke its callback with a nil DirEntry and
non-nil error — a detail I missed when converting from fastwalk.Walk
to filepath.WalkDir in CL 508506. Even though gopathwalk.Walk ignores
filesystem errors, its callback needs to check for that one.

Looking into this bug, I also realized that the walk callback used to
return only SkipDir or ErrTraverseLink. In retrospect, that makes
sense given that Walk and WalkDir don't return errors, so I have
documented that property and fixed a few places where it no longer
held.

To help to simplify the code enough to rule out other similar errors,
we also inline walker.shouldTraverse and simplify the case for
traversing symlinks. This results in slight changes in the visited
package directories when a root contains cyclic symlink paths,
but since we never expect symlink cycles to occur in the first place
I don't expect that the changes will affect any users in practice.

Fixes golang/go#58054.
Updates golang/go#58035.

Change-Id: Ie8cf1e4b5186b0d7b4650305bfc863330541f080
Reviewed-on: https://go-review.googlesource.com/c/tools/+/539480
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>