Significantly refactor the gopls benchmarks to turn them into proper
benchmarks, eliminate the need for passing flags, and allow running them
on external gopls processes so that they may be used to test older gopls
versions.

Doing this required decoupling the benchmarks themselves from the
regtest.Runner. Instead, they just create their own regtest.Env to use
for scripting operations. In order to facilitate this, I tried to
redefine Env as a convenience wrapper around other primitives.

By using a separate environment setup for benchmarks, I was able to
eliminate a lot of regtest.Options that existed only to prevent the
regtest runner from adding instrumentation that would affect
benchmarking. This also helped clean up Runner.Run somewhat, though it
is still too complicated.

Also eliminate the unused AnyDiagnosticAtCurrentVersion, and make a few
other TODOs about future cleanup.

For golang/go#53992
For golang/go#53538

Change-Id: Idbf923178d4256900c3c05bc8999c0c9839a3c07
Reviewed-on: https://go-review.googlesource.com/c/tools/+/419988
gopls-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Peter Weinberger <pjw@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

The go command can't always resolve arbitrary commits as versions, and
some commits have an x/tools replace directive that must be honored, so
use 'git clone' with an arbitrary commit ref, instead of 'go install'
with a Go module version, to install gopls.

Also rename BenchmarkIWL to BenchmarkInitialWorkspaceLoad.

For golang/go#53538

Change-Id: Ic3a08e4c023e0292f6595cc5b2ab59954d073546
Reviewed-on: https://go-review.googlesource.com/c/tools/+/422908
Run-TryBot: Robert Findley <rfindley@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>

For golang/go#53538

Change-Id: I0eb1d2f00ca1977bdd909e0a4bfe4367d388ba16
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/453502
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Dylan Le <dungtuanle@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

The tools repo will soon have benchmark results. Split the dashboard by
repository (default 'go'), so that results aren't mixed together.

This is pretty straightforward. The Influx queries filter by repository
(backfilling any points without the 'repository' field as 'go') via a
URL parameter. The frontend adds a drop-down box to select the
repository.

The "All benchmarks" and "Regressions first" links unfortunately don't
maintain the repository setting (or duration/end time). That requires a
bit more JS magic.

For golang/go#53538.

Change-Id: I1d46a23558a37feef4c8c89cfb9c3927304ebb60
Reviewed-on: https://go-review.googlesource.com/c/build/+/458996
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Dylan Le <dungtuanle@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

For golang/go#53538.

Change-Id: Ic68861b65a7b5cd259ab6c8c82a10a08323f1a6d
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/459096
Auto-Submit: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>

AllComparisonSeries shouldn't panic on bad input, it should return an
error.

For golang/go#53538.

Change-Id: I988b546ea3ad5bb9026956b3aa4feec809915b73
Reviewed-on: https://go-review.googlesource.com/c/perf/+/459177
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

First and foremost, we must ingest uploads on alternative branches.
Right now ingest only branch master.

On the frontend, add a dropdown for branch, very similar to the
repository dropdown. In the future, we should probably fetch a list of
branches rather than hard-coding them.

Note that the branch here is the Go branch that the perf builder
triggered against, which is a bit confusing for subrepos, but I've added
a note to try to explain.

For golang/go#53538.

Change-Id: Ib5b74b9e5d7b67ce2b04bc2bb22e3521dffaee36
Reviewed-on: https://go-review.googlesource.com/c/build/+/459155
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

…441d3

This fixes AllComparisonSeries to return errors rather than panicking.

For golang/go#53538.

Change-Id: I872a1a77c8aedc3827dec650c21afa7c68b540af
Reviewed-on: https://go-review.googlesource.com/c/build/+/459195
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>

I had initially added known issues fairly aggressively in order to use
them to reduce noise in 'greplogs -triage'. Now that we are using
'watchflakes' for triage, that noise reduction is no longer important
(the failures are already clustered to their respective known issues),
and having greyed-out cells on the dashboard makes new regressions too
easy to miss.

Concretely:

- golang/go#42212 is mostly specific to x/net at this point (as
  golang/go#57841)

- There have been no failures matching golang/go#51001 since October.

- golang/go#52724 has been so rare lately that we hadn't yet added a
  'watchflakes' pattern for it.

- There have been no failures matching golang/go#51443 since May.

- There have been no failures matching golang/go#53116 or
  golang/go#53093 since I enabled 'watchflakes' for the builder in
  December.

- The linux-amd64-perf builder seems to be passing consistently for
  x/benchmarks and x/tools, so there is no need to refer to
  golang/go#53538 to explain failures on it.

Change-Id: Ia16db2a23e5fa037a299f1f56fb26f1cf84521e1
Reviewed-on: https://go-review.googlesource.com/c/build/+/465156
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

Extract benchmark state into a new repo type, so that we may run
benchmarks in multiple shared workspaces. Also, add missing cleanup
code.

Additionally, simplify to always run gopls in a separate process. This
means that the normal test profiling flags won't be useful, so add
support for threading through profiling flags to the external gopls
process.

For golang/go#53538

Change-Id: Ib9ab5920dc59f102c62b53b761379dd8ca2d7141
Reviewed-on: https://go-review.googlesource.com/c/tools/+/468940
TryBot-Result: Gopher Robot <gobot@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>

Extend existing benchmarks to run in more repos, choosing an initial set
with different features that may affect performance. Some of these take
too long if run in the same batch, so guard with -short.

Several benchmarks need a location within the codebase. For these, I
have chosen somewhat arbitrarily, but tried to select locations within
the core of the codebase. We can always adjust in the future.

Additionally:
 - fix the fake file polling to scale to larger codebases, by avoiding
   reading files if it isn't necessary
 - fix a polling bug related to symlinks
 - fix a couple places the benchmarks weren't cleaning up after
   themselves correctly
 - fix a bug where the gopls install used the wrong directory

For golang/go#53538

Change-Id: I559031cb068086cd5ec19e36bb12da396194933c
Reviewed-on: https://go-review.googlesource.com/c/tools/+/469355
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>