The types symbol_section, symbol_type, and goos_arch are added. The
tables symbols and symbol_history are added.

These will be used to track to package version when a symbol is added to
package.

For golang/go#37102

Change-Id: I6f06d69f5ef09c4a5e3627878f2ba97ec9fb3704
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/289249
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
TryBot-Result: kokoro <noreply+kokoro@google.com>

The docIsEmpty helper function is added, which indicates if a package
doc is empty.

For golang/go#37102

Change-Id: I633e2a842b8e6324ead0c935e3547f340b4f2540
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/288757
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>

A bug in frontend.ParseVersionsDetails and symbol.CompareAPIVersions is
fixed to handle cases when the version is a Go tag.

For golang/go#37102

Change-Id: Iffc444956baa394cd40fad97096220a3721522ba
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/317016
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>

An integration test is added for these cases:

- A symbol changes across different build contexts
- A symbol is introduced at different versions for different build contexts
- The package symbol for a symbol changes because its parent changes,
  but the name and synopsis are the same

For golang/go#37102

Change-Id: Ieca17042e11b2fd583e2133a3064ab6043a29858
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/316969
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>

Indexes are added to these columns, since they are used in WHERE clauses
in various queries:

- sort_version
- goos
- goarch

For golang/go#37102

Change-Id: I5e4351cbfb06bac76bd8b5ed5e83421b8f615524
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/317309
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>

For golang/go#37102

Change-Id: Icda2856dd6af29267affd075a95974a3c65c0adc
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/317429
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>

For golang/go#37102

Change-Id: Ie3554874b515abda558fb58cbd5f28b814476ac8
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/317490
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>

Previously on the versions page, we were not handling the case when
different identifiers are added for the same type for different build
contexts, and which build context was surfaced was based on chance.

To support this case, it is now possible to show multiple of the same
type at the same version. For example,
https://pkg.go.dev/internal/poll?tab=versions at go1.10 will show:

```
type FD — windows/amd64
+ func (fd *FD) ReadMsg(p []byte, oob []byte) (int, int, int, syscall.Sockaddr, error)
+ func (fd *FD) WriteMsg(p []byte, oob []byte, sa syscall.Sockaddr) (int, int, error)

type FD — darwin/amd64, linux/amd64
+ func (fd *FD) SetBlocking() error
+ func (fd *FD) WriteOnce(p []byte) (int, error)
```

For golang/go#37102

Change-Id: I19e6ef12f1f8f9c412aab7cea2782409eecf29f9
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/317489
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>

Since symbols don't appear in documentation for commands, also don't
show its symbol history.

For golang/go#37102

Change-Id: I09ad461ba12b91fc3bb7e6a314f35616e9668ba2
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/333379
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>

The ReadSymbolHistory is deleted, since the symbol_history table
has been fully populated. When either ExperimentSymbolHistoryMainPage
or ExperimentSymbolHistoryVersionsPage is active, data is now
always read from symbol_history instead of a JOIN on
package_symbols and documentation_symbols.

For golang/go#37102

Change-Id: I8ca0aece399217b804a9a6c55bbfec5efe5b3a18
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/338450
Trust: Julie Qiu <julie@golang.org>
Reviewed-by: Jamal Carvalho <jamal@golang.org>

A script is add which generates the API for a package from its source
code, and writes this data to tests/api/testdata. The script is largely
adapted from cmd/api/goapi.go.

The resulting testdata will be used in following CLs to compare against
the symbol history data on the frontend.

For golang/go#37102

Change-Id: Iae3d9b6f55de94bdd81821f374aaf53276f306b9
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/339452
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>

The tests/api/main.go script is updated to compare the frontend symbol
history data with data in tests/api/testdata.

For golang/go#37102

Change-Id: I5eb5fc367343ce386c7a2d2d7a53c0d3fad9da4f
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/339490
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>

A -all flag is added to the compare script, which compares all of the
package data in tests/api/testdata.

For golang/go#37102

Change-Id: If10f0896fb3eb30e5cb4eb37da59f904062bcce1
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/339492
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>

tests/api/run.sh is added, which runs the symbol history comparison
script in docker.

For golang/go#37102

Change-Id: I11d7f3a61277d6756308f854a78504c3b63ac4fd
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/339395
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>

parseRow is updated to support package paths with hyphens.

For golang/go#37102

Change-Id: I9ba6a9928a8f11779ff025cb68ca0c287506403f
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/340090
Trust: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>

go list sometimes "go: downloading...", when a module is not in the GOMODCACHE.

This will cause Decode to return the error:
"invalid character 'g' looking for beginning of value", which can fixed by rerunning the script.

Instead, just re-run go list when we see that message in the error.

For golang/go#37102

Change-Id: Ic864a4d475eeb594afa7fda90a7b63d073bae898
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/340091
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>