Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/gopls/internal/regtest/workspace: failures with ‘Unmet: completed work "diagnosing initial workspace load"’ #50582

Open
bcmills opened this issue Jan 12, 2022 · 3 comments
Labels
gopls Issues related to the Go language server, gopls. Testing An issue that has been verified to require only test changes, not just a test failure. Tools This label describes issues relating to any tools in the x/tools repository.
Milestone

Comments

@bcmills
Copy link
Contributor

bcmills commented Jan 12, 2022

--- FAIL: TestFilters (27.91s)
    --- FAIL: TestFilters/module_root (27.91s)
        --- FAIL: TestFilters/module_root/experimental (25.12s)
            runner.go:316: waiting on:
                	Unmet: completed work "diagnosing initial workspace load" 1 times
                
                err:context deadline exceeded
                
                state:
                #### log messages (see RPC logs for full text):
                	Info: "2022/01/12 18:58:13 go env for C:\\Users\\gopher\\...
                	Info: "2022/01/12 18:58:31 go/packages.Load\n\tsnapshot=0...
                
                #### diagnostics:
                
                #### outstanding work:
                	Load: 0.00
                	diagnosing initial workspace load: 0.00
                	Setting up workspace: 0.00
                #### completed work:
            runner.go:312: closing editor: Shutdown: context deadline exceeded

greplogs --dashboard -md -l -e 'FAIL: TestFilters'

2022-01-12T19:47:40-d7a4bb4-f005df8/windows-arm64-10

And then there are a whole lot of openbsd-386 failures on the builders affected by #49209, which IMO we can safely ignore:

2022-01-12T01:49:33-d7a4bb4-3d3f5d9/openbsd-386-70-n2d
2022-01-11T18:17:05-d7a4bb4-a20724d/openbsd-386-70-n2d
2022-01-11T01:22:53-52e9527-9de1ac6/openbsd-386-70-n2d
2022-01-10T04:23:42-94bfe68-6df0957/openbsd-386-70-n2d
2022-01-09T18:43:51-94bfe68-2639f2f/openbsd-386-70-n2d
2022-01-08T00:24:25-94bfe68-90860e0/openbsd-386-70-n2d
2022-01-07T17:27:56-94bfe68-9de1ac6/openbsd-386-70-n2d
2022-01-07T06:34:04-351aaa6-11b28e7/openbsd-386-70-n2d
2022-01-07T05:34:58-351aaa6-9de1ac6/openbsd-386-70-n2d
2022-01-07T01:36:17-a1300ed-c1e7c51/openbsd-386-70-n2d
2022-01-06T21:41:55-a1300ed-9de1ac6/openbsd-386-70-n2d
2022-01-06T16:18:51-4754748-dbdf055/openbsd-386-70-n2d
2022-01-06T00:26:47-04daf88-b5bfaf4/openbsd-386-70-n2d
2022-01-05T21:22:03-04daf88-2b39d86/openbsd-386-70-n2d
2022-01-05T09:50:29-15409b5-2c58bb2/openbsd-386-70-n2d
2022-01-05T01:53:39-15409b5-301db3f/openbsd-386-70-n2d
2022-01-04T20:54:25-15409b5-1242f43/openbsd-386-70-n2d
2022-01-03T23:45:12-ee1ca4f-95b240b/openbsd-386-70-n2d
2022-01-02T14:27:43-ee1ca4f-c886143/openbsd-386-70
2021-12-29T20:20:32-ee1ca4f-6178d25/openbsd-386-70-n2d
2021-12-28T19:29:29-ee1ca4f-1242f43/openbsd-386-70-n2d
2021-12-23T20:03:38-8d38310-b357b05/openbsd-386-70-n2d
2021-12-23T18:58:36-8d38310-af3b8cf/openbsd-386-70-n2d
2021-12-22T18:43:55-8d38310-0f3becf/openbsd-386-70-n2d
2021-12-16T11:15:33-8d38310-de690c2/openbsd-386-70-n2d
2021-12-15T23:51:57-c6ae451-6e7c691/openbsd-386-70-n2d
2021-12-15T16:50:15-c6ae451-de690c2/openbsd-386-70-n2d
2021-12-15T00:33:55-27fc764-9d0ca26/openbsd-386-70
2021-12-15T00:33:55-27fc764-9d0ca26/openbsd-386-70-n2d
2021-12-13T22:45:26-27fc764-5b9207f/openbsd-386-70-n2d
2021-12-12T06:14:07-27fc764-9c6e8f6/openbsd-386-70-n2d
2021-12-10T19:26:50-27fc764-766f89b/openbsd-386-70-n2d
2021-12-09T17:16:12-27fc764-78b4518/openbsd-386-70-n2d
2021-12-09T15:47:16-27fc764-de690c2/openbsd-386-70-n2d
2021-12-09T13:07:36-3fca6a0-48d9489/openbsd-386-70-n2d
2021-12-08T23:38:20-3fca6a0-d6c4583/openbsd-386-70-n2d
2021-12-08T18:30:00-3fca6a0-61011de/openbsd-386-70-n2d
2021-12-08T16:15:56-3fca6a0-61317ef/openbsd-386-70-n2d
2021-12-08T04:14:00-fd2bfb7-a19e72c/openbsd-386-70-n2d
2021-12-07T20:13:17-d3358c1-61317ef/openbsd-386-70-n2d
2021-12-07T03:44:02-feb39d0-dc65c48/openbsd-386-70-n2d
2021-12-06T16:11:25-feb39d0-0f2d0d0/openbsd-386-70-n2d
2021-12-04T04:50:55-c882a49-549cfef/openbsd-386-70-n2d
2021-12-03T18:55:11-c882a49-c4a8550/openbsd-386-70-n2d
2021-12-03T18:55:11-c882a49-0f2d0d0/openbsd-386-70-n2d
2021-12-03T01:09:21-e212aff-0985990/openbsd-386-70-n2d
2021-12-02T20:02:34-e212aff-0f2d0d0/openbsd-386-70-n2d
2021-12-02T16:48:07-e212aff-36be0be/openbsd-386-70-n2d
2021-12-01T19:09:57-d99d6fa-0103fd2/openbsd-386-70-n2d
2021-12-01T15:05:46-615f9a6-b7651e5/openbsd-386-70-n2d
2021-11-30T22:42:17-1fd30d2-b7651e5/openbsd-386-70-n2d
2021-11-30T22:42:17-1fd30d2-7ccbcc9/openbsd-386-70-n2d
2021-11-30T18:09:02-2c9b078-931d80e/openbsd-386-70-n2d
2021-11-30T16:43:31-2c9b078-682435d/openbsd-386-70-n2d
2021-11-29T20:12:13-6e52f51-b7651e5/openbsd-386-70-n2d
2021-11-25T00:07:28-cb80a01-f7e34e7/openbsd-386-70

CC @findleyr

@gopherbot gopherbot added Tools This label describes issues relating to any tools in the x/tools repository. gopls Issues related to the Go language server, gopls. labels Jan 12, 2022
@gopherbot gopherbot added this to the Unreleased milestone Jan 12, 2022
@suzmue suzmue modified the milestones: Unreleased, gopls/on-deck Jan 13, 2022
@bcmills
Copy link
Contributor Author

bcmills commented Jan 18, 2022

This appears to be more builder-specific than test-specific; maybe it's a bug in the test-runner infrastructure?

--- FAIL: TestClearAnalysisDiagnostics (35.55s)
    --- FAIL: TestClearAnalysisDiagnostics/experimental (20.07s)
        runner.go:316: waiting on:
            	Unmet: completed work "diagnosing initial workspace load" 1 times
            
            err:context deadline exceeded
            
            state:
            #### log messages (see RPC logs for full text):
            	Info: "2022/01/18 18:08:15 go env for C:\\Users\\gopher\\...
            
            #### diagnostics:
            
            #### outstanding work:
            	diagnosing initial workspace load: 0.00
            	Setting up workspace: 0.00
            	Load: 0.00
            #### completed work:
        runner.go:312: closing editor: Shutdown: io: read/write on closed pipe
FAIL
FAIL	golang.org/x/tools/gopls/internal/regtest/workspace	138.379s

greplogs --dashboard -md -l -e 'Unmet: completed work "diagnosing initial workspace load"' --since=2022-01-12

2022-01-18T18:56:51-f29bdf1-75bcdd5/windows-arm64-10

@bcmills bcmills changed the title x/tools/gopls/internal/regtest/workspace: TestFilters failure during initial workspace load x/tools/gopls/internal/regtest/workspace: failures with ‘Unmet: completed work "diagnosing initial workspace load"]0 Jan 18, 2022
@bcmills bcmills changed the title x/tools/gopls/internal/regtest/workspace: failures with ‘Unmet: completed work "diagnosing initial workspace load"]0 x/tools/gopls/internal/regtest/workspace: failures with ‘Unmet: completed work "diagnosing initial workspace load"’ Jan 18, 2022
@findleyr
Copy link
Contributor

This means that gopls' initial workspace load is timing out on these builders:

Consider the following excerpt from the logs:

[Trace - 18:08:14.808 PM] Received notification '$/progress'.
Params: {"token":"6426100070888298971","value":{"kind":"begin","title":"Load","message":"Loading query=[builtin mod.com/...]"}}

...

[Trace - 18:08:32.828 PM] Received notification '$/progress'.
Params: {"token":"7981306761429961588","value":{"kind":"end","message":"Finished loading packages."}}

I think a reasonable next step is to add additional instrumentation around loading: how long did go list take vs other phases in loading.

@gopherbot
Copy link

Change https://golang.org/cl/380497 mentions this issue: internal/lsp/regtest: eliminate arbitrary timeouts

gopherbot pushed a commit to golang/tools that referenced this issue Jan 25, 2022
We care that gopls operations complete within a reasonable time.
However, what is “reasonable” depends strongly on the specifics of the
user and the hardware they are running on: a timeout that would be
perfectly reasonable on a high-powered user workstation with little
other load may be far too short on an overloaded and/or underpowered
CI builder.

This change adjusts the regtest runner to use the test deadline
instead of an arbitrary, flag-defined timeout; we expect the user or
system running the test to scale the test timeout appropriately to the
specific platform and system load.

When the testing package gains support for per-test timeouts
(golang/go#48157), this approach will automatically apply those
timeouts too.

If we decide that we also want to test specific performance and/or
latency targets, we can set up specific configurations for that (as
either aggressive per-test timeouts or benchmarks) in a followup
change.

For golang/go#50582

Change-Id: I1ab11b2049effb097aa620046fe11609269f91c4
Reviewed-on: https://go-review.googlesource.com/c/tools/+/380497
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
@findleyr findleyr added Testing An issue that has been verified to require only test changes, not just a test failure. and removed gopls/testing labels May 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
gopls Issues related to the Go language server, gopls. Testing An issue that has been verified to require only test changes, not just a test failure. Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Development

No branches or pull requests

4 participants