Skip to content

x/tools/go/packages: race detected during execution of various tests #36605

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
dmitshur opened this issue Jan 16, 2020 · 15 comments
Closed

x/tools/go/packages: race detected during execution of various tests #36605

dmitshur opened this issue Jan 16, 2020 · 15 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Tools This label describes issues relating to any tools in the x/tools repository.
Milestone
Unreleased

Comments

@dmitshur
Copy link
Contributor

dmitshur commented Jan 16, 2020 •
edited
Loading

From https://build.golang.org/?repo=golang.org%2fx%2ftools, there are a number of instances where the freebsd-amd64-race builder reported a failure. Many of them failed due to data races:


==================
WARNING: DATA RACE
Write at 0x00c001b41198 by goroutine 632:
  runtime.slicecopy()
      /tmp/workdir/go/src/runtime/slice.go:197 +0x0
  math/big.(*Rat).Float64()
      /tmp/workdir/go/src/math/big/nat.go:95 +0x1e1
  go/constant.Float64Val()
      /tmp/workdir/go/src/go/constant/value.go:554 +0x33c
  go/types.roundFloat64()
      /tmp/workdir/go/src/go/types/expr.go:175 +0x46
  go/types.representableConst()
      /tmp/workdir/go/src/go/types/expr.go:279 +0x89d
  go/types.(*Checker).representable()
      /tmp/workdir/go/src/go/types/expr.go:335 +0xb7
  go/types.(*Checker).convertUntyped()
      /tmp/workdir/go/src/go/types/expr.go:517 +0xe35
  go/types.(*Checker).binary()
      /tmp/workdir/go/src/go/types/expr.go:797 +0x1ba
  go/types.(*Checker).exprInternal()
      /tmp/workdir/go/src/go/types/expr.go:1504 +0x2d6f
  go/types.(*Checker).rawExpr()
      /tmp/workdir/go/src/go/types/expr.go:983 +0x91
  go/types.(*Checker).multiExpr()
      /tmp/workdir/go/src/go/types/expr.go:1600 +0x68
  go/types.(*Checker).expr()
      /tmp/workdir/go/src/go/types/expr.go:1594 +0x56
  go/types.(*Checker).binary()
      /tmp/workdir/go/src/go/types/expr.go:780 +0xd0
  go/types.(*Checker).exprInternal()
      /tmp/workdir/go/src/go/types/expr.go:1504 +0x2d6f
  go/types.(*Checker).rawExpr()
      /tmp/workdir/go/src/go/types/expr.go:983 +0x91
  go/types.(*Checker).multiExpr()
      /tmp/workdir/go/src/go/types/expr.go:1600 +0x68
  go/types.(*Checker).expr()
      /tmp/workdir/go/src/go/types/expr.go:1594 +0x56
  go/types.(*Checker).stmt()
      /tmp/workdir/go/src/go/types/stmt.go:492 +0x6a90
  go/types.(*Checker).stmtList()
      /tmp/workdir/go/src/go/types/stmt.go:120 +0xee
  go/types.(*Checker).funcBody()
      /tmp/workdir/go/src/go/types/stmt.go:42 +0x390
  go/types.(*Checker).funcDecl.func1()
      /tmp/workdir/go/src/go/types/decl.go:561 +0xee
  go/types.(*Checker).processDelayed()
      /tmp/workdir/go/src/go/types/resolver.go:616 +0x59
  go/types.(*Checker).checkFiles()
      /tmp/workdir/go/src/go/types/check.go:256 +0xf1
  golang.org/x/tools/go/packages.(*loader).loadPackage()
      /tmp/workdir/go/src/go/types/check.go:245 +0xd22
  golang.org/x/tools/go/packages.(*loader).loadRecursive.func1()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:685 +0x258
  sync.(*Once).doSlow()
      /tmp/workdir/go/src/sync/once.go:66 +0x100
  sync.(*Once).Do()
      /tmp/workdir/go/src/sync/once.go:57 +0x68
  golang.org/x/tools/go/packages.(*loader).loadRecursive()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:673 +0x74
  golang.org/x/tools/go/packages.(*loader).refine.func2()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:619 +0x42

Previous write at 0x00c001b41198 by goroutine 1217:
  runtime.slicecopy()
      /tmp/workdir/go/src/runtime/slice.go:197 +0x0
  math/big.(*Rat).Float64()
      /tmp/workdir/go/src/math/big/nat.go:95 +0x1e1
  go/constant.Float64Val()
      /tmp/workdir/go/src/go/constant/value.go:554 +0x33c
  go/types.roundFloat64()
      /tmp/workdir/go/src/go/types/expr.go:175 +0x46
  go/types.representableConst()
      /tmp/workdir/go/src/go/types/expr.go:279 +0x89d
  go/types.(*Checker).representable()
      /tmp/workdir/go/src/go/types/expr.go:335 +0xb7
  go/types.(*Checker).convertUntyped()
      /tmp/workdir/go/src/go/types/expr.go:517 +0xe35
  go/types.(*Checker).binary()
      /tmp/workdir/go/src/go/types/expr.go:801 +0x229
  go/types.(*Checker).exprInternal()
      /tmp/workdir/go/src/go/types/expr.go:1504 +0x2d6f
  go/types.(*Checker).rawExpr()
      /tmp/workdir/go/src/go/types/expr.go:983 +0x91
  go/types.(*Checker).multiExpr()
      /tmp/workdir/go/src/go/types/expr.go:1600 +0x68
  go/types.(*Checker).initVars.func1()
      /tmp/workdir/go/src/go/types/assignments.go:209 +0xa3
  go/types.unpack()
      /tmp/workdir/go/src/go/types/call.go:181 +0xbb
  go/types.(*Checker).initVars()
      /tmp/workdir/go/src/go/types/assignments.go:209 +0xea
  go/types.(*Checker).shortVarDecl()
      /tmp/workdir/go/src/go/types/assignments.go:322 +0x3a8
  go/types.(*Checker).stmt()
      /tmp/workdir/go/src/go/types/stmt.go:398 +0x59ee
  go/types.(*Checker).stmtList()
      /tmp/workdir/go/src/go/types/stmt.go:120 +0xee
  go/types.(*Checker).funcBody()
      /tmp/workdir/go/src/go/types/stmt.go:42 +0x390
  go/types.(*Checker).funcDecl.func1()
      /tmp/workdir/go/src/go/types/decl.go:561 +0xee
  go/types.(*Checker).processDelayed()
      /tmp/workdir/go/src/go/types/resolver.go:616 +0x59
  go/types.(*Checker).checkFiles()
      /tmp/workdir/go/src/go/types/check.go:256 +0xf1
  golang.org/x/tools/go/packages.(*loader).loadPackage()
      /tmp/workdir/go/src/go/types/check.go:245 +0xd22
  golang.org/x/tools/go/packages.(*loader).loadRecursive.func1()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:685 +0x258
  sync.(*Once).doSlow()
      /tmp/workdir/go/src/sync/once.go:66 +0x100
  sync.(*Once).Do()
      /tmp/workdir/go/src/sync/once.go:57 +0x68
  golang.org/x/tools/go/packages.(*loader).loadRecursive()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:673 +0x74
  golang.org/x/tools/go/packages.(*loader).refine.func2()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:619 +0x42

Goroutine 632 (running) created at:
  golang.org/x/tools/go/packages.(*loader).refine()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:618 +0x1ce7
  golang.org/x/tools/go/packages.Load()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:215 +0x1b8
  golang.org/x/tools/go/packages_test.TestStdlibMetadata()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/stdlib_test.go:37 +0x1ac
  testing.tRunner()
      /tmp/workdir/go/src/testing/testing.go:909 +0x199

Goroutine 1217 (finished) created at:
  golang.org/x/tools/go/packages.(*loader).refine()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:618 +0x1ce7
  golang.org/x/tools/go/packages.Load()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:215 +0x1b8
  golang.org/x/tools/go/packages_test.TestStdlibMetadata()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/stdlib_test.go:37 +0x1ac
  testing.tRunner()
      /tmp/workdir/go/src/testing/testing.go:909 +0x199
==================
==================
WARNING: DATA RACE
Write at 0x00c001b41240 by goroutine 632:
  runtime.slicecopy()
      /tmp/workdir/go/src/runtime/slice.go:197 +0x0
  math/big.(*Rat).Float64()
      /tmp/workdir/go/src/math/big/nat.go:95 +0x1e1
  go/constant.Float64Val()
      /tmp/workdir/go/src/go/constant/value.go:554 +0x33c
  go/types.roundFloat64()
      /tmp/workdir/go/src/go/types/expr.go:175 +0x46
  go/types.representableConst()
      /tmp/workdir/go/src/go/types/expr.go:279 +0x89d
  go/types.(*Checker).representable()
      /tmp/workdir/go/src/go/types/expr.go:335 +0xb7
  go/types.(*Checker).convertUntyped()
      /tmp/workdir/go/src/go/types/expr.go:517 +0xe35
  go/types.(*Checker).binary()
      /tmp/workdir/go/src/go/types/expr.go:801 +0x229
  go/types.(*Checker).exprInternal()
      /tmp/workdir/go/src/go/types/expr.go:1504 +0x2d6f
  go/types.(*Checker).rawExpr()
      /tmp/workdir/go/src/go/types/expr.go:983 +0x91
  go/types.(*Checker).multiExpr()
      /tmp/workdir/go/src/go/types/expr.go:1600 +0x68
  go/types.(*Checker).expr()
      /tmp/workdir/go/src/go/types/expr.go:1594 +0x56
  go/types.(*Checker).binary()
      /tmp/workdir/go/src/go/types/expr.go:781 +0x108
  go/types.(*Checker).exprInternal()
      /tmp/workdir/go/src/go/types/expr.go:1504 +0x2d6f
  go/types.(*Checker).rawExpr()
      /tmp/workdir/go/src/go/types/expr.go:983 +0x91
  go/types.(*Checker).multiExpr()
      /tmp/workdir/go/src/go/types/expr.go:1600 +0x68
  go/types.(*Checker).expr()
      /tmp/workdir/go/src/go/types/expr.go:1594 +0x56
  go/types.(*Checker).stmt()
      /tmp/workdir/go/src/go/types/stmt.go:492 +0x6a90
  go/types.(*Checker).stmtList()
      /tmp/workdir/go/src/go/types/stmt.go:120 +0xee
  go/types.(*Checker).funcBody()
      /tmp/workdir/go/src/go/types/stmt.go:42 +0x390
  go/types.(*Checker).funcDecl.func1()
      /tmp/workdir/go/src/go/types/decl.go:561 +0xee
  go/types.(*Checker).processDelayed()
      /tmp/workdir/go/src/go/types/resolver.go:616 +0x59
  go/types.(*Checker).checkFiles()
      /tmp/workdir/go/src/go/types/check.go:256 +0xf1
  golang.org/x/tools/go/packages.(*loader).loadPackage()
      /tmp/workdir/go/src/go/types/check.go:245 +0xd22
  golang.org/x/tools/go/packages.(*loader).loadRecursive.func1()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:685 +0x258
  sync.(*Once).doSlow()
      /tmp/workdir/go/src/sync/once.go:66 +0x100
  sync.(*Once).Do()
      /tmp/workdir/go/src/sync/once.go:57 +0x68
  golang.org/x/tools/go/packages.(*loader).loadRecursive()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:673 +0x74
  golang.org/x/tools/go/packages.(*loader).refine.func2()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:619 +0x42

Previous write at 0x00c001b41240 by goroutine 1217:
  runtime.slicecopy()
      /tmp/workdir/go/src/runtime/slice.go:197 +0x0
  math/big.(*Rat).Float64()
      /tmp/workdir/go/src/math/big/nat.go:95 +0x1e1
  go/constant.Float64Val()
      /tmp/workdir/go/src/go/constant/value.go:554 +0x33c
  go/types.roundFloat64()
      /tmp/workdir/go/src/go/types/expr.go:175 +0x46
  go/types.representableConst()
      /tmp/workdir/go/src/go/types/expr.go:279 +0x89d
  go/types.(*Checker).representable()
      /tmp/workdir/go/src/go/types/expr.go:335 +0xb7
  go/types.(*Checker).convertUntyped()
      /tmp/workdir/go/src/go/types/expr.go:517 +0xe35
  go/types.(*Checker).binary()
      /tmp/workdir/go/src/go/types/expr.go:801 +0x229
  go/types.(*Checker).exprInternal()
      /tmp/workdir/go/src/go/types/expr.go:1504 +0x2d6f
  go/types.(*Checker).rawExpr()
      /tmp/workdir/go/src/go/types/expr.go:983 +0x91
  go/types.(*Checker).multiExpr()
      /tmp/workdir/go/src/go/types/expr.go:1600 +0x68
  go/types.(*Checker).initVars.func1()
      /tmp/workdir/go/src/go/types/assignments.go:209 +0xa3
  go/types.unpack()
      /tmp/workdir/go/src/go/types/call.go:181 +0xbb
  go/types.(*Checker).initVars()
      /tmp/workdir/go/src/go/types/assignments.go:209 +0xea
  go/types.(*Checker).shortVarDecl()
      /tmp/workdir/go/src/go/types/assignments.go:322 +0x3a8
  go/types.(*Checker).stmt()
      /tmp/workdir/go/src/go/types/stmt.go:398 +0x59ee
  go/types.(*Checker).stmtList()
      /tmp/workdir/go/src/go/types/stmt.go:120 +0xee
  go/types.(*Checker).funcBody()
      /tmp/workdir/go/src/go/types/stmt.go:42 +0x390
  go/types.(*Checker).funcDecl.func1()
      /tmp/workdir/go/src/go/types/decl.go:561 +0xee
  go/types.(*Checker).processDelayed()
      /tmp/workdir/go/src/go/types/resolver.go:616 +0x59
  go/types.(*Checker).checkFiles()
      /tmp/workdir/go/src/go/types/check.go:256 +0xf1
  golang.org/x/tools/go/packages.(*loader).loadPackage()
      /tmp/workdir/go/src/go/types/check.go:245 +0xd22
  golang.org/x/tools/go/packages.(*loader).loadRecursive.func1()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:685 +0x258
  sync.(*Once).doSlow()
      /tmp/workdir/go/src/sync/once.go:66 +0x100
  sync.(*Once).Do()
      /tmp/workdir/go/src/sync/once.go:57 +0x68
  golang.org/x/tools/go/packages.(*loader).loadRecursive()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:673 +0x74
  golang.org/x/tools/go/packages.(*loader).refine.func2()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:619 +0x42

Goroutine 632 (running) created at:
  golang.org/x/tools/go/packages.(*loader).refine()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:618 +0x1ce7
  golang.org/x/tools/go/packages.Load()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:215 +0x1b8
  golang.org/x/tools/go/packages_test.TestStdlibMetadata()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/stdlib_test.go:37 +0x1ac
  testing.tRunner()
      /tmp/workdir/go/src/testing/testing.go:909 +0x199

Goroutine 1217 (finished) created at:
  golang.org/x/tools/go/packages.(*loader).refine()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:618 +0x1ce7
  golang.org/x/tools/go/packages.Load()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/packages.go:215 +0x1b8
  golang.org/x/tools/go/packages_test.TestStdlibMetadata()
      /tmp/workdir/gopath/src/golang.org/x/tools/go/packages/stdlib_test.go:37 +0x1ac
  testing.tRunner()
      /tmp/workdir/go/src/testing/testing.go:909 +0x199
==================
--- FAIL: TestStdlibMetadata (7.92s)
    stdlib_test.go:50: Loaded 199 packages
    stdlib_test.go:58: GOMAXPROCS:  8
    stdlib_test.go:59: Metadata:    7.737338684s
    stdlib_test.go:60: #MB:         273
    testing.go:853: race detected during execution of test
FAIL
FAIL	golang.org/x/tools/go/packages	201.281s


==================
WARNING: DATA RACE
Write at 0x00c0044b8f30 by goroutine 861:
  golang.org/x/tools/internal/lsp/cmd.(*cmdClient).PublishDiagnostics()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cmd/cmd.go:373 +0x1b4
  golang.org/x/tools/internal/lsp/protocol.clientHandler.Deliver()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/protocol/tsclient.go:78 +0x1407
  golang.org/x/tools/internal/lsp/protocol.(*clientHandler).Deliver()
      <autogenerated>:1 +0x94
  golang.org/x/tools/internal/jsonrpc2.(*Conn).Run.func1()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/jsonrpc2/jsonrpc2.go:370 +0x1c5

Previous read at 0x00c0044b8f30 by goroutine 363:
  golang.org/x/tools/internal/lsp/cmd.(*check).Run()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cmd/check.go:63 +0x4cb
  golang.org/x/tools/internal/tool.Run()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/tool/tool.go:152 +0x304
  golang.org/x/tools/internal/lsp/cmd.(*Application).Run()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cmd/cmd.go:144 +0x3f8
  golang.org/x/tools/internal/tool.Run()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/tool/tool.go:152 +0x304
  golang.org/x/tools/internal/lsp/cmd/test.(*runner).RunGoplsCmd()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cmd/test/cmdtest.go:124 +0x5ee
  golang.org/x/tools/internal/lsp/cmd/test.(*runner).Diagnostics()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cmd/test/check.go:25 +0x1e0
  golang.org/x/tools/internal/lsp/tests.Run.func9.1()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/tests/tests.go:421 +0xff
  testing.tRunner()
      /tmp/workdir/go/src/testing/testing.go:954 +0x1eb

Goroutine 861 (running) created at:
  golang.org/x/tools/internal/jsonrpc2.(*Conn).Run()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/jsonrpc2/jsonrpc2.go:354 +0x803

Goroutine 363 (running) created at:
  testing.(*T).Run()
      /tmp/workdir/go/src/testing/testing.go:1005 +0x660
  golang.org/x/tools/internal/lsp/tests.Run.func9()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/tests/tests.go:419 +0x150
  testing.tRunner()
      /tmp/workdir/go/src/testing/testing.go:954 +0x1eb
==================
2020/01/15 20:45:37 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:37 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:38 Error:2020/01/15 20:45:37 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:38 Error:2020/01/15 20:45:37 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:38 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:38 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:38 Error:2020/01/15 20:45:38 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:38 Error:2020/01/15 20:45:38 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:40 failed to run analyses: must supply Converter for file "/tmp/workdir/gocache/19/19b90960c3b6fa9dd8a17f12bfd923fb82c129a44b442382dc949b36b5211152-d" containing lines from "/tmp/workdir/tmp/TestCommandLine_GOPATH582845144/lsp/src/golang.org/x/tools/internal/lsp/cgo/declarecgo.go"
	package = golang.org/x/tools/internal/lsp/cgo
2020/01/15 20:45:40 Error:2020/01/15 20:45:40 failed to run analyses: must supply Converter for file "/tmp/workdir/gocache/19/19b90960c3b6fa9dd8a17f12bfd923fb82c129a44b442382dc949b36b5211152-d" containing lines from "/tmp/workdir/tmp/TestCommandLine_GOPATH582845144/lsp/src/golang.org/x/tools/internal/lsp/cgo/declarecgo.go"
	package = golang.org/x/tools/internal/lsp/cgo
2020/01/15 20:45:40 no dep handle: no metadata for golang.org/x/tools/internal/lsp/protocol
	package = golang.org/x/tools/internal/lsp/protocol
2020/01/15 20:45:40 no dep handle: no metadata for golang.org/x/tools/internal/lsp/protocol
	package = golang.org/x/tools/internal/lsp/protocol
2020/01/15 20:45:40 Error:2020/01/15 20:45:40 no dep handle: no metadata for golang.org/x/tools/internal/lsp/protocol
	package = golang.org/x/tools/internal/lsp/protocol
2020/01/15 20:45:40 Error:2020/01/15 20:45:40 no dep handle: no metadata for golang.org/x/tools/internal/lsp/protocol
	package = golang.org/x/tools/internal/lsp/protocol
2020/01/15 20:45:40 no dep handle: no metadata for golang.org/x/xerrors
	package = golang.org/x/xerrors
2020/01/15 20:45:40 Error:2020/01/15 20:45:40 no dep handle: no metadata for golang.org/x/xerrors
	package = golang.org/x/xerrors
2020/01/15 20:45:43 no dep handle: no metadata for golang.org/x/xerrors
	package = golang.org/x/xerrors
2020/01/15 20:45:43 Error:2020/01/15 20:45:43 no dep handle: no metadata for golang.org/x/xerrors
	package = golang.org/x/xerrors
2020/01/15 20:45:46 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:46 Error:2020/01/15 20:45:46 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:46 no dep handle: no metadata for golang.org/x/tools/internal/lsp/protocol
	package = golang.org/x/tools/internal/lsp/protocol
2020/01/15 20:45:46 Error:2020/01/15 20:45:46 no dep handle: no metadata for golang.org/x/tools/internal/lsp/protocol
	package = golang.org/x/tools/internal/lsp/protocol
2020/01/15 20:45:46 no dep handle: no metadata for golang.org/x/xerrors
	package = golang.org/x/xerrors
2020/01/15 20:45:46 Error:2020/01/15 20:45:46 no dep handle: no metadata for golang.org/x/xerrors
	package = golang.org/x/xerrors
2020/01/15 20:45:46 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:46 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:46 Error:2020/01/15 20:45:46 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:47 Error:2020/01/15 20:45:46 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:55 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:55 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:55 Error:2020/01/15 20:45:55 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:55 Error:2020/01/15 20:45:55 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:56 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:56 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:56 Error:2020/01/15 20:45:56 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:56 Error:2020/01/15 20:45:56 no dep handle: no metadata for nosuchpkg
	package = nosuchpkg
2020/01/15 20:45:58 no dep handle: no metadata for golang.org/x/tools/internal/lsp/protocol
	package = golang.org/x/tools/internal/lsp/protocol
2020/01/15 20:45:58 no dep handle: no metadata for golang.org/x/tools/internal/lsp/protocol
	package = golang.org/x/tools/internal/lsp/protocol
2020/01/15 20:45:58 Error:2020/01/15 20:45:58 no dep handle: no metadata for golang.org/x/tools/internal/lsp/protocol
	package = golang.org/x/tools/internal/lsp/protocol
2020/01/15 20:45:58 Error:2020/01/15 20:45:58 no dep handle: no metadata for golang.org/x/tools/internal/lsp/protocol
	package = golang.org/x/tools/internal/lsp/protocol
2020/01/15 20:45:59 no dep handle: no metadata for golang.org/x/xerrors
	package = golang.org/x/xerrors
2020/01/15 20:45:59 Error:2020/01/15 20:45:59 no dep handle: no metadata for golang.org/x/xerrors
	package = golang.org/x/xerrors
2020/01/15 20:46:04 no dep handle: no metadata for golang.org/x/xerrors
	package = golang.org/x/xerrors
2020/01/15 20:46:04 Error:2020/01/15 20:46:04 no dep handle: no metadata for golang.org/x/xerrors
	package = golang.org/x/xerrors
--- FAIL: TestCommandLine (32.75s)
    --- FAIL: TestCommandLine/GOPATH (12.18s)
        --- FAIL: TestCommandLine/GOPATH/Diagnostics (6.16s)
            --- FAIL: TestCommandLine/GOPATH/Diagnostics/testy_test (0.65s)
                testing.go:888: race detected during execution of test
            testing.go:888: race detected during execution of test
        testing.go:888: race detected during execution of test
    testing.go:888: race detected during execution of test
FAIL
FAIL	golang.org/x/tools/gopls/test	33.062s


==================
WARNING: DATA RACE
Read at 0x00c0003058d0 by goroutine 45:
  go/ast.NewPackage()
      /tmp/workdir/go/src/go/ast/resolve.go:160 +0xda5
  golang.org/x/tools/internal/lsp/cache.(*view).buildBuiltinPackage()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/builtin.go:73 +0x5c4
  golang.org/x/tools/internal/lsp/cache.(*view).initialize.func1.2()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:489 +0x53
  golang.org/x/sync/errgroup.(*Group).Go.func1()
      /tmp/workdir/gopath/pkg/mod/golang.org/x/sync@v0.0.0-20190423024810-112230192c58/errgroup/errgroup.go:57 +0x85

Previous write at 0x00c0003058d0 by goroutine 57:
  go/ast.NewPackage()
      /tmp/workdir/go/src/go/ast/resolve.go:168 +0xffc
  golang.org/x/tools/internal/lsp/cache.(*view).buildBuiltinPackage()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/builtin.go:73 +0x5c4
  golang.org/x/tools/internal/lsp/cache.(*view).initialize.func1.2()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:489 +0x53
  golang.org/x/sync/errgroup.(*Group).Go.func1()
      /tmp/workdir/gopath/pkg/mod/golang.org/x/sync@v0.0.0-20190423024810-112230192c58/errgroup/errgroup.go:57 +0x85

Goroutine 45 (running) created at:
  golang.org/x/sync/errgroup.(*Group).Go()
      /tmp/workdir/gopath/pkg/mod/golang.org/x/sync@v0.0.0-20190423024810-112230192c58/errgroup/errgroup.go:54 +0x73
  golang.org/x/tools/internal/lsp/cache.(*view).initialize.func1()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:488 +0x336
  sync.(*Once).doSlow()
      /tmp/workdir/go/src/sync/once.go:66 +0x103
  sync.(*Once).Do()
      /tmp/workdir/go/src/sync/once.go:57 +0x68
  golang.org/x/tools/internal/lsp/cache.(*view).initialize()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:467 +0x93

Goroutine 57 (finished) created at:
  golang.org/x/sync/errgroup.(*Group).Go()
      /tmp/workdir/gopath/pkg/mod/golang.org/x/sync@v0.0.0-20190423024810-112230192c58/errgroup/errgroup.go:54 +0x73
  golang.org/x/tools/internal/lsp/cache.(*view).initialize.func1()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:488 +0x336
  sync.(*Once).doSlow()
      /tmp/workdir/go/src/sync/once.go:66 +0x103
  sync.(*Once).Do()
      /tmp/workdir/go/src/sync/once.go:57 +0x68
  golang.org/x/tools/internal/lsp/cache.(*view).initialize()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:467 +0x93
==================
--- FAIL: TestDiagnostics (5.04s)
    --- FAIL: TestDiagnostics/invalidrequire (0.67s)
        testing.go:888: race detected during execution of test
    testing.go:888: race detected during execution of test
FAIL
FAIL	golang.org/x/tools/internal/lsp/mod	5.382s

See:

/cc @stamblerre @matloob @findleyr
@gopherbot gopherbot added this to the Unreleased milestone Jan 16, 2020
@gopherbot gopherbot added Tools This label describes issues relating to any tools in the x/tools repository. gopls Issues related to the Go language server, gopls. labels Jan 16, 2020
@dmitshur dmitshur mentioned this issue Jan 16, 2020
Closed
@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 16, 2020
@stamblerre
Copy link
Contributor

I believe that the two lower data races were briefly introduced in gopls and then fixed over the last few days. For one of them, the file referenced (golang.org/x/tools/internal/lsp/cache/builtin.go) no longer exists. Please let me know if anything comes up again, but I think we should be ok on the gopls side.

@stamblerre stamblerre modified the milestones: Unreleased, gopls/v0.3.0 Jan 17, 2020
@dmitshur
Copy link
Contributor Author

Thanks for the update @stamblerre.

I'll retitle this to be about the remaining races in go/packages then, and update this issue if something new comes up.

@stamblerre
Copy link
Contributor

Noticed one more that I don't think we've debugged yet:


==================
WARNING: DATA RACE
Read at 0x00c0004120e8 by goroutine 317:
  golang.org/x/tools/internal/lsp/cache.(*view).storeModFileVersions()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:395 +0x312
  golang.org/x/tools/internal/lsp/cache.(*view).RunProcessEnvFunc()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:313 +0x217
  golang.org/x/tools/internal/lsp/source.AllImportsFixes()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/source/format.go:90 +0x927
  golang.org/x/tools/internal/lsp.(*Server).codeAction()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/code_action.go:74 +0xb6b
  golang.org/x/tools/internal/lsp.(*Server).CodeAction()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/server.go:219 +0x66
  golang.org/x/tools/internal/lsp/protocol.serverHandler.Deliver()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/protocol/tsserver.go:404 +0x3424
  golang.org/x/tools/internal/lsp/protocol.(*serverHandler).Deliver()
      <autogenerated>:1 +0x94
  golang.org/x/tools/internal/jsonrpc2.(*Conn).Run.func1()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/jsonrpc2/jsonrpc2.go:370 +0x1c5

Previous write at 0x00c0004120e8 by goroutine 2045:
  golang.org/x/tools/internal/imports.(*ModuleResolver).init()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/mod.go:144 +0xf1b
  golang.org/x/tools/internal/imports.(*ModuleResolver).scan()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/mod.go:391 +0x5e
  golang.org/x/tools/internal/imports.getCandidatePkgs()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/fix.go:643 +0xa39
  golang.org/x/tools/internal/imports.PrimeCache()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/fix.go:659 +0x143
  golang.org/x/tools/internal/lsp/cache.(*view).refreshProcessEnv()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:338 +0x137
  golang.org/x/tools/internal/lsp/cache.(*view).refreshProcessEnv-fm()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:328 +0x41

Goroutine 317 (running) created at:
  golang.org/x/tools/internal/jsonrpc2.(*Conn).Run()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/jsonrpc2/jsonrpc2.go:354 +0x803
  golang.org/x/tools/internal/lsp/cmd.(*Application).connect.func1()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/server.go:64 +0x1f1

Goroutine 2045 (running) created at:
  time.goFunc()
      /tmp/workdir/go/src/time/sleep.go:168 +0x51
==================
==================
WARNING: DATA RACE
Read at 0x00c0004120f8 by goroutine 317:
  golang.org/x/tools/internal/lsp/cache.(*view).storeModFileVersions()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:402 +0x103
  golang.org/x/tools/internal/lsp/cache.(*view).RunProcessEnvFunc()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:313 +0x217
  golang.org/x/tools/internal/lsp/source.AllImportsFixes()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/source/format.go:90 +0x927
  golang.org/x/tools/internal/lsp.(*Server).codeAction()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/code_action.go:74 +0xb6b
  golang.org/x/tools/internal/lsp.(*Server).CodeAction()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/server.go:219 +0x66
  golang.org/x/tools/internal/lsp/protocol.serverHandler.Deliver()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/protocol/tsserver.go:404 +0x3424
  golang.org/x/tools/internal/lsp/protocol.(*serverHandler).Deliver()
      <autogenerated>:1 +0x94
  golang.org/x/tools/internal/jsonrpc2.(*Conn).Run.func1()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/jsonrpc2/jsonrpc2.go:370 +0x1c5

Previous write at 0x00c0004120f8 by goroutine 2045:
  golang.org/x/tools/internal/imports.(*ModuleResolver).initAllMods()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/mod.go:165 +0x44d
  golang.org/x/tools/internal/imports.(*ModuleResolver).init()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/mod.go:82 +0xcf
  golang.org/x/tools/internal/imports.(*ModuleResolver).scan()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/mod.go:391 +0x5e
  golang.org/x/tools/internal/imports.getCandidatePkgs()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/fix.go:643 +0xa39
  golang.org/x/tools/internal/imports.PrimeCache()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/fix.go:659 +0x143
  golang.org/x/tools/internal/lsp/cache.(*view).refreshProcessEnv()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:338 +0x137
  golang.org/x/tools/internal/lsp/cache.(*view).refreshProcessEnv-fm()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:328 +0x41

Goroutine 317 (running) created at:
  golang.org/x/tools/internal/jsonrpc2.(*Conn).Run()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/jsonrpc2/jsonrpc2.go:354 +0x803
  golang.org/x/tools/internal/lsp/cmd.(*Application).connect.func1()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/server.go:64 +0x1f1

Goroutine 2045 (running) created at:
  time.goFunc()
      /tmp/workdir/go/src/time/sleep.go:168 +0x51
==================
==================
WARNING: DATA RACE
Read at 0x00c00b468428 by goroutine 317:
  golang.org/x/tools/internal/lsp/cache.(*view).storeModFileVersions()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:403 +0x16e
  golang.org/x/tools/internal/lsp/cache.(*view).RunProcessEnvFunc()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:313 +0x217
  golang.org/x/tools/internal/lsp/source.AllImportsFixes()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/source/format.go:90 +0x927
  golang.org/x/tools/internal/lsp.(*Server).codeAction()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/code_action.go:74 +0xb6b
  golang.org/x/tools/internal/lsp.(*Server).CodeAction()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/server.go:219 +0x66
  golang.org/x/tools/internal/lsp/protocol.serverHandler.Deliver()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/protocol/tsserver.go:404 +0x3424
  golang.org/x/tools/internal/lsp/protocol.(*serverHandler).Deliver()
      <autogenerated>:1 +0x94
  golang.org/x/tools/internal/jsonrpc2.(*Conn).Run.func1()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/jsonrpc2/jsonrpc2.go:370 +0x1c5

Previous write at 0x00c00b468428 by goroutine 2045:
  golang.org/x/tools/internal/imports.(*ModuleResolver).initAllMods()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/mod.go:154 +0x1c9
  golang.org/x/tools/internal/imports.(*ModuleResolver).init()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/mod.go:82 +0xcf
  golang.org/x/tools/internal/imports.(*ModuleResolver).scan()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/mod.go:391 +0x5e
  golang.org/x/tools/internal/imports.getCandidatePkgs()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/fix.go:643 +0xa39
  golang.org/x/tools/internal/imports.PrimeCache()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/fix.go:659 +0x143
  golang.org/x/tools/internal/lsp/cache.(*view).refreshProcessEnv()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:338 +0x137
  golang.org/x/tools/internal/lsp/cache.(*view).refreshProcessEnv-fm()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:328 +0x41

Goroutine 317 (running) created at:
  golang.org/x/tools/internal/jsonrpc2.(*Conn).Run()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/jsonrpc2/jsonrpc2.go:354 +0x803
  golang.org/x/tools/internal/lsp/cmd.(*Application).connect.func1()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/server.go:64 +0x1f1

Goroutine 2045 (running) created at:
  time.goFunc()
      /tmp/workdir/go/src/time/sleep.go:168 +0x51
==================
==================
WARNING: DATA RACE
Read at 0x00c00b468440 by goroutine 317:
  golang.org/x/tools/internal/lsp/cache.(*view).storeModFileVersions()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:403 +0x294
  golang.org/x/tools/internal/lsp/cache.(*view).RunProcessEnvFunc()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:313 +0x217
  golang.org/x/tools/internal/lsp/source.AllImportsFixes()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/source/format.go:90 +0x927
  golang.org/x/tools/internal/lsp.(*Server).codeAction()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/code_action.go:74 +0xb6b
  golang.org/x/tools/internal/lsp.(*Server).CodeAction()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/server.go:219 +0x66
  golang.org/x/tools/internal/lsp/protocol.serverHandler.Deliver()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/protocol/tsserver.go:404 +0x3424
  golang.org/x/tools/internal/lsp/protocol.(*serverHandler).Deliver()
      <autogenerated>:1 +0x94
  golang.org/x/tools/internal/jsonrpc2.(*Conn).Run.func1()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/jsonrpc2/jsonrpc2.go:370 +0x1c5

Previous write at 0x00c00b468440 by goroutine 2045:
  golang.org/x/tools/internal/imports.(*ModuleResolver).initAllMods()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/mod.go:154 +0x1c9
  golang.org/x/tools/internal/imports.(*ModuleResolver).init()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/mod.go:82 +0xcf
  golang.org/x/tools/internal/imports.(*ModuleResolver).scan()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/mod.go:391 +0x5e
  golang.org/x/tools/internal/imports.getCandidatePkgs()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/fix.go:643 +0xa39
  golang.org/x/tools/internal/imports.PrimeCache()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/imports/fix.go:659 +0x143
  golang.org/x/tools/internal/lsp/cache.(*view).refreshProcessEnv()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:338 +0x137
  golang.org/x/tools/internal/lsp/cache.(*view).refreshProcessEnv-fm()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/cache/view.go:328 +0x41

Goroutine 317 (running) created at:
  golang.org/x/tools/internal/jsonrpc2.(*Conn).Run()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/jsonrpc2/jsonrpc2.go:354 +0x803
  golang.org/x/tools/internal/lsp/cmd.(*Application).connect.func1()
      /tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/server.go:64 +0x1f1

Goroutine 2045 (running) created at:
  time.goFunc()
      /tmp/workdir/go/src/time/sleep.go:168 +0x51
==================

/cc @heschik

@dmitshur dmitshur changed the title x/tools/go/packages, x/tools/gopls/test, x/tools/internal/lsp/cmd: race detected during execution of various tests x/tools/go/packages: race detected during execution of various tests Jan 17, 2020
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/215319 mentions this issue: internal/lsp/cache: let gopls track go.mod files

gopherbot pushed a commit to golang/tools that referenced this issue Jan 17, 2020
@heschi
internal/lsp/cache: let gopls track go.mod files
354bea8 
We used to read the go.mod file information out of the imports.Resolver.
Now that gopls tracks go.mod itself, we can use that instead. This is a
slight regression, in that go.mods in replace targets will no longer be
watched, but I don't think that's too important.

This allows us to stop reading the ModuleResolver's internals, which
were not sufficiently locked.

Updates golang/go#36605.

Change-Id: I42939e0248cba1f6b3850a003de67fcc11ab10b1
Reviewed-on: https://go-review.googlesource.com/c/tools/+/215319
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
@stamblerre
Copy link
Contributor

The above CL fixed the gopls race, leaving this open for go/packages.

@stamblerre stamblerre modified the milestones: gopls/v0.3.0, Unreleased Jan 17, 2020
@stamblerre stamblerre removed the gopls Issues related to the Go language server, gopls. label Jan 21, 2020
@matloob
Copy link
Contributor

matloob commented Jan 21, 2020

I'll take a look at the go/packages races

@matloob
Copy link
Contributor

matloob commented Jan 21, 2020

The failure in go/packages looks like #34919. (Which explains why it's only occurring in 1.13 -- #34919 has been fixed on tip).

The fix is a pretty simple cl: golang.org/cl/201205, but I don't know what the bar is for cherry picking something like that in. go/packages relies on it being safe to do concurrent reads of package types, and changing that would be a pretty invasive change, so I don't think we'd want to go down the route of modifying go/packages.

I tried cherrypicking the fix CL (golang.org/cl/201205) and running the tests in race mode and it seems to fix the issue.

@dmitshur, What do you think?

@dmitshur dmitshur mentioned this issue Jan 21, 2020
Closed
@dmitshur
Copy link
Contributor Author

Thanks for the analysis @matloob.

I'm not sure if those math/big fixes meet the bar for backporting. I think we need to have a better understanding of what workarounds are available for Go 1.13 and 1.12. I posted #34919 (comment) to seek feedback.

@matloob
Copy link
Contributor

matloob commented Jan 22, 2020 •
edited
Loading

I suspected they wouldn't... the problem is that the user of the code (package go/types) is also in the standard library, so the primary workaround is unworkable for the same reason.

@dmitshur
Copy link
Contributor Author

I understand there is currently a data race in x/tools/go/packages for Go 1.13 and 1.12 users, and there are multiple approaches to fixing that we can consider. We should see what approach is most safe and least invasive. If all of the approaches turn out to be infeasible, then we should skip the failing tests by re-applying CL 200819 but with skips applied to 1.13 and older. But that should be a last resort.

Do you know how big of a change it'd be to update go/types in 1.13 and 1.12 to use math/big in a way that doesn't cause data races?

@dmitshur dmitshur mentioned this issue Jan 22, 2020
Closed
This was referenced Jan 22, 2020
Closed
Closed
Closed
@bcmills bcmills mentioned this issue Jan 29, 2020
Closed
This was referenced Feb 21, 2020
Closed
Closed
@bcmills bcmills mentioned this issue Mar 2, 2020
Open
@matloob
Copy link
Contributor

matloob commented Mar 10, 2020

This is a race in the go/types API. The only workarounds are to not use the API concurrently, which we can do in go/packages, but which would show up to other users.

Fixing this in go/types would require writing new code, so the backport seems preferable.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/227859 mentions this issue: dashboard: upsize freebsd-amd64-race builder to 16 vCPUs, 14.4 GB mem

@gopherbot

This comment has been minimized.

Sign in to view
gopherbot pushed a commit to golang/build that referenced this issue Apr 10, 2020
@dmitshur
dashboard: upsize freebsd-amd64-race builder to 16 vCPUs, 14.4 GB RAM
baa2dcb 
This is a followup to CL 214433. Start using n1-highcpu-16 machine type
instead of n1-highcpu-8 for the freebsd-amd64-race builder.

Increasing the RAM from 3.6 GB to 7.2 GB has helped golang/go#36444
significantly: the builder stopped failing consistently on x/tools
and resulted in many data races being uncovered in golang/go#36605.

However, by now, it has started to fail consistently again. This
time it seems to be due to low performance, causing the tests in
golang.org/x/tools/internal/lsp/regtest package to fail due with
"context deadline exceeded" errors.

FreeBSD is one of the ports that stays visible when "show only first-
class ports" is checked on build.golang.org. The other -race builders
have all been upgraded to the n1-highcpu-16 machine type by now out
of necessity.

It seems fair to provide the FreeBSD port with an equal amount of
resources, even if the increased memory isn't strictly required yet.
Once this change is applied, if the failures persist, we can be more
confident that the problem is due to the code or the port, rather
than due to this -race builder having 2𝗑 less CPU and RAM resources
compared to other -race builders.

An alternative is to increase timeout for this builder type, but I'm
opting to defer exploring that after equalizing the machine type.

For golang/go#36444.
For golang/go#34621.
For golang/go#29252.
For golang/go#33986.

Change-Id: I41f149365128c7bc6f576c778ac07618acc04612
Reviewed-on: https://go-review.googlesource.com/c/build/+/227859
Reviewed-by: Alexander Rakoczy <alex@golang.org>
@dmitshur
Copy link
Contributor Author

The race in go/types API was fixed in #36689. /cc @matloob

I believe this issue should be resolved now. We can watch it for 2 weeks and close if nothing new happens.

@bcmills
Copy link
Contributor

bcmills commented May 3, 2023

We can watch it for 2 weeks and close if nothing new happens.

It's been a bit longer than two weeks, so closing. 😅

@bcmills bcmills closed this as completed May 3, 2023
@golang golang locked and limited conversation to collaborators May 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
5 participants
@dmitshur @bcmills @stamblerre @gopherbot @matloob