Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/benchmarks/sweet/cmd/sweet: TestSweetEndToEnd failures #56958

Open
gopherbot opened this issue Nov 28, 2022 · 39 comments
Open

x/benchmarks/sweet/cmd/sweet: TestSweetEndToEnd failures #56958

gopherbot opened this issue Nov 28, 2022 · 39 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@gopherbot
Copy link

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"

Issue created automatically to collect these failures.

Example (log):

panic: test timed out after 10m0s
running tests:
	TestSweetEndToEnd (10m0s)

sync.runtime_Semacquire(0xc0004504e8?)
	/workdir/go/src/runtime/sema.go:62 +0x27
sync.(*WaitGroup).Wait(0xc0004504e0)
	/workdir/go/src/sync/waitgroup.go:116 +0xa5
golang.org/x/benchmarks/sweet/cmd/sweet_test.TestSweetEndToEnd(0xc000102ea0)
	/workdir/gopath/src/golang.org/x/benchmarks/sweet/cmd/sweet/integration_test.go:185 +0x1638
testing.tRunner(0xc000102ea0, 0x10e2888)

watchflakes

@gopherbot gopherbot added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Nov 28, 2022
@gopherbot gopherbot added this to the Unreleased milestone Nov 28, 2022
@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2022-11-22 03:06 linux-amd64-longtest-race benchmarks@0e4f958d go@cc7530da x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
panic: test timed out after 10m0s
running tests:
	TestSweetEndToEnd (10m0s)

sync.runtime_Semacquire(0xc0004504e8?)
	/workdir/go/src/runtime/sema.go:62 +0x27
sync.(*WaitGroup).Wait(0xc0004504e0)
	/workdir/go/src/sync/waitgroup.go:116 +0xa5
golang.org/x/benchmarks/sweet/cmd/sweet_test.TestSweetEndToEnd(0xc000102ea0)
	/workdir/gopath/src/golang.org/x/benchmarks/sweet/cmd/sweet/integration_test.go:185 +0x1638
testing.tRunner(0xc000102ea0, 0x10e2888)

watchflakes

@cherrymui cherrymui changed the title x/benchmarks/sweet/cmd/sweet: TestSweetEndToEnd failures x/benchmarks/sweet/cmd/sweet: TestSweetEndToEnd timeout Nov 28, 2022
@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2022-11-22 03:06 linux-amd64-longtest benchmarks@0e4f958d go@60525dc3 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
panic: test timed out after 10m0s
running tests:
	TestSweetEndToEnd (10m0s)

sync.runtime_Semacquire(0xc000112690?)
	/workdir/go/src/runtime/sema.go:62 +0x27
sync.(*WaitGroup).Wait(0xc000136aa0?)
	/workdir/go/src/sync/waitgroup.go:116 +0x4b
golang.org/x/benchmarks/sweet/cmd/sweet_test.TestSweetEndToEnd(0xc000682820)
	/workdir/gopath/src/golang.org/x/benchmarks/sweet/cmd/sweet/integration_test.go:185 +0xfc5
testing.tRunner(0xc000682820, 0xc7fa08)

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2022-11-22 03:06 linux-amd64-longtest-race benchmarks@0e4f958d go@45f5ef4e x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
panic: test timed out after 10m0s
running tests:
	TestSweetEndToEnd (10m0s)

sync.runtime_Semacquire(0xc00003e6e8?)
	/workdir/go/src/runtime/sema.go:62 +0x27
sync.(*WaitGroup).Wait(0xc00003e6e0)
	/workdir/go/src/sync/waitgroup.go:116 +0xa5
golang.org/x/benchmarks/sweet/cmd/sweet_test.TestSweetEndToEnd(0xc00038ad00)
	/workdir/gopath/src/golang.org/x/benchmarks/sweet/cmd/sweet/integration_test.go:185 +0x1638
testing.tRunner(0xc00038ad00, 0x10e2868)

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2022-12-13 17:27 linux-amd64-longtest benchmarks@6a564b04 go@cb077650 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (366.41s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs per config (1))
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [shell] git -C /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite checkout 0a8194a898a1ceff6a0b29e3419650daf43d8567
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/results-1/go-build
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/bin
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/tmp
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/assets
        [shell] pushd install
        [shell] GOROOT=/workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/bin/goroot /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/bin/goroot/bin/go cmd/compile cmd/link
        [shell] popd
        [sweet] error: build go-build for go.profile: error building cmd/compile and cmd/link: chdir install: no such file or directory
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1
2022-12-13 17:27 linux-amd64-longtest benchmarks@6a564b04 go@5ba98b97 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (406.86s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs per config (1))
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [shell] git -C /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite checkout 0a8194a898a1ceff6a0b29e3419650daf43d8567
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/results-1/go-build
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/bin
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/tmp
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/assets
        [shell] pushd install
        [shell] GOROOT=/workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/bin/goroot /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/bin/goroot/bin/go cmd/compile cmd/link
        [shell] popd
        [sweet] error: build go-build for go.profile: error building cmd/compile and cmd/link: chdir install: no such file or directory
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1
2022-12-13 17:27 linux-amd64-longtest-race benchmarks@6a564b04 go@cb077650 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (400.32s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs per config (1))
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [shell] git -C /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite checkout 0a8194a898a1ceff6a0b29e3419650daf43d8567
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/results-1/go-build
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/bin
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/tmp
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/assets
        [shell] pushd install
        [shell] GOROOT=/workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/bin/goroot /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/bin/goroot/bin/go cmd/compile cmd/link
        [shell] popd
        [sweet] error: build go-build for go.profile: error building cmd/compile and cmd/link: chdir install: no such file or directory
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1
2022-12-13 17:27 linux-amd64-longtest-race benchmarks@6a564b04 go@5ba98b97 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (410.95s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs per config (1))
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [shell] git -C /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite checkout 0a8194a898a1ceff6a0b29e3419650daf43d8567
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/results-1/go-build
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/bin
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/tmp
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/assets
        [shell] pushd install
        [shell] GOROOT=/workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/bin/goroot /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/go.profile/bin/goroot/bin/go cmd/compile cmd/link
        [shell] popd
        [sweet] error: build go-build for go.profile: error building cmd/compile and cmd/link: chdir install: no such file or directory
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2022-12-22 15:34 linux-amd64-longtest-race benchmarks@03c43f28 go@82f09b75 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (493.74s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs per config (1))
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [sweet] error: retrieving source for go-build: exit status 128
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1

watchflakes

@bcmills bcmills changed the title x/benchmarks/sweet/cmd/sweet: TestSweetEndToEnd timeout x/benchmarks/sweet/cmd/sweet: TestSweetEndToEnd failures Jan 11, 2023
@bcmills bcmills added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jan 11, 2023
@bcmills
Copy link
Contributor

bcmills commented Jan 11, 2023

(attn @golang/runtime, @golang/release per https://dev.golang.org/owners)

@mknyszek mknyszek self-assigned this Jan 13, 2023
@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-01-26 18:16 linux-amd64-longtest benchmarks@565f2d42 go@dd44d981 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (429.49s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs per config (1))
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [sweet] error: retrieving source for go-build: exit status 128
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1
2023-01-26 18:16 linux-amd64-longtest-race benchmarks@565f2d42 go@dd44d981 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (440.34s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs per config (1))
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [sweet] error: retrieving source for go-build: exit status 128
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-02-23 23:03 linux-amd64-longtest-race benchmarks@11773106 go@4a9c84d0 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (285.68s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [sweet] error: retrieving source for go-build: exit status 128
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-03-07 14:28 linux-amd64-longtest-race benchmarks@2652dad2 go@92261b38 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (703.23s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [sweet] error: retrieving source for go-build: exit status 128
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-03-07 14:28 linux-amd64-longtest-race benchmarks@2652dad2 go@1a64574f x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (467.20s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [sweet] error: retrieving source for go-build: exit status 128
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-03-07 14:28 linux-amd64-longtest benchmarks@2652dad2 go@94850c6f x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (396.66s)
    integration_test.go:148: output for bleve-index:
    integration_test.go:156: BenchmarkBleveIndexBatch100 1 203132639 ns/op 32378880 average-RSS-bytes 34869248 peak-RSS-bytes 1867689984 peak-VM-bytes

    integration_test.go:148: output for gopher-lua:
    integration_test.go:156: A 30.297
        T 30.151
        C 19.798
        G 19.755

        panic: runtime error: invalid memory address or nil pointer dereference
        [signal SIGSEGV: segmentation violation code=0x1 addr=0x12 pc=0x4fea75]

        goroutine 21 [running]:
        golang.org/x/benchmarks/sweet/benchmarks/internal/driver.init.func3.1()
        	/workdir/gopath/src/golang.org/x/benchmarks/sweet/benchmarks/internal/driver/driver.go:49 +0x15
        golang.org/x/benchmarks/sweet/benchmarks/internal/driver.(*B).startRSSSampler.func1()
        	/workdir/gopath/src/golang.org/x/benchmarks/sweet/benchmarks/internal/driver/driver.go:297 +0x13a
        created by golang.org/x/benchmarks/sweet/benchmarks/internal/driver.(*B).startRSSSampler in goroutine 1
        	/workdir/gopath/src/golang.org/x/benchmarks/sweet/benchmarks/internal/driver/driver.go:287 +0x99
...
        [shell] /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go/bin/markdown-bench /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go/assets
        [shell] rm -rf /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go/tmp/*
        [shell] rm -rf /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go/assets/*
        [shell] unzip /tmp/go-sweet-assets/assets-v0.3.0.zip 'markdown/*' -d /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/assets
        [sweet] Running benchmark markdown for go.pgo: run 1
        [shell] /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/bin/markdown-bench /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/assets
        [shell] rm -rf /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/tmp/*
        [shell] rm -rf /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/assets/*
        [sweet] error: failed to execute benchmarks, see log for details
    integration_test.go:163: exit status 1
2023-03-07 14:28 linux-amd64-longtest benchmarks@2652dad2 go@2068d565 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (325.92s)
    integration_test.go:148: output for bleve-index:
    integration_test.go:156: BenchmarkBleveIndexBatch100 1 187695257 ns/op 31870976 average-RSS-bytes 35819520 peak-RSS-bytes 2094985216 peak-VM-bytes

    integration_test.go:148: output for gopher-lua:
    integration_test.go:156: A 30.297
        T 30.151
        C 19.798
        G 19.755

        panic: runtime error: invalid memory address or nil pointer dereference
        [signal SIGSEGV: segmentation violation code=0x1 addr=0x12 pc=0x4feb95]

        goroutine 14 [running]:
        golang.org/x/benchmarks/sweet/benchmarks/internal/driver.init.func3.1()
        	/workdir/gopath/src/golang.org/x/benchmarks/sweet/benchmarks/internal/driver/driver.go:49 +0x15
        golang.org/x/benchmarks/sweet/benchmarks/internal/driver.(*B).startRSSSampler.func1()
        	/workdir/gopath/src/golang.org/x/benchmarks/sweet/benchmarks/internal/driver/driver.go:297 +0x13a
        created by golang.org/x/benchmarks/sweet/benchmarks/internal/driver.(*B).startRSSSampler in goroutine 1
        	/workdir/gopath/src/golang.org/x/benchmarks/sweet/benchmarks/internal/driver/driver.go:287 +0x99
...
        [shell] /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go/bin/markdown-bench /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go/assets
        [shell] rm -rf /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go/tmp/*
        [shell] rm -rf /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go/assets/*
        [shell] unzip /tmp/go-sweet-assets/assets-v0.3.0.zip 'markdown/*' -d /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/assets
        [sweet] Running benchmark markdown for go.pgo: run 1
        [shell] /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/bin/markdown-bench /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/assets
        [shell] rm -rf /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/tmp/*
        [shell] rm -rf /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/assets/*
        [sweet] error: failed to execute benchmarks, see log for details
    integration_test.go:163: exit status 1
2023-03-07 14:28 linux-amd64-longtest benchmarks@2652dad2 go@f8162a0e x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (411.23s)
    integration_test.go:148: output for bleve-index:
    integration_test.go:156: BenchmarkBleveIndexBatch100 1 204211308 ns/op 32440320 average-RSS-bytes 34390016 peak-RSS-bytes 2170224640 peak-VM-bytes

    integration_test.go:148: output for gopher-lua:
    integration_test.go:156: A 30.297
        T 30.151
        C 19.798
        G 19.755

        panic: runtime error: invalid memory address or nil pointer dereference
        [signal SIGSEGV: segmentation violation code=0x1 addr=0x12 pc=0x4feb95]

        goroutine 10 [running]:
        golang.org/x/benchmarks/sweet/benchmarks/internal/driver.init.func3.1()
        	/workdir/gopath/src/golang.org/x/benchmarks/sweet/benchmarks/internal/driver/driver.go:49 +0x15
        golang.org/x/benchmarks/sweet/benchmarks/internal/driver.(*B).startRSSSampler.func1()
        	/workdir/gopath/src/golang.org/x/benchmarks/sweet/benchmarks/internal/driver/driver.go:297 +0x13a
        created by golang.org/x/benchmarks/sweet/benchmarks/internal/driver.(*B).startRSSSampler in goroutine 1
        	/workdir/gopath/src/golang.org/x/benchmarks/sweet/benchmarks/internal/driver/driver.go:287 +0x99
...
        [shell] /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go/bin/markdown-bench /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go/assets
        [shell] rm -rf /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go/tmp/*
        [shell] rm -rf /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go/assets/*
        [shell] unzip /tmp/go-sweet-assets/assets-v0.3.0.zip 'markdown/*' -d /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/assets
        [sweet] Running benchmark markdown for go.pgo: run 1
        [shell] /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/bin/markdown-bench /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/assets
        [shell] rm -rf /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/tmp/*
        [shell] rm -rf /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/assets/*
        [sweet] error: failed to execute benchmarks, see log for details
    integration_test.go:163: exit status 1
2023-03-07 14:28 linux-amd64-longtest-race benchmarks@2652dad2 go@94850c6f x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (391.74s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4
        [sweet] Benchmarks: bleve-index gopher-lua markdown (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: bleve-index
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/results-4/bleve-index
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/bleve-index/go.profile/bin
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/bleve-index/src
...
        [shell] cd /workdir/gopath/src/golang.org/x/benchmarks/sweet/benchmarks/markdown
        [shell] /workdir/go/bin/go build -o /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.profile/bin/markdown-bench
        [shell] cd /workdir/gopath/src/golang.org/x/benchmarks/sweet/cmd/sweet
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/results-4/markdown/go.profile.debug
        [shell] unzip /tmp/go-sweet-assets/assets-v0.3.0.zip 'markdown/*' -d /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.profile/assets
        [sweet] Running benchmark markdown for go.profile: run 1
        [shell] /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.profile/bin/markdown-bench -cpuprofile /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/results-4/markdown/go.profile.debug /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.profile/assets
        [sweet] error: run benchmark markdown for config go.profile: exit status 2
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1
2023-03-07 14:28 linux-amd64-longtest-race benchmarks@2652dad2 go@2068d565 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (391.74s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4
        [sweet] Benchmarks: bleve-index gopher-lua markdown (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: bleve-index
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/results-4/bleve-index
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/bleve-index/go.profile/bin
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/bleve-index/src
...
        [shell] cd /workdir/gopath/src/golang.org/x/benchmarks/sweet/benchmarks/markdown
        [shell] /workdir/go/bin/go build -o /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.profile/bin/markdown-bench
        [shell] cd /workdir/gopath/src/golang.org/x/benchmarks/sweet/cmd/sweet
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/results-4/markdown/go.profile.debug
        [shell] unzip /tmp/go-sweet-assets/assets-v0.3.0.zip 'markdown/*' -d /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.profile/assets
        [sweet] Running benchmark markdown for go.profile: run 1
        [shell] /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.profile/bin/markdown-bench -cpuprofile /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/results-4/markdown/go.profile.debug /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.profile/assets
        [sweet] error: run benchmark markdown for config go.profile: exit status 2
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1
2023-03-07 14:28 linux-amd64-longtest-race benchmarks@2652dad2 go@f8162a0e x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (444.43s)
    integration_test.go:148: output for bleve-index:
    integration_test.go:156: BenchmarkBleveIndexBatch100 1 199809554 ns/op 29319168 average-RSS-bytes 35770368 peak-RSS-bytes 2170814464 peak-VM-bytes

    integration_test.go:148: output for gopher-lua:
    integration_test.go:156: A 30.297
        T 30.151
        C 19.798
        G 19.755

...
        fatal error: found pointer to free object

        runtime stack:
        runtime.throw({0x5978a4?, 0xc0000b2f00?})
        	/workdir/go/src/runtime/panic.go:1076 +0x5c fp=0xc00009bce0 sp=0xc00009bcb0 pc=0x43491c
        runtime.(*mspan).reportZombies(0x7fdc8766f368)
        	/workdir/go/src/runtime/mgcsweep.go:846 +0x2e5 fp=0xc00009bd60 sp=0xc00009bce0 pc=0x424545
        runtime.(*sweepLocked).sweep(0x2030003?, 0x0)
        	/workdir/go/src/runtime/mgcsweep.go:629 +0x339 fp=0xc00009be68 sp=0xc00009bd60 pc=0x423619
        runtime.(*mcentral).uncacheSpan(0xc00009bee0?, 0x42b34e?)
...
        [shell] /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go/bin/markdown-bench /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go/assets
        [shell] rm -rf /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go/tmp/*
        [shell] rm -rf /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go/assets/*
        [shell] unzip /tmp/go-sweet-assets/assets-v0.3.0.zip 'markdown/*' -d /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/assets
        [sweet] Running benchmark markdown for go.pgo: run 1
        [shell] /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/bin/markdown-bench /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/assets
        [shell] rm -rf /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/tmp/*
        [shell] rm -rf /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/markdown/go.pgo/assets/*
        [sweet] error: failed to execute benchmarks, see log for details
    integration_test.go:163: exit status 1

watchflakes

@mknyszek
Copy link
Contributor

These latest failures are fixed by https://go.dev/cl/485498.

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-03-07 14:28 linux-amd64-longtest benchmarks@2652dad2 go@14ab998f x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (340.99s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [sweet] error: retrieving source for go-build: exit status 128
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-05-10 17:32 linux-amd64-longtest-race benchmarks@439d592e go@956d31ec x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
panic: test timed out after 50m0s
running tests:
	TestSweetEndToEnd (50m0s)

sync.runtime_Semacquire(0xc000014528?)
	/workdir/go/src/runtime/sema.go:62 +0x25
sync.(*WaitGroup).Wait(0xc000014520)
	/workdir/go/src/sync/waitgroup.go:116 +0xa5
golang.org/x/benchmarks/sweet/cmd/sweet_test.TestSweetEndToEnd(0xc0004489c0)
	/workdir/gopath/src/golang.org/x/benchmarks/sweet/cmd/sweet/integration_test.go:185 +0x15bb
testing.tRunner(0xc0004489c0, 0xd624e8)

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-05-10 17:32 linux-amd64-longtest-race benchmarks@439d592e go@374bd6a3 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (442.46s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [sweet] error: retrieving source for go-build: exit status 128
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-05-10 17:32 linux-amd64-longtest benchmarks@439d592e go@8c445b7c x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (698.34s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [sweet] error: retrieving source for go-build: exit status 128
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1
2023-05-10 17:32 linux-amd64-longtest-race benchmarks@439d592e go@8c445b7c x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (691.12s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [sweet] error: retrieving source for go-build: exit status 128
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-05-10 17:32 linux-amd64-longtest-race benchmarks@439d592e go@2ed6a54a x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (391.90s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [sweet] error: retrieving source for go-build: exit status 128
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1

watchflakes

@mknyszek
Copy link
Contributor

This seems to be getting worse. Sorry for the delay. I'll take a closer look very soon.

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-05-24 17:54 linux-amd64-longtest-race benchmarks@c4c5b3d1 go@08458804 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (381.43s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [sweet] error: retrieving source for go-build: exit status 128
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-05-24 17:54 linux-amd64-longtest benchmarks@c4c5b3d1 go@82d5ebce x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (297.93s)
    integration_test.go:140: no results produced.
    integration_test.go:162: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone --depth 1 -b v1.22.1 https://github.com/kubernetes/kubernetes /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/kubernetes
        [shell] git clone --depth 1 -b 1.11.1 https://github.com/istio/istio /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/istio
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [sweet] error: retrieving source for go-build: exit status 128
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:163: exit status 1

watchflakes

@mknyszek
Copy link
Contributor

OK this seems to consistently be an issue with cloning repositories from GitHub. I think this might actually be an easy fix, since we already only benchmark building pkgsite in these tests. We just erroneously also clone this other repos. (And GitHub is probably rate-limiting us, or something.)

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-06-27 19:24 linux-amd64-longtest benchmarks@6468fc12 go@089e37a9 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (181.61s)
    integration_test.go:36: phase setup @0s (duration: 768.633314ms)
    integration_test.go:36: phase sweet-get @768.633314ms (duration: 34.871397026s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @35.64017699s (duration: 52.725813198s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @35.640274197s (duration: 58.493207588s)
    integration_test.go:131: 	phase sweet-run-bleve-index @1m34.133536435s (duration: 25.83265054s)
    integration_test.go:161: no results produced.
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-0
        [sweet] Benchmarks: tile38 (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: tile38
        [shell] git clone --depth 1 -b 1.29.1 https://github.com/tidwall/tile38 /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-0/tile38/src
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/results-0/tile38
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-0/tile38/go.profile/bin
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-0/tile38/src
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-0/tile38/go.profile/tmp
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-0/tile38/go.profile/assets
        [shell] GOFLAGS=-pgo=off PATH=/workdir/go/bin:/workdir/go/bin:/workdir/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin make -C /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-0/tile38/src
        [sweet] error: build tile38 for go.profile: exit status 2
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-tile38 @35.64031128s (duration: 1m30.498092712s)
    integration_test.go:131: 	phase sweet-run-markdown @2m6.138431571s (duration: 18.116087902s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @1m59.966231935s (duration: 29.471001536s)
    integration_test.go:131: 	phase sweet-run-go-build @35.640227394s (duration: 2m4.896556886s)
    integration_test.go:131: 	phase sweet-run-etcd @1m28.366054747s (duration: 1m32.664405253s)
    integration_test.go:36: phase sweet-run @35.64003034s (duration: 2m25.39048386s)

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-06-27 19:24 linux-amd64-longtest-race benchmarks@6468fc12 go@c3da3bcd x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (166.76s)
    integration_test.go:36: phase setup @0s (duration: 9.474778481s)
    integration_test.go:36: phase sweet-get @9.474778481s (duration: 33.301882292s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @42.777331994s (duration: 1m2.841555988s)
    integration_test.go:161: no results produced.
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4
        [sweet] Benchmarks: etcd (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: etcd
        [shell] git clone --depth 1 -b v3.6.0-alpha.0 https://github.com/etcd-io/etcd /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/etcd/src
        [sweet] error: retrieving source for etcd: exit status 128
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-etcd @1m45.619031481s (duration: 2.228616113s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @42.777070004s (duration: 1m10.61726595s)
    integration_test.go:131: 	phase sweet-run-bleve-index @1m47.847759873s (duration: 27.928862387s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @1m53.394455994s (duration: 22.536088496s)
    integration_test.go:131: 	phase sweet-run-tile38 @42.777109434s (duration: 1m33.580424849s)
    integration_test.go:131: 	phase sweet-run-markdown @2m15.77672567s (duration: 18.0275475s)
    integration_test.go:131: 	phase sweet-run-go-build @42.777069964s (duration: 2m3.320943254s)
    integration_test.go:36: phase sweet-run @42.776660773s (duration: 2m3.321430235s)

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-06-27 19:24 linux-amd64-longtest benchmarks@6468fc12 go@2eca0b1e x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (242.76s)
    integration_test.go:36: phase setup @0s (duration: 1.13501001s)
    integration_test.go:36: phase sweet-get @1.13501001s (duration: 39.024184777s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @40.159373273s (duration: 1m37.557152152s)
    integration_test.go:161: no results produced.
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4
        [sweet] Benchmarks: etcd (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: etcd
        [shell] git clone --depth 1 -b v3.6.0-alpha.0 https://github.com/etcd-io/etcd /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/etcd/src
        [sweet] error: retrieving source for etcd: exit status 128
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-etcd @2m17.71659701s (duration: 411.52093ms)
    integration_test.go:131: 	phase sweet-run-biogo-igor @40.15955967s (duration: 2m6.708912273s)
    integration_test.go:131: 	phase sweet-run-bleve-index @2m18.128147086s (duration: 47.407138608s)
    integration_test.go:131: 	phase sweet-run-tile38 @40.159608805s (duration: 2m27.514653884s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @2m46.868536714s (duration: 29.906676013s)
    integration_test.go:131: 	phase sweet-run-markdown @3m5.535338971s (duration: 21.058319315s)
    integration_test.go:131: 	phase sweet-run-go-build @40.159495419s (duration: 3m21.692207726s)
    integration_test.go:36: phase sweet-run @40.159194787s (duration: 3m21.692541902s)

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-06-27 19:24 linux-amd64-longtest benchmarks@6468fc12 go@2ad666de x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (181.39s)
    integration_test.go:36: phase setup @0s (duration: 768.023634ms)
    integration_test.go:36: phase sweet-get @768.023634ms (duration: 55.227327418s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @55.995520043s (duration: 51.679721172s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @55.995611802s (duration: 57.670218538s)
    integration_test.go:161: no results produced.
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
        [sweet] error: retrieving source for go-build: exit status 128
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-go-build @55.995531712s (duration: 1m7.67887249s)
    integration_test.go:131: 	phase sweet-run-tile38 @55.995650673s (duration: 1m20.048231657s)
    integration_test.go:131: 	phase sweet-run-bleve-index @1m53.66589834s (duration: 22.447349873s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @2m3.674440912s (duration: 18.661187371s)
    integration_test.go:131: 	phase sweet-run-markdown @2m16.04392155s (duration: 10.663961772s)
    integration_test.go:131: 	phase sweet-run-etcd @1m47.675288545s (duration: 1m13.339873078s)
    integration_test.go:36: phase sweet-run @55.995351052s (duration: 2m5.019848841s)

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-06-27 19:24 linux-amd64-longtest benchmarks@6468fc12 go@008ab9ad x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (218.82s)
    integration_test.go:36: phase setup @0s (duration: 1.187347884s)
    integration_test.go:36: phase sweet-get @1.187347884s (duration: 42.207681712s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @43.395225103s (duration: 1m29.799743379s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @43.395323082s (duration: 1m40.159842466s)
    integration_test.go:161: no results produced.
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4
        [sweet] Benchmarks: etcd (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
...
        [sweet] error: run benchmark etcd for config go.profile: exit status 1
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-etcd @2m13.195015122s (duration: 44.637625909s)
    integration_test.go:131: 	phase sweet-run-tile38 @43.395383979s (duration: 2m16.754422677s)
    integration_test.go:131: 	phase sweet-run-bleve-index @2m23.555213812s (duration: 42.934524294s)
    integration_test.go:131: 	phase sweet-run-markdown @3m0.149859853s (duration: 17.763583188s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @2m57.832667415s (duration: 38.06194982s)
    integration_test.go:131: 	phase sweet-run-go-build @43.395234142s (duration: 2m54.463070102s)
    integration_test.go:36: phase sweet-run @43.395029596s (duration: 2m54.463310931s)
2023-06-27 19:24 linux-amd64-longtest benchmarks@6468fc12 go@aca65771 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (203.91s)
    integration_test.go:36: phase setup @0s (duration: 1.177783325s)
    integration_test.go:36: phase sweet-get @1.177783325s (duration: 35.192711401s)
    integration_test.go:161: no results produced.
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
...
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-go-build @36.370821957s (duration: 1m7.281499005s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @36.370741038s (duration: 1m28.226669038s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @36.370921787s (duration: 1m38.308456184s)
    integration_test.go:131: 	phase sweet-run-bleve-index @2m4.597448969s (duration: 37.753215333s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @2m14.679429874s (duration: 31.008242248s)
    integration_test.go:131: 	phase sweet-run-tile38 @36.370761218s (duration: 2m14.780648334s)
    integration_test.go:131: 	phase sweet-run-markdown @2m42.350733713s (duration: 12.01906835s)
    integration_test.go:131: 	phase sweet-run-etcd @1m43.65235981s (duration: 1m39.511699963s)
    integration_test.go:36: phase sweet-run @36.370494726s (duration: 2m46.793608184s)
2023-06-27 19:24 linux-amd64-longtest-race benchmarks@6468fc12 go@5f674f64 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (225.94s)
    integration_test.go:36: phase setup @0s (duration: 13.508228099s)
    integration_test.go:36: phase sweet-get @13.508228099s (duration: 38.685734285s)
    integration_test.go:161: no results produced.
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
...
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-go-build @52.195159048s (duration: 1m10.214966563s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @52.194668829s (duration: 1m32.681050148s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @52.194607546s (duration: 1m45.11965943s)
    integration_test.go:131: 	phase sweet-run-bleve-index @2m24.875863775s (duration: 42.529850243s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @2m37.314402946s (duration: 32.543194388s)
    integration_test.go:131: 	phase sweet-run-tile38 @52.194638981s (duration: 2m21.198977086s)
    integration_test.go:131: 	phase sweet-run-markdown @3m7.405917411s (duration: 19.915302875s)
    integration_test.go:131: 	phase sweet-run-etcd @2m2.410263699s (duration: 1m42.70187684s)
    integration_test.go:36: phase sweet-run @52.193962384s (duration: 2m52.91829379s)
2023-06-27 19:24 linux-amd64-longtest-race benchmarks@6468fc12 go@008ab9ad x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (223.48s)
    integration_test.go:36: phase setup @0s (duration: 13.04935933s)
    integration_test.go:36: phase sweet-get @13.04935933s (duration: 34.687707541s)
    integration_test.go:161: no results produced.
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
...
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-go-build @47.737985512s (duration: 1m14.957378924s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @47.737704747s (duration: 1m31.397870785s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @47.737798993s (duration: 1m42.404455084s)
    integration_test.go:131: 	phase sweet-run-bleve-index @2m19.135752747s (duration: 40.918118266s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @2m30.142442671s (duration: 31.951445743s)
    integration_test.go:131: 	phase sweet-run-tile38 @47.738102908s (duration: 2m16.050437011s)
    integration_test.go:131: 	phase sweet-run-markdown @3m0.054050688s (duration: 18.771001327s)
    integration_test.go:131: 	phase sweet-run-etcd @2m2.69552251s (duration: 1m39.982513611s)
    integration_test.go:36: phase sweet-run @47.737066871s (duration: 2m54.941037259s)

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-08-18 22:04 linux-amd64-longtest benchmarks@c7f59e45 go@463c3445 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (208.73s)
    integration_test.go:36: phase setup @0s (duration: 1.125027403s)
    integration_test.go:36: phase sweet-get @1.125027403s (duration: 52.502801352s)
    integration_test.go:161: no results produced.
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: go-build
        [shell] git clone -b master https://go.googlesource.com/pkgsite /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1/go-build/src/pkgsite
...
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-go-build @53.628058868s (duration: 50.648384867s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @53.628045022s (duration: 1m22.936495504s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @53.628122166s (duration: 1m27.479492071s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @2m21.107686291s (duration: 31.739239227s)
    integration_test.go:131: 	phase sweet-run-bleve-index @2m16.565992878s (duration: 37.793933016s)
    integration_test.go:131: 	phase sweet-run-markdown @2m52.84700329s (duration: 12.750054055s)
    integration_test.go:131: 	phase sweet-run-tile38 @53.628230713s (duration: 2m14.944633699s)
    integration_test.go:131: 	phase sweet-run-etcd @1m44.276476715s (duration: 1m43.710256445s)
    integration_test.go:36: phase sweet-run @53.627828755s (duration: 2m34.358945975s)

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-11-08 21:52 linux-amd64-longtest-race benchmarks@444331c9 go@acc8cb6a x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (264.59s)
    integration_test.go:36: phase setup @0s (duration: 14.059644724s)
    integration_test.go:36: phase sweet-get @14.059644724s (duration: 36.730541437s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @50.790800618s (duration: 1m43.754908346s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @50.791178099s (duration: 1m55.339711983s)
    integration_test.go:169: output for go-build:
    integration_test.go:177: 
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
...
        <autogenerated>:1: internal compiler error: 'BinaryUnmarshaler.UnmarshalBinary': panic during expand calls while compiling BinaryUnmarshaler.UnmarshalBinary:

        runtime error: invalid memory address or nil pointer dereference

        goroutine 6 [running]:
        cmd/compile/internal/ssa.Compile.func1()
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/ssa/compile.go:49 +0x6c
        panic({0xd7b920?, 0x148b6d0?})
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/runtime/panic.go:763 +0x132
        cmd/compile/internal/ssa.(*expandState).rewriteSelectOrArg(0xc002b03990, {0x1, 0x1002}, 0xc0005043b0, 0xc0004cdef0, 0xc0004ce200, 0xc0004cde80, 0xc0004786c0, {0x0, 0x8, ...})
...
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/gc/compile.go:163 +0x247
        [sweet] error: failed to execute benchmarks, see log for details
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-go-build @50.790723624s (duration: 2m7.676946396s)
    integration_test.go:131: 	phase sweet-run-tile38 @50.790762456s (duration: 2m19.15877389s)
    integration_test.go:131: 	phase sweet-run-bleve-index @2m46.131105796s (duration: 40.884089165s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @2m58.467773559s (duration: 30.351085682s)
    integration_test.go:131: 	phase sweet-run-markdown @3m9.949684763s (duration: 21.205247701s)
    integration_test.go:131: 	phase sweet-run-etcd @2m34.545916211s (duration: 1m48.939298427s)
    integration_test.go:36: phase sweet-run @50.790186161s (duration: 3m32.695116101s)
2023-11-08 21:52 linux-amd64-longtest-race benchmarks@444331c9 go@31f0af1e x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (263.49s)
    integration_test.go:36: phase setup @0s (duration: 13.855073898s)
    integration_test.go:36: phase sweet-get @13.855073898s (duration: 36.945259805s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @50.801041924s (duration: 1m43.430555928s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @50.801352517s (duration: 1m54.370569365s)
    integration_test.go:169: output for go-build:
    integration_test.go:177: 
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
...
        panic: runtime error: invalid memory address or nil pointer dereference
        [signal SIGSEGV: segmentation violation code=0x1 addr=0x9 pc=0x416b57]

        goroutine 33 [running]:
        cmd/compile/internal/ssagen.(*state).addNamedValue(...)
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/ssagen/ssa.go:6934
        cmd/compile/internal/ssagen.(*state).assignWhichMayOverlap(0xc00007e000, {0xf75118, 0xc0005c86e0}, 0xc00069e000, 0x0, 0x0, 0x0)
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/ssagen/ssa.go:3940 +0x809
        cmd/compile/internal/ssagen.(*state).stmt(0xc00007e000, {0xf75298, 0xc0005ce000})
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/ssagen/ssa.go:1714 +0x8c2a
...
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/gc/compile.go:163 +0x247
        [sweet] error: failed to execute benchmarks, see log for details
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-go-build @50.80098904s (duration: 2m5.742621781s)
    integration_test.go:131: 	phase sweet-run-tile38 @50.801224s (duration: 2m17.099488241s)
    integration_test.go:131: 	phase sweet-run-bleve-index @2m45.172138739s (duration: 41.291185839s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @2m56.543725435s (duration: 31.291321667s)
    integration_test.go:131: 	phase sweet-run-markdown @3m7.900837437s (duration: 21.094699367s)
    integration_test.go:131: 	phase sweet-run-etcd @2m34.231861062s (duration: 1m48.152644711s)
    integration_test.go:36: phase sweet-run @50.800333703s (duration: 3m31.584269357s)
2023-11-08 21:52 linux-amd64-longtest-race benchmarks@444331c9 go@dd88f23a x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (292.57s)
    integration_test.go:36: phase setup @0s (duration: 15.407251654s)
    integration_test.go:36: phase sweet-get @15.407251654s (duration: 38.471946561s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @53.879883978s (duration: 1m58.075893028s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @53.880203547s (duration: 2m3.667350191s)
    integration_test.go:169: output for go-build:
    integration_test.go:177: 
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
...
        fatal error: fault
        [signal SIGSEGV: segmentation violation code=0x1 addr=0x1800577a320 pc=0x47c8ff]

        goroutine 8 [running]:
        runtime.throw({0xdffae1?, 0x7fefb82437c8?})
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/runtime/panic.go:1016 +0x5c fp=0xc0000a6728 sp=0xc0000a66f8 pc=0x44695c
        runtime.sigpanic()
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/runtime/signal_unix.go:880 +0x285 fp=0xc0000a6788 sp=0xc0000a6728 pc=0x45e765
        aeshashbody()
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/runtime/asm_amd64.s:1466 +0x39f fp=0xc0000a6790 sp=0xc0000a6788 pc=0x47c8ff
...
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/gc/compile.go:163 +0x247
        [sweet] error: failed to execute benchmarks, see log for details
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-go-build @53.880007646s (duration: 2m27.183562193s)
    integration_test.go:131: 	phase sweet-run-tile38 @53.879874145s (duration: 2m39.930977882s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @3m21.063689354s (duration: 33.448571457s)
    integration_test.go:131: 	phase sweet-run-bleve-index @2m57.547793442s (duration: 57.447411892s)
    integration_test.go:131: 	phase sweet-run-markdown @3m33.811158138s (duration: 25.093782229s)
    integration_test.go:131: 	phase sweet-run-etcd @2m51.956031523s (duration: 1m57.666870308s)
    integration_test.go:36: phase sweet-run @53.879198215s (duration: 3m55.7438056s)

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-11-08 21:52 linux-amd64-longtest-race benchmarks@444331c9 go@3073f3f9 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (266.19s)
    integration_test.go:36: phase setup @0s (duration: 13.80441237s)
    integration_test.go:36: phase sweet-get @13.80441237s (duration: 39.512148632s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @53.317580879s (duration: 1m43.128894734s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @53.317240974s (duration: 1m55.915230187s)
    integration_test.go:169: output for go-build:
    integration_test.go:177: 
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
...
        <autogenerated>:1: internal compiler error: '.eq.internal/coverage/rtcov.CovMetaBlob': panic during prove while compiling .eq.internal/coverage/rtcov.CovMetaBlob:

        runtime error: invalid memory address or nil pointer dereference

        goroutine 21 [running]:
        cmd/compile/internal/ssa.Compile.func1()
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/ssa/compile.go:49 +0x6c
        panic({0xd7ac00?, 0x14896f0?})
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/runtime/panic.go:763 +0x132
        cmd/compile/internal/ssa.(*factsTable).restore(...)
...
        cmd/compile/internal/ssagen.Compile(0xc0031330e0, 0x1)
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/ssagen/pgen.go:216 +0x3e fp=0xc00256df70 sp=0xc00256dea0 pc=0xab19fe
        cmd/compile/internal/gc.compileFunctions.func5.1(0xc0003f0280?)
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/gc/compile.go:182 +0x34 fp=0xc00256dfb0 sp=0xc00256df70 pc=0xcf4054
        cmd/compile/internal/gc.compileFunctions.func3.1()
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/gc/compile.go:164 +0x30 fp=0xc00256dfe0 sp=0xc00256dfb0 pc=0xcf4470
        runtime.goexit({})
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/runtime/asm_amd64.s:1694 +0x1 fp=0xc00256dfe8 sp=0xc00256dfe0 pc=0x47dd41
        created by cmd/compile/internal/gc.compileFunctions.func3 in goroutine 33
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/gc/compile.go:163 +0x247
2023-11-08 21:52 linux-amd64-longtest-race benchmarks@444331c9 go@67d555e0 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (252.53s)
    integration_test.go:36: phase setup @0s (duration: 12.758505236s)
    integration_test.go:36: phase sweet-get @12.758505236s (duration: 43.179259231s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @55.938395033s (duration: 1m36.566055233s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @55.938311042s (duration: 1m52.576782752s)
    integration_test.go:169: output for go-build:
    integration_test.go:177: 
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
...
        panic: runtime error: invalid memory address or nil pointer dereference
        [signal SIGSEGV: segmentation violation code=0x1 addr=0x9 pc=0x416f57]

        goroutine 18 [running]:
        cmd/compile/internal/ssagen.(*state).addNamedValue(...)
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/ssagen/ssa.go:6934
        cmd/compile/internal/ssagen.(*state).assignWhichMayOverlap(0xc002742a00, {0xf70a38, 0xc002762790}, 0xc00056b160, 0x0, 0x0, 0x0)
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/ssagen/ssa.go:3940 +0x8c9
        cmd/compile/internal/ssagen.(*state).stmt(0xc002742a00, {0xf720b8, 0xc00275ce10})
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/ssagen/ssa.go:1714 +0x76da
...
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/gc/compile.go:163 +0x247
        [sweet] error: failed to execute benchmarks, see log for details
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-go-build @55.93857561s (duration: 2m1.775965196s)
    integration_test.go:131: 	phase sweet-run-tile38 @55.938347151s (duration: 2m13.544670698s)
    integration_test.go:131: 	phase sweet-run-bleve-index @2m48.515371019s (duration: 40.731165215s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @2m57.714658134s (duration: 31.954755523s)
    integration_test.go:131: 	phase sweet-run-markdown @3m9.48316064s (duration: 22.518913555s)
    integration_test.go:131: 	phase sweet-run-etcd @2m32.504597016s (duration: 1m38.964859655s)
    integration_test.go:36: phase sweet-run @55.937764467s (duration: 3m15.531790198s)

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-11-08 21:52 linux-amd64-longtest benchmarks@444331c9 go@6ef98ac8 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (245.74s)
    integration_test.go:36: phase setup @0s (duration: 1.26589676s)
    integration_test.go:36: phase sweet-get @1.26589676s (duration: 41.44978289s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @42.716265893s (duration: 1m45.655662582s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @42.716041195s (duration: 1m55.783318267s)
    integration_test.go:169: output for go-build:
    integration_test.go:177: 
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
...
        <autogenerated>:1: internal compiler error: '.eq.internal/coverage/rtcov.CovMetaBlob': panic during prove while compiling .eq.internal/coverage/rtcov.CovMetaBlob:

        runtime error: invalid memory address or nil pointer dereference

        goroutine 18 [running]:
        cmd/compile/internal/ssa.Compile.func1()
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/ssa/compile.go:49 +0x6c
        panic({0xd7d8e0?, 0x1490690?})
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/runtime/panic.go:763 +0x132
        cmd/compile/internal/ssa.(*factsTable).update(0xc002b73968, 0xc000520220, 0x0, 0xc0004f0c60, 0x8, 0x5)
...
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/gc/compile.go:163 +0x247
        [sweet] error: failed to execute benchmarks, see log for details
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-go-build @42.716058324s (duration: 2m10.642964602s)
    integration_test.go:131: 	phase sweet-run-tile38 @42.71610869s (duration: 2m21.518052052s)
    integration_test.go:131: 	phase sweet-run-bleve-index @2m38.499437963s (duration: 39.379610011s)
    integration_test.go:131: 	phase sweet-run-markdown @3m4.234241376s (duration: 14.630075799s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @2m53.359078149s (duration: 31.264638478s)
    integration_test.go:131: 	phase sweet-run-etcd @2m28.372056634s (duration: 1m36.381165807s)
    integration_test.go:36: phase sweet-run @42.71567965s (duration: 3m22.037580941s)

watchflakes

@mknyszek mknyszek removed their assignment Nov 15, 2023
@bcmills
Copy link
Contributor

bcmills commented Nov 15, 2023

The earlier git failures could be a symptom of port exhaustion (see #52545 (comment)).

The new clusters of failures are mostly segfaults in the compiler, though. 🤔

@prattmic
Copy link
Member

The later failures start on the 13th and are during PGO builds, so they are probably related to one of the PGO CLs I submitted on the 13th: https://go.dev/cl/539699 and child. I'll take a look.

@gopherbot
Copy link
Author

Change https://go.dev/cl/542621 mentions this issue: DO NOT SUBMIT: attempt to reproduce PGO ICE

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-11-08 21:52 linux-amd64-longtest-race benchmarks@444331c9 go@2f5bd4e4 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (261.94s)
    integration_test.go:36: phase setup @0s (duration: 13.571490133s)
    integration_test.go:36: phase sweet-get @13.571490133s (duration: 44.147228088s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @57.719547726s (duration: 1m41.230247251s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @57.71933239s (duration: 1m54.294601841s)
    integration_test.go:169: output for go-build:
    integration_test.go:177: 
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
...
        fatal error: fault
        [signal SIGSEGV: segmentation violation code=0x1 addr=0x180026e8690 pc=0x47b67f]

        goroutine 21 [running]:
        runtime.throw({0xdf0db7?, 0xd8d100?})
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/runtime/panic.go:1023 +0x5c fp=0xc0022295a0 sp=0xc002229570 pc=0x445fbc
        runtime.sigpanic()
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/runtime/signal_unix.go:880 +0x285 fp=0xc002229600 sp=0xc0022295a0 pc=0x45d945
        aeshashbody()
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/runtime/asm_amd64.s:1466 +0x39f fp=0xc002229608 sp=0xc002229600 pc=0x47b67f
...
        cmd/compile/internal/ssagen.Compile(0xc00306aea0, 0x1)
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/ssagen/pgen.go:216 +0x45 fp=0xc0029f9f70 sp=0xc0029f9dd0 pc=0xab14e5
        cmd/compile/internal/gc.compileFunctions.func5.1(0x0?)
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/gc/compile.go:182 +0x34 fp=0xc0029f9fb0 sp=0xc0029f9f70 pc=0xcf0d94
        cmd/compile/internal/gc.compileFunctions.func3.1()
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/gc/compile.go:164 +0x30 fp=0xc0029f9fe0 sp=0xc0029f9fb0 pc=0xcf11b0
        runtime.goexit({})
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/runtime/asm_amd64.s:1694 +0x1 fp=0xc0029f9fe8 sp=0xc0029f9fe0 pc=0x47df41
        created by cmd/compile/internal/gc.compileFunctions.func3 in goroutine 44
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/gc/compile.go:163 +0x247

watchflakes

@gopherbot
Copy link
Author

Change https://go.dev/cl/543115 mentions this issue: cmd/compile: disable PGO function value devirtualization by default

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-11-08 21:52 linux-amd64-longtest-race benchmarks@444331c9 go@d411b319 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (200.11s)
    integration_test.go:36: phase setup @0s (duration: 9.719636039s)
    integration_test.go:36: phase sweet-get @9.719636039s (duration: 37.558436324s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @47.278894843s (duration: 1m11.087433269s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @47.278482993s (duration: 1m18.730483883s)
    integration_test.go:169: output for go-build:
    integration_test.go:177: 
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
...
        <autogenerated>:1: internal compiler error: '.eq.internal/coverage/rtcov.CovMetaBlob': panic during prove while compiling .eq.internal/coverage/rtcov.CovMetaBlob:

        runtime error: invalid memory address or nil pointer dereference

        goroutine 21 [running]:
        cmd/compile/internal/ssa.Compile.func1()
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/ssa/compile.go:49 +0x6c
        panic({0xd6b9c0?, 0x1472690?})
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/runtime/panic.go:770 +0x132
        cmd/compile/internal/ssa.(*factsTable).update(0xc001f75930, 0xc0005a0220, 0x0, 0xc000570c60, 0x8, 0x5)
...
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/gc/compile.go:163 +0x247
        [sweet] error: failed to execute benchmarks, see log for details
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-go-build @47.278688513s (duration: 1m29.823751561s)
    integration_test.go:131: 	phase sweet-run-tile38 @47.278773013s (duration: 1m37.188254526s)
    integration_test.go:131: 	phase sweet-run-bleve-index @2m6.009115046s (duration: 27.159983066s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @2m17.102547164s (duration: 20.084350647s)
    integration_test.go:131: 	phase sweet-run-markdown @2m24.467116339s (duration: 13.451645717s)
    integration_test.go:131: 	phase sweet-run-etcd @1m58.366531953s (duration: 1m21.022715924s)
    integration_test.go:36: phase sweet-run @47.278072363s (duration: 2m32.111252454s)

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2023-11-08 21:52 linux-amd64-longtest benchmarks@444331c9 go@3f04f959 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (239.32s)
    integration_test.go:36: phase setup @0s (duration: 1.104761613s)
    integration_test.go:36: phase sweet-get @1.104761613s (duration: 42.407145471s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @43.51242766s (duration: 1m37.672387803s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @43.512183791s (duration: 1m50.647647204s)
    integration_test.go:169: output for go-build:
    integration_test.go:177: 
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-1
        [sweet] Benchmarks: go-build (1 runs, 1 pgo runs)
...
        fatal error: fault
        [signal SIGSEGV: segmentation violation code=0x1 addr=0x18004dc5410 pc=0x47cabf]

        goroutine 21 [running]:
        runtime.throw({0xdf6097?, 0x7f22358e49a8?})
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/runtime/panic.go:1023 +0x5c fp=0xc0001595d8 sp=0xc0001595a8 pc=0x4471dc
        runtime.sigpanic()
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/runtime/signal_unix.go:880 +0x285 fp=0xc000159638 sp=0xc0001595d8 pc=0x45e9e5
        aeshashbody()
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/runtime/asm_amd64.s:1466 +0x39f fp=0xc000159640 sp=0xc000159638 pc=0x47cabf
...
        	../../tmp/tmp-1/go-build/go.pgo/bin/goroot/src/cmd/compile/internal/gc/compile.go:163 +0x247
        [sweet] error: failed to execute benchmarks, see log for details
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-go-build @43.512209811s (duration: 1m59.913339829s)
    integration_test.go:131: 	phase sweet-run-tile38 @43.51219563s (duration: 2m11.999332477s)
    integration_test.go:131: 	phase sweet-run-markdown @2m55.511597747s (duration: 12.808431115s)
    integration_test.go:131: 	phase sweet-run-bleve-index @2m34.159903886s (duration: 36.699448691s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @2m43.42558986s (duration: 30.201363523s)
    integration_test.go:131: 	phase sweet-run-etcd @2m21.184930325s (duration: 1m37.192407567s)
    integration_test.go:36: phase sweet-run @43.511907084s (duration: 3m14.865471091s)

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2024-01-08 19:14 linux-amd64-longtest benchmarks@921b81ae go@c4696665 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (263.90s)
    integration_test.go:36: phase setup @0s (duration: 1.357853891s)
    integration_test.go:36: phase sweet-get @1.357853891s (duration: 40.214683994s)
    integration_test.go:169: output for tile38:
    integration_test.go:177: 
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-0
        [sweet] Benchmarks: tile38 (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: tile38
...
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-tile38 @41.57298777s (duration: 1m6.388084177s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @41.573168586s (duration: 1m52.819151549s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @41.572874916s (duration: 1m57.367569428s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @2m38.94049777s (duration: 42.845988368s)
    integration_test.go:131: 	phase sweet-run-bleve-index @2m34.392502295s (duration: 1m9.330675108s)
    integration_test.go:131: 	phase sweet-run-markdown @3m21.786553773s (duration: 26.302700377s)
    integration_test.go:131: 	phase sweet-run-etcd @1m47.961103429s (duration: 2m25.780162685s)
    integration_test.go:131: 	phase sweet-run-go-build @41.572890216s (duration: 3m39.523948901s)
    integration_test.go:36: phase sweet-run @41.572537885s (duration: 3m39.524344452s)
2024-01-08 19:14 linux-amd64-longtest-race benchmarks@921b81ae go@c4696665 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (175.22s)
    integration_test.go:36: phase setup @0s (duration: 9.287433891s)
    integration_test.go:36: phase sweet-get @9.287433891s (duration: 32.145387059s)
    integration_test.go:169: output for tile38:
    integration_test.go:177: 
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-0
        [sweet] Benchmarks: tile38 (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: tile38
...
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-tile38 @41.433229221s (duration: 36.848944537s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @41.433535841s (duration: 1m7.092571466s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @41.433182951s (duration: 1m7.82524205s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @1m49.258560671s (duration: 26.438185566s)
    integration_test.go:131: 	phase sweet-run-bleve-index @1m48.527881474s (duration: 34.545615666s)
    integration_test.go:131: 	phase sweet-run-markdown @2m15.696868927s (duration: 19.537273445s)
    integration_test.go:131: 	phase sweet-run-go-build @41.433551581s (duration: 2m5.227843932s)
    integration_test.go:131: 	phase sweet-run-etcd @1m18.282299888s (duration: 1m36.234384193s)
    integration_test.go:36: phase sweet-run @41.43282095s (duration: 2m13.083920251s)

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "golang.org/x/benchmarks/sweet/cmd/sweet" && test == "TestSweetEndToEnd"
2024-01-08 19:14 linux-amd64-longtest benchmarks@921b81ae go@0b1b6c15 x/benchmarks/sweet/cmd/sweet.TestSweetEndToEnd (log)
--- FAIL: TestSweetEndToEnd (208.37s)
    integration_test.go:36: phase setup @0s (duration: 1.109917692s)
    integration_test.go:36: phase sweet-get @1.109917692s (duration: 38.512118346s)
    integration_test.go:131: 	phase sweet-run-biogo-igor @39.622509143s (duration: 1m31.428507474s)
    integration_test.go:131: 	phase sweet-run-biogo-krishna @39.622334498s (duration: 1m36.321134331s)
    integration_test.go:161: no results produced.
    integration_test.go:183: command output:
        [sweet] Work directory: /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4
        [sweet] Benchmarks: etcd (1 runs, 1 pgo runs)
        [sweet] Running profile collection runs
        [sweet] Setting up benchmark: etcd
        [shell] git clone --depth 1 -b v3.6.0-alpha.0 https://github.com/etcd-io/etcd /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/etcd/src
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/results-4/etcd
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/etcd/go.profile/bin
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/etcd/src
        [shell] mkdir -p /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/etcd/go.profile/tmp
        [shell] GOFLAGS=-pgo=off PATH=/workdir/go/bin:/workdir/go/bin:/workdir/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin make -C /workdir/gopath/src/golang.org/x/benchmarks/sweet/tmp/tmp-4/etcd/src build
        [sweet] error: build etcd for go.profile: exit status 2
        [sweet] error: error preparing PGO profiles: failed to execute profile collection benchmarks, see log for details
    integration_test.go:184: exit status 1
    integration_test.go:131: 	phase sweet-run-etcd @2m11.051100421s (duration: 16.176104872s)
    integration_test.go:131: 	phase sweet-run-tile38 @39.622341294s (duration: 2m9.187620748s)
    integration_test.go:131: 	phase sweet-run-bleve-index @2m15.943507699s (duration: 39.081711119s)
    integration_test.go:131: 	phase sweet-run-gopher-lua @2m27.227250295s (duration: 34.427979769s)
    integration_test.go:131: 	phase sweet-run-markdown @2m48.810017131s (duration: 17.194242434s)
    integration_test.go:131: 	phase sweet-run-go-build @39.622394523s (duration: 2m47.904093466s)
    integration_test.go:36: phase sweet-run @39.622036038s (duration: 2m47.904512711s)

watchflakes

DarrylWong pushed a commit to DarrylWong/go-benchmarks that referenced this issue Apr 22, 2024
For golang/go#56958.

Change-Id: I0180f6f1afb939417628919db2b32c462df468e8
DarrylWong pushed a commit to DarrylWong/go-benchmarks that referenced this issue Apr 22, 2024
For golang/go#56958.

Change-Id: I0180f6f1afb939417628919db2b32c462df468e8
DarrylWong pushed a commit to DarrylWong/go-benchmarks that referenced this issue Apr 22, 2024
For golang/go#56958.

Change-Id: I0180f6f1afb939417628919db2b32c462df468e8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
Status: In Progress
Status: No status
Development

No branches or pull requests

4 participants