Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

archive/zip: sporadic checksum errors on s390x target #53797

Closed
byo opened this issue Jul 11, 2022 · 5 comments
Closed

archive/zip: sporadic checksum errors on s390x target #53797

byo opened this issue Jul 11, 2022 · 5 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Backlog

Comments

@byo
Copy link

byo commented Jul 11, 2022

What version of Go are you using (go version)?

$ go version
go version go1.17.11 linux/amd64

Does this issue reproduce with the latest release?

yes (1.18.3 linux/amd64)

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/runner/.cache/go-build"
GOENV="/home/runner/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd[6](https://github.com/byo/go-s390x-test/runs/7290379650?check_suite_focus=true#step:5:7)4"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/runner/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/runner/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/hostedtoolcache/go/1.1[7](https://github.com/byo/go-s390x-test/runs/7290379650?check_suite_focus=true#step:5:8).11/x64"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/hostedtoolcache/go/1.17.11/x64/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.17.11"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build321[8](https://github.com/byo/go-s390x-test/runs/7290379650?check_suite_focus=true#step:5:9)18[9](https://github.com/byo/go-s390x-test/runs/7290379650?check_suite_focus=true#step:5:10)947=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Created a simple quine in go using statik to embed the source code. Statik uses internally zip compression.

Cross-compiled the binary for linux/s390x OS/ARCH and executed the binary on a linux/amd64 os using qemu-s390x.

What did you expect to see?

Binary running without errors printing its source code.

What did you see instead?

Once in a while the binary fails with the following error message:

error unzipping file "main.go": zip: checksum error

I've also caught different error (captured when using go 1.18) which may indicate that this is not strictly related to the archive/zip package:

runtime: asyncPreemptStack=336
fatal error: async stack too large

goroutine 1 [running, locked to thread]:
runtime.throw({0x19d5ae, 0x15})
        /usr/lib/go-1.18/src/runtime/panic.go:992 +0x70 fp=0xc00003c618 sp=0xc00003c5f0 pc=0x4aa70
runtime.init.4()
        /usr/lib/go-1.18/src/runtime/preempt.go:336 +0xf8 fp=0xc00003c640 sp=0xc00003c618 pc=0x4c068
runtime.doInit(0x283200)
        /usr/lib/go-1.18/src/runtime/proc.go:6222 +0x14a fp=0xc00003c780 sp=0xc00003c640 pc=0x5d24a
runtime.main()
        /usr/lib/go-1.18/src/runtime/proc.go:199 +0x126 fp=0xc00003c7d8 sp=0xc00003c780 pc=0x4d6a6
runtime.goexit()
        /usr/lib/go-1.18/src/runtime/asm_s390x.s:742 +0x2 fp=0xc00003c7d8 sp=0xc00003c7d8 pc=0x7e142

The frequency of the bug seems to be very random and vary between different environments (like ~10% chance on my laptop, 2% on Github actions runners).

A project that shows the issue: https://github.com/byo/go-s390x-test
Example GitHub actions run showing the failure: https://github.com/byo/go-s390x-test/runs/7290506324?check_suite_focus=true
@mknyszek mknyszek changed the title archive/zip: Sporadic checksum errors on s390x target archive/zip: sporadic checksum errors on s390x target Jul 11, 2022
@mknyszek mknyszek added this to the Backlog milestone Jul 11, 2022
@mknyszek mknyszek added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 11, 2022
@mknyszek
Copy link
Contributor

CC @dsnet

@cherrymui
Copy link
Member

runtime: asyncPreemptStack=336
fatal error: async stack too large

This error makes no sense. It fails when asyncPreemptStack > _StackLimit. But _StackLimit is

_StackGuard - _StackSystem - _StackSmall
= 928*sys.StackGuardMultiplier + _StackSystem - _StackSystem - _StackSmall
= 928*sys.StackGuardMultiplier - _StackSmall
= 928*1 - 128 = 800

Also, this runs at runtime startup so if it fails it should fail 100%.

Cross-compiled the binary for linux/s390x OS/ARCH and executed the binary on a linux/amd64 os using qemu-s390x.

It looks like you're using QEMU user-mode emulation, which is not a supported target. User-mode emulation has been known to be buggy. This looks like a bug in the emulator.

If you can reproduce the failure in real hardware, feel free to reopen. Thanks.

@cherrymui cherrymui closed this as not planned Won't fix, can't repro, duplicate, stale Jul 11, 2022
@dsnet
Copy link
Member

dsnet commented Jul 12, 2022

Given that zip doesn't use any unsafe or goroutines, I'm having a hard time imagining what zip could be doing that is buggy on s390x but not other architectures.

The only thing I can imagine is that zip uses hash/crc32 which does have a s390x-specific implementation. In that case, the bug lies with hash/crc32 and not with zip.

@byo
Copy link
Author

byo commented Jul 16, 2022

I can now fully confirm this was indeed a qemu bug. The bogus behavior does not appear anymore with newest qemu version.

Sorry for false report.

@ianlancetaylor
Copy link
Contributor

Thanks for following up.

@BenTheElder BenTheElder mentioned this issue Nov 3, 2022
Closed
@golang golang locked and limited conversation to collaborators Jul 16, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
6 participants
@mknyszek @byo @ianlancetaylor @dsnet @gopherbot @cherrymui