Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime: fatal error: AllThreadsSyscall6 results differ between threads; runtime corrupted [mipsle] #56426

Closed
bradfitz opened this issue Oct 26, 2022 · 9 comments
Closed

runtime: fatal error: AllThreadsSyscall6 results differ between threads; runtime corrupted [mipsle] #56426

bradfitz opened this issue Oct 26, 2022 · 9 comments
Assignees
@mengzhuo @4a6f656c @milanknezevic
Labels
arch-mips compiler/runtime Issues related to the Go compiler and/or runtime. ExpertNeeded FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Backlog

Comments

@bradfitz
Copy link
Contributor

We just got a bug report from a user that AllThreadsSyscall6 (via a call to syscall.Setuid without cgo) crashed.

Does AllThreadsSyscall not work on mipsle?

Go 1.19.2
GOARCH=mipsle (https://openwrt.org/toh/hwdata/gehua/gehua_ghl-r-001)
CGO_ENABLED=0

trap:4081, a123456=[1,30851488,0,0,0,0]
results: got {r1=0,r2=4081,errno=0}, want {r1=0,r2=30851488,errno=0
fatal error: AllThreadsSyscall6 results differ between threads; runtime corrupted
trap:4081, a123456=[1,30851488,0,0,0,0]
results: got {r1=0,r2=4081,errno=0}, want {r1=0,r2=30851488,errno=0
fatal error: AllThreadsSyscall6 results differ between threads; runtime corrupted
trap:4081, a123456=[1,30851488,0,0,0,0]
results: got {r1=0,r2=4081,errno=0}, want {r1=0,r2=30851488,errno=0
fatal error: AllThreadsSyscall6 results differ between threads; runtime corrupted
trap:4081, a123456=[1,30851488,0,0,0,0]
results: got {r1=0,r2=4081,errno=0}, want {r1=0,r2=30851488,errno=0
fatal error: AllThreadsSyscall6 results differ between threads; runtime corrupted

goroutine 0 [idle]:
runtime.fatal({0xb8af82, 0x44})
        runtime/panic.go:1066 +0x54
runtime.runPerThreadSyscall()
        runtime/os_linux.go:884 +0x3e0
runtime.sighandler(0x21, 0x1c53d08, 0x1c53d88, 0x1c005a0)
        runtime/signal_unix.go:647 +0x710
runtime.sigtrampgo(0x21, 0x1c53d08, 0x1c53d88)
        runtime/signal_unix.go:479 +0x208
runtime.sigtramp()
        runtime/sys_linux_mipsx.s:325 +0x40

goroutine 1 [running]:
        goroutine running on other thread; stack unavailable

goroutine 0 [idle]:
runtime.fatal({0xb8af82, 0x44})
        runtime/panic.go:1066 +0x54
runtime.runPerThreadSyscall()
        runtime/os_linux.go:884 +0x3e0
runtime.sighandler(0x21, 0x1c5bd08, 0x1c5bd88, 0x1c00780)
        runtime/signal_unix.go:647 +0x710
runtime.sigtrampgo(0x21, 0x1c5bd08, 0x1c5bd88)
        runtime/signal_unix.go:479 +0x208
runtime.sigtramp()
        runtime/sys_linux_mipsx.s:325 +0x40

goroutine 0 [idle]:
runtime.fatal({0xb8af82, 0x44})
        runtime/panic.go:1066 +0x54
runtime.runPerThreadSyscall()
        runtime/os_linux.go:884 +0x3e0
runtime.sighandler(0x21, 0x1c8bd08, 0x1c8bd88, 0x1c820f0)
        runtime/signal_unix.go:647 +0x710
runtime.sigtrampgo(0x21, 0x1c8bd08, 0x1c8bd88)
        runtime/signal_unix.go:479 +0x208
runtime.sigtramp()
        runtime/sys_linux_mipsx.s:325 +0x40

goroutine 0 [idle]:
runtime.fatal({0xb8af82, 0x44})
        runtime/panic.go:1066 +0x54
runtime.runPerThreadSyscall()
        runtime/os_linux.go:884 +0x3e0
runtime.sighandler(0x21, 0x1c43d08, 0x1c43d88, 0x1c002d0)
        runtime/signal_unix.go:647 +0x710
runtime.sigtrampgo(0x21, 0x1c43d08, 0x1c43d88)
        runtime/signal_unix.go:479 +0x208
runtime.sigtramp()
        runtime/sys_linux_mipsx.s:325 +0x40

(from tailscale/tailscale#6060)

/cc @hdhoang @maisem @ianlancetaylor @aclements
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Oct 26, 2022
@bradfitz
Copy link
Contributor Author

I don't have this hardware but I could try to reproduce it on qemu at least if that'd help and this isn't a known dup?

@bradfitz bradfitz mentioned this issue Oct 26, 2022
Closed
@bcmills
Copy link
Contributor

bcmills commented Oct 26, 2022

This might be related to #51192? (CC @golang/runtime)

@bcmills
Copy link
Contributor

bcmills commented Oct 26, 2022

Hmm, https://man7.org/linux/man-pages/man2/syscall.2.html#NOTES seems to indicate that the syscall ABI for MIPS does return two values. 🤔

@bcmills
Copy link
Contributor

bcmills commented Oct 26, 2022

(attn @golang/mips)

@bcmills bcmills added this to the Backlog milestone Oct 26, 2022
@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Oct 26, 2022
@prattmic
Copy link
Member

We've previously fixed this for ppc64 (#42178) with https://go.dev/cl/266202 and later https://go.dev/cl/385796. The latter is probably what we want if mips really doesn't use r2.

@mknyszek
Copy link
Contributor

Since we have a good idea of what this might be and the fix is straightforward, assigning to @golang/mips to follow up.

Feel free to toss it back if you run into issues.

@prattmic
Copy link
Member

@cherrymui pointed out that this register may only be used for a few system calls. But if the kernel does not clobber it, it may be best to have Syscall6 zero the register prior to the SYSCALL instruction.

@gopherbot
Copy link

Change https://go.dev/cl/445695 mentions this issue: runtime: add missing closing curly brace in runtime corruption error message

gopherbot pushed a commit that referenced this issue Oct 26, 2022
@bradfitz @gopherbot
runtime: add missing closing curly brace in runtime corruption error …
ed24b37 
…message

(Fixing the most important part of this bug.)

Updates #56426

Change-Id: If657ae47a5fe7dacc31d2c487e53e9f2dd5d03bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/445695
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Auto-Submit: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
romaindoumenc pushed a commit to TroutSoftware/go that referenced this issue Nov 3, 2022
@bradfitz @romaindoumenc
runtime: add missing closing curly brace in runtime corruption error …
e25ad05 
…message

(Fixing the most important part of this bug.)

Updates golang#56426

Change-Id: If657ae47a5fe7dacc31d2c487e53e9f2dd5d03bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/445695
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Auto-Submit: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Rongronggg9 added a commit to Rongronggg9/go that referenced this issue Nov 10, 2022
@Rongronggg9
syscall, runtime/internal/syscall: zero r2 before mips linux syscalls
ff245d3 
All mips variant perform syscalls similarly. R2 (v0) holds r1 and R3
(v1) holds r2 of a syscall. The latter is only used by 2-ret syscalls.
A 1-ret syscall would not touch R3 but keeps it as is, making r2 be a
random value. Always reset it to 0 before SYSCALL to fix the issue.

Fixes golang#56426
Rongronggg9 added a commit to Rongronggg9/go that referenced this issue Nov 10, 2022
@Rongronggg9
syscall, runtime/internal/syscall: zero r2 before mips linux syscalls
1d16277 
All mips variant perform syscalls similarly. R2 (v0) holds r1 and R3
(v1) holds r2 of a syscall. The latter is only used by 2-ret syscalls.
A 1-ret syscall would not touch R3 but keeps it as is, making r2 be a
random value. Always reset it to 0 before SYSCALL to fix the issue.

Fixes golang#56426
@gopherbot
Copy link

Change https://go.dev/cl/452975 mentions this issue: syscall, runtime/internal/syscall: zero r2 before mips linux syscalls

@tianon tianon mentioned this issue Dec 19, 2022
Merged
@golang golang locked and limited conversation to collaborators Nov 24, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@mengzhuo mengzhuo

@4a6f656c 4a6f656c

@milanknezevic milanknezevic

Labels
arch-mips compiler/runtime Issues related to the Go compiler and/or runtime. ExpertNeeded FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
8 participants
@bradfitz @mengzhuo @mknyszek @prattmic @bcmills @4a6f656c @gopherbot @milanknezevic