Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: large shifts miscompiled on riscv64 #64285

Closed
bcmills opened this issue Nov 20, 2023 · 11 comments
Closed

cmd/compile: large shifts miscompiled on riscv64 #64285

bcmills opened this issue Nov 20, 2023 · 11 comments
Labels
arch-riscv Issues solely affecting the riscv64 architecture. compiler/runtime Issues related to the Go compiler and/or runtime. NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@bcmills
Copy link
Contributor

bcmills commented Nov 20, 2023

#!watchflakes
post <- goarch == "riscv64" && pkg == "internal/chacha8rand" && test == "TestMarshal" && `Unmarshal: invalid ChaCha8 encoding`

https://build.golang.org/log/2031961837b8091c07edd2b489ea824efae86fe6:

--- FAIL: TestMarshal (0.00s)
    rand_test.go:42: #1: Unmarshal: invalid ChaCha8 encoding
FAIL
FAIL	internal/chacha8rand	0.006s

(attn @rsc @FiloSottile @golang/riscv64)

@bcmills bcmills added arch-riscv Issues solely affecting the riscv64 architecture. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Nov 20, 2023
@bcmills bcmills added this to the Go1.22 milestone Nov 20, 2023
@bcmills
Copy link
Contributor Author

bcmills commented Nov 20, 2023

Note that this is a test of a new API added for Go 1.22 (#61716).

@gopherbot
Copy link

Change https://go.dev/cl/543716 mentions this issue: internal/chacha8rand: disable ChaCha8 tests on certain platforms

@gopherbot
Copy link

Change https://go.dev/cl/543895 mentions this issue: Revert "math/rand/v2: add ChaCha8"

gopherbot pushed a commit that referenced this issue Nov 20, 2023
This reverts commit 6382893.

Reason for revert: Causes failures on big endian platforms and riscv64.
Possibly a bug in the generic implementation.

For #64284.
For #64285.

Change-Id: Ic1bb8533d9641fae28d0337b36d434b9a575cd7e
Reviewed-on: https://go-review.googlesource.com/c/go/+/543895
Reviewed-by: Heschi Kreinick <heschi@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
@4a6f656c
Copy link
Contributor

4a6f656c commented Nov 21, 2023

/cc @markdryan

This code is triggering a code generation bug in the riscv64 rewrite rules (presumably introduced by https://go.dev/cl/528975 - 561bf04).

I'm investigating a fix.

@markdryan
Copy link
Contributor

markdryan commented Nov 21, 2023

@4a6f656c Thanks for letting me know. I can reproduce this locally as well and can confirm that the issue goes away with https://go-review.googlesource.com/c/go/+/528975 and https://go-review.googlesource.com/c/go/+/535115 reverted. The reproducer is embarrassingly simple.

package main

import "fmt"

func badShift(c uint32) {
        fmt.Println(uint64(c) >> 56)
}

func badShiftSigned(c int32) {
        fmt.Println(int64(c) >> 56)
}

func main() {
        badShift(0xbadbaad)
        badShiftSigned(0xbadbaad)
}

This should print

0
0

but it actually prints

11
11

Without the patches, badShift generates

sll     t0,a0,0x20
srl     t0,t0,0x20
srl     a0,t0,0x38

and with them it generates

srlw    a0,a0,0x18 

which obviously isn't going to zero what was originally a 32 bit value in the same way that shifting by 0x38 will.

@markdryan
Copy link
Contributor

The issue is also reproducible with variable shifts, e.g.,

func badShiftVar(c uint32, s uint64) {
	fmt.Println(uint64(c) >> s)
}

func badShiftSignedVar(c int32, s uint64) {
	fmt.Println(int64(c) >> s)
}

where s > 32

e.g.,

        badShiftVar(0xbadbaad, 56)
        badShiftSignedVar(0xbadbaad, 56)

also prints

11
11

@bcmills bcmills added the compiler/runtime Issues related to the Go compiler and/or runtime. label Nov 21, 2023
@bcmills bcmills changed the title internal/chacha8rand: TestMarshal failing on riscv64 cmd/compile: large shifts miscompiled on riscv64 Nov 21, 2023
@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Nov 21, 2023
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Nov 21, 2023
@markdryan
Copy link
Contributor

@4a6f656c do let me know if there's anything I can do to help.

Note that removing

// Avoid unnecessary zero and sign extension when right shifting.
(SRL <t> (MOVWUreg x) y) => (SRLW <t> x y)
(SRLI <t> [x] (MOVWUreg y)) => (SRLIW <t> [int64(x&31)] y)
(SRA <t> (MOVWreg x) y) => (SRAW <t> x y)
(SRAI <t> [x] (MOVWreg y)) => (SRAIW <t> [int64(x&31)] y)

and modifying the 32 bit shift rules directly, for example by replacing

(Rsh32Ux64 <t> x y) && !shiftIsBounded(v) => (AND (SRL <t> (ZeroExt32to64 x) y) (Neg32 <t> (SLTIU <t> [64] y)))
(Rsh32Ux(64|32|16|8) x y) && shiftIsBounded(v) => (SRL (ZeroExt32to64 x) y)

with

(Rsh32Ux64 <t> x y) && !shiftIsBounded(v) => (AND (SRLW <t> x                y) (Neg32 <t> (SLTIU <t> [32] y)))
(Rsh32Ux(64|32|16|8) x y) && shiftIsBounded(v) => (SRLW x                y)

does fix the issue, at the cost of missing out on the optimisation that is causing the problem, i.e., this disables the broken optimisation for uint64(c) >> s (where c is an uint32) rather than fixing it, while retaining the optimisations for shifts of 32 bit ints. Perhaps there's a more elegant solutions though, ...

@4a6f656c
Copy link
Contributor

@markdryan thanks for investigating/confirming.

Unfortunately, my suggestion to optimise more generally has lead to a problem where by a valid type conversion and overflow shift is resulting in bad/incorrect code. The following two rules:

(SRL <t> (MOVWUreg x) y) => (SRLW <t> x y)
(SRLI <t> [x] (MOVWUreg y)) => (SRLIW <t> [int64(x&31)] y)

result in uint64(uint32(i)) >> 33 becoming SRLIW $1, y when it should obviously be zero (similarly with int64(int32(i)) >> 33). This can be resolved with appropriate bounds, however I think it is overall safer to take your original approach and move the SRAW/SRLW into the Rsh* rules - we just need a couple of (more carefully crafted) additional rules to keep the SRAIW/SRLIW optimisations.

@gopherbot
Copy link

Change https://go.dev/cl/545035 mentions this issue: cmd/compile: correct code generation for right shifts on riscv64

@markdryan
Copy link
Contributor

Thanks @4a6f656c. I'll give the patch a good test this morning.

@gopherbot
Copy link

Change https://go.dev/cl/546020 mentions this issue: math/rand/v2: add ChaCha8

gopherbot pushed a commit that referenced this issue Dec 5, 2023
This is a replay of CL 516859, after its rollback in CL 543895,
with big-endian systems fixed and the tests disabled on RISC-V
since the compiler is broken there (#64285).

ChaCha8 provides a cryptographically strong generator
alongside PCG, so that people who want stronger randomness
have access to that. On systems with 128-bit vector math
assembly (amd64 and arm64), ChaCha8 runs at about the same
speed as PCG (25% slower on amd64, 2% faster on arm64).

Fixes #64284.

Change-Id: I6290bb8ace28e1aff9a61f805dbe380ccdf25b94
Reviewed-on: https://go-review.googlesource.com/c/go/+/546020
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
ezz-no pushed a commit to ezz-no/go-ezzno that referenced this issue Feb 18, 2024
This is a replay of CL 516859, after its rollback in CL 543895,
with big-endian systems fixed and the tests disabled on RISC-V
since the compiler is broken there (golang#64285).

ChaCha8 provides a cryptographically strong generator
alongside PCG, so that people who want stronger randomness
have access to that. On systems with 128-bit vector math
assembly (amd64 and arm64), ChaCha8 runs at about the same
speed as PCG (25% slower on amd64, 2% faster on arm64).

Fixes golang#64284.

Change-Id: I6290bb8ace28e1aff9a61f805dbe380ccdf25b94
Reviewed-on: https://go-review.googlesource.com/c/go/+/546020
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arch-riscv Issues solely affecting the riscv64 architecture. compiler/runtime Issues related to the Go compiler and/or runtime. NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

4 participants