Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

internal/fuzz: go test -fuzz= outputs failing input which contains wrong datatypes #58824

Open
hitzhangjie opened this issue Mar 2, 2023 · 3 comments · May be fixed by #58827
Open

internal/fuzz: go test -fuzz= outputs failing input which contains wrong datatypes #58824

hitzhangjie opened this issue Mar 2, 2023 · 3 comments · May be fixed by #58827
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Backlog

Comments

@hitzhangjie
Copy link
Contributor

hitzhangjie commented Mar 2, 2023 •
edited

What version of Go are you using (go version)?

$ go version
go version go1.19.6 darwin/arm64

Does this issue reproduce with the latest release?

Yes, go1.20.1 still reproduces this.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="auto"
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/xxxxxxx/Library/Caches/go-build"
GOENV="/Users/xxxxxx/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/xxxxx/go/pkg/mod"
GOOS="darwin"
GOPATH="/Users/xxxxxx/go"
GOPROXY="direct"
GOROOT="/usr/local/go"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.19.6"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/xxxxx/Github/codemaster/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/4m/2_pn1sln1fzcg2zg_4mh95z80000gn/T/go-build1425731567=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I want to use go fuzztest to help finding the potential bugs, actually I'm learning how it works.

package math_test

import (
	"errors"
	"testing"
)

func Fuzz_overflow_int32(f *testing.F) {
	f.Add(int32(0x7fffffff), int32(0))
	f.Add(int32(-1<<31), int32(0))
	f.Fuzz(func(t *testing.T, a, b int32) {
		if c, err := safeSignedAdd(a, b); err != nil {
			t.Errorf("%v + %v = %v, err: %v", a, b, c, err)
		}
	})
}

var errOverflow = errors.New("overflow")

func safeSignedAdd[T ~int | ~int32 | ~int64](a, b T) (T, error) {
	c := a + b
	if a > 0 && b > 0 && c <= 0 ||
		a < 0 && b < 0 && c >= 0 {
		return c, errOverflow
	}
	return c, nil
}

What did you expect to see?

I want to see the failing and want to check failing input from the log in testdata/......

It ouputs:

go test -count=1 -fuzz=Fuzz*
fuzz: elapsed: 0s, gathering baseline coverage: 0/2 completed
fuzz: elapsed: 0s, gathering baseline coverage: 2/2 completed, now fuzzing with 10 workers
fuzz: elapsed: 0s, execs: 5 (203/sec), new interesting: 0 (total: 2)
--- FAIL: Fuzz_overflow_int32 (0.03s)
    --- FAIL: Fuzz_overflow_int32 (0.00s)
        math_test.go:13: 2147483645 + 84 = -2147483567, err: overflow
    
    Failing input written to testdata/fuzz/Fuzz_overflow_int32/fc5bf10796c021a99a0336732723622d6438cc5b9e43a3619ffd8e1d5c525266
    To re-run:
    go test -run=Fuzz_overflow_int32/fc5bf10796c021a99a0336732723622d6438cc5b9e43a3619ffd8e1d5c525266
FAIL
exit status 1
FAIL    github.com/hitzhangjie/codemaster/math  0.465s

The data in the log is:

cat testdata/fuzz/Fuzz_overflow_int32/fc5bf10796c021a99a0336732723622d6438cc5b9e43a3619ffd8e1d5c525266
go test fuzz v1
int32(2147483645)
rune('T')

What did you see instead?

The data in the log, I want to see the right datatype and value:

cat testdata/fuzz/Fuzz_overflow_int32/fc5bf10796c021a99a0336732723622d6438cc5b9e43a3619ffd8e1d5c525266
go test fuzz v1
int32(2147483645)
int32(84)
@hitzhangjie hitzhangjie changed the title cmd-*: go test -fuzz= outputs failing input which contains wrong datatypes internal/fuzz: go test -fuzz= outputs failing input which contains wrong datatypes Mar 2, 2023
@hitzhangjie
Copy link
Contributor Author

hitzhangjie commented Mar 2, 2023 •
edited

In go, type rune = int32, so the code internal/fuzz/encoding.go:L70 cannot differeniate rune and int32.
If there's no better solutions to solve this.

Could we output more data to help? like: rune('T')/int32(84) instead of only rune('T').
Because some int32 may have no characters, int32(84)/rune('T') may be better?

hitzhangjie added a commit to hitzhangjie/go that referenced this issue Mar 2, 2023
@hitzhangjie
fix/fuzz: output int32(v)/rune(q) in failing log
58ff6c4 
When fuzz testing outputs failing input into testdata/<log>, it needs to
marshal the input, In go, int32 and rune are the same datatype, so it
cannot differentiate the two types in typeswitch.

Before, when the data is a valid unicode point, it will outputs `rune(%q)`
in the log, but the real datatype maybe a int32. In this case, we have
to query the ASCII or unicode table to get the int32 value. It's
inconvenient.

So, here we output `int32(%v)/rune(%q)` to solve this.

close golang#58824
@hitzhangjie hitzhangjie linked a pull request Mar 2, 2023 that will close this issue
Open
hitzhangjie added a commit to hitzhangjie/go that referenced this issue Mar 2, 2023
@hitzhangjie
fix/fuzz: output int32(v)/rune(q) in failing log
a38e4ec 
When fuzz testing outputs failing input into testdata/<log>, it needs to
marshal the input, In go, int32 and rune are the same datatype, so it
cannot differentiate the two types in typeswitch.

Before, when the data is a valid unicode point, it will outputs `rune(%q)`
in the log, but the real datatype maybe a int32. In this case, we have
to query the ASCII or unicode table to get the int32 value. It's
inconvenient.

So, here we output `int32(%v)/rune(%q)` to solve this.

close golang#58824
@gopherbot
Copy link

Change https://go.dev/cl/472815 mentions this issue: fix/fuzz: output int32(v)/rune(q) in failing log

@dmitshur
Copy link
Contributor

dmitshur commented Mar 2, 2023

CC @golang/fuzzing.

@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 2, 2023
@dmitshur dmitshur added this to the Backlog milestone Mar 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Backlog
Development

Successfully merging a pull request may close this issue.

fix/fuzz: output int32(v)/rune(q) in failing log hitzhangjie/go
3 participants
@dmitshur @hitzhangjie @gopherbot