Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: fuzzing doesn't work with environment startup #52133

Closed
Ferrany1 opened this issue Apr 4, 2022 · 7 comments
Closed

cmd/go: fuzzing doesn't work with environment startup #52133

Ferrany1 opened this issue Apr 4, 2022 · 7 comments
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.

Comments

@Ferrany1
Copy link

Ferrany1 commented Apr 4, 2022

What version of Go are you using (go version)?

go version go1.18 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

GOHOSTARCH="amd64"
GOHOSTOS="darwin"

What did you do?

Running fuzz test with environment (containers with https://github.com/testcontainers/testcontainers-go and services in runtime) to test api routes

//go:build fuzzing

package main

import (
	"bytes"
	"context"
	"net/http"
	"testing"

	"github.com/stretchr/testify/require"
	"github.com/testcontainers/testcontainers-go"
	"github.com/testcontainers/testcontainers-go/wait"
)

func FuzzFoo(f *testing.F) {
	initEnvironment(f)
	f.Add("first")
	f.Fuzz(
		func(t *testing.T, tenant string) {
			t.Run("testFoo", func(t *testing.T) { testFoo(t, tenant) })
		},
	)
}

func initEnvironment(tb testing.TB) {
	ctx := context.Background()
	_, err := testcontainers.GenericNetwork(
		ctx,
		testcontainers.GenericNetworkRequest{
			NetworkRequest: testcontainers.NetworkRequest{
				Name:        "test_fuzz",
				ReaperImage: testcontainers.ReaperDefaultImage,
			},
		},
	)
	require.NoError(tb, err)

	containerRequest := testcontainers.ContainerRequest{
		Name:  "cocroach-fuzz",
		Image: "cockroachdb/cockroach:v21.1.11",
		Cmd:   []string{"start-single-node", "--insecure"},
		ExposedPorts: []string{
			"127.0.0.1:26257:26257/tcp",
			"127.0.0.1:8080:8080/tcp",
		},
		WaitingFor:  wait.ForListeningPort("26257/tcp"),
		ReaperImage: testcontainers.ReaperDefaultImage,
		Networks:    []string{"test_fuzz"},
	}
	require.NoError(tb, err)

	container, err := testcontainers.GenericContainer(
		ctx,
		testcontainers.GenericContainerRequest{
			ContainerRequest: containerRequest,
			Started:          false,
		},
	)
	require.NoError(tb, err)
	err = container.Start(ctx)
	require.NoError(tb, err)

	err = container.StartLogProducer(ctx)
	require.NoError(tb, err)

	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(http.StatusOK)
	})
	s := http.Server{Addr: "0.0.0.0:8090"}
	go s.ListenAndServe()
}

func testFoo(tb testing.TB, tenant string) {
	request, err := http.NewRequest(http.MethodPut, "http://127.0.0.1:8090/", bytes.NewBuffer([]byte(tenant)))
	require.NoError(tb, err)

	response, err := http.DefaultClient.Do(request)
	require.NoError(tb, err)
	defer func() {
		require.NoError(tb, response.Body.Close())
	}()
	require.Equal(tb, http.StatusOK, response.StatusCode)
}

Command: go test -fuzz=FuzzFoo -tags fuzzing -test.fuzztime 3m ./fuzz_test.go

Starting without initializing environment produces correct output

What did you expect to see?

fuzz: elapsed: 0s, gathering baseline coverage: 1/1 completed, now fuzzing with 4 workers

What did you see instead?

fuzz: elapsed: 0s, gathering baseline coverage: 0/1 completed
fuzz: elapsed: 0s, gathering baseline coverage: 0/1 completed
--- FAIL: FuzzFoo (15.07s)
    fuzzing process terminated without fuzzing: EOF
FAIL
exit status 1
@Ferrany1
Copy link
Author

Ferrany1 commented Apr 4, 2022

Seems like problem occurs because of testcontainers lib

@seankhliao
Copy link
Member

please fill out the issue template properly and include a self contained, reproducible example

@seankhliao seankhliao added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Apr 4, 2022
@Ferrany1
Copy link
Author

Ferrany1 commented Apr 5, 2022

please fill out the issue template properly and include a self contained, reproducible example

Added example with command, as I mentioned above it seems like testcontainers lib somehow affects fuzzing, because without it everything is fine

@Ferrany1
Copy link
Author

please fill out the issue template properly and include a self contained, reproducible example

Anything else needed?

@seankhliao
Copy link
Member

Fuzzing runs multiple subprocesses so it can catch crashes. Your use of testcontainers isn't compatible with that, the processes compete for shared global (OS) resources.
The bad error message is from testcontainers not reporting a proper error.

@Ferrany1
Copy link
Author

Fuzzing runs multiple subprocesses so it can catch crashes. Your use of testcontainers isn't compatible with that, the processes compete for shared global (OS) resources. The bad error message is from testcontainers not reporting a proper error.

But it seems okay if i start containers manually in docker, so it occurs because of some inner testcontainer work?
However it work okay in unit tests without any errors produced on side of testcontainers

@mdonkers
Copy link

mdonkers commented Sep 1, 2022

In case others come across this, the solution is adding -parallel 1 as parameter to the fuzzing test.
At least in my scenario running fuzzing against a testcontainer (DB to verify string formatting) worked fine.

@golang golang locked and limited conversation to collaborators Sep 1, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

4 participants