Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: limit the number of errors #12276

Closed
josharian opened this issue Aug 23, 2015 · 6 comments
Closed

testing: limit the number of errors #12276

josharian opened this issue Aug 23, 2015 · 6 comments

Comments

@josharian
Copy link
Contributor

Exhaustive tests (such as found in math/big and strconv) can generate a phenomenal number of errors when they fail. At some point, there is no value in additional errors.

I think that we should maintain an error counter in testing.T and call t.Fatal("too many errors") once we hit (say) 100 errors in a test.

cc @adg and @robpike for feedback. I'll implement if you're ok with it, since this is a pain point for SSA development.

@robpike
Copy link
Contributor

robpike commented Aug 23, 2015

go test | sed 100q seems good enough to me. If you add features like this, you need features to control the features, new features are justified because they are in the same style, and they need more features to control them, and it never stops.

@josharian
Copy link
Contributor Author

Ok. FTR, on digging, a large part of the pain is that I'm running with GOGC=off (by necessity) and all the output is buffered (not fixed by sed), which ends up causing massive memory usage.

@robpike
Copy link
Contributor

robpike commented Aug 23, 2015

The place to fix that is certainly not with new flags in the testing package.

@josharian
Copy link
Contributor Author

I didn't have in mind new flags, FWIW, just an unexported counter field in testing.T and something at the beginning of Error(f) like

t.nerr++
if t.nerr > 1000 {
  t.Fatal("too many errors")
}

But it's not really worth more words.

@robpike
Copy link
Contributor

robpike commented Aug 23, 2015

But you know someone will then ask to have a flag to change that 1000. Where to draw the line?

@josharian
Copy link
Contributor Author

Yep. :/

@golang golang locked and limited conversation to collaborators Aug 22, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants