Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: cmd/cover: test coverage should report coverage of non-error-handling lines #66713

Open
pjweinb opened this issue Apr 7, 2024 · 5 comments
Labels
Milestone

Comments

@pjweinb
Copy link

pjweinb commented Apr 7, 2024

Proposal Details

Coverage summarizes its results with a one line output like "coverage: 89.1% of statements"

It would be more useful if it said something like "coverage: 89.1% of statements, 97.3% without error handling"

The new number would remove from its denominator the the number of statements inside 'if err != nil {...}' blocks.

Among other deficiencies, the current number gets worse as more error handling gets added.

@ianlancetaylor ianlancetaylor changed the title test coverage should report coverage of non-error-handling lines proposal: cmd/cover: test coverage should report coverage of non-error-handling lines Apr 8, 2024
@ianlancetaylor
Copy link
Contributor

CC @thanm

@gopherbot gopherbot added this to the Proposal milestone Apr 8, 2024
@gophun
Copy link

gophun commented Apr 8, 2024

What's the definition of error handling lines? 'if err != nil {...}' can't be it as errors can be handled in many other ways. Starting with different variable names (or no variable at all) to using switch statements or other control flow mechanisms, sometimes putting them into data structures to inspect them later, as errors are just values.

Also, error cases are usually just as important to test as any other cases. This would give the impression that error handling is somehow less important code.

@pjweinb
Copy link
Author

pjweinb commented Apr 8, 2024 via email

@thanm
Copy link
Contributor

thanm commented Apr 8, 2024

@pjweinb I feel your pain regarding collecting coverage numbers for code that is very heavily riddled with error handling code.

When writing tests for the new coverage code I had some of the same problems in that the code that writes out new-style coverage data files basically looked like

   func writeTheWholeThing(w io.Writer) error {
      if _, err := w.Write(...); err != nil {
        return err
      }
      <... repeat 50 times...>
      if _, err := w.Write(...); err != nil {
        return err
      }
      return nil
   }

and of course in most normal test runs none of the writes above will fail, and the % statements covered will reflect that of course.

If there is a way you can write your code in a way that allows you to inject errors, that would be my suggestion.

An example (which works well for code like the func above): https://go.googlesource.com/go/+/45b641ce15159e29fa4494b837493042d1e10384/src/runtime/coverage/testdata/harness.go#166

@earthboundkid
Copy link
Contributor

This is a special case of #53271.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Incoming
Development

No branches or pull requests

6 participants