runtime/pprof: testCPUProfile hides real issues #13817
Labels
FrozenDueToAge
Testing
An issue that has been verified to require only test changes, not just a test failure.
Milestone
In response to #7095, we made the profiling tests pass if they get at least one sample. This is masking real issues. For example, on ARM, TestStackBarrierProfiling usually gets fewer than 10 samples, when it should be getting around 500 (discovered in #13405). It would be better if, when this got fewer samples than expected, it simply retried the test a few times. This would keep the test fast under common circumstances while still keeping it robust. (This is technically a repeated fixed-horizon hypothesis test, which technically means it's bad. What we really want is a sequential hypothesis test, but as long as the difference between "working" and "broken" [effect size] we're looking for is large enough, the simple thing is fine.)
@randall77 @RLH @dvyukov
The text was updated successfully, but these errors were encountered: