This sounds like a good idea, in order to catch issues like this in the future during trybot runs.

@bradfitz The js-wasm builder currently has numTryTestHelpers set to 4, which seems quite close to most other builders. What do you think would be a good number to set it to in order to be able js-wasm trybot to run all dist tests? Any suggestions on how to evaluate how that number correlates to the trybot execution time?

Any suggestions on how to evaluate how that number correlates to the trybot execution time?

Write some SQL to query BigQuery. All the test timing data is in there for every build (both trybots and post-submit builds).

Ok. I'll try to look for when a numTryTestHelpers value has changed and see how it affected corresponding trybot/post-submit times.

For js-wasm trybots specifically, I can't predict how the execution time will change when its ShouldRunDistTest policy is modified to not skip tests, other than to make that change and see the effect.

Edit: Perhaps I can get a sense for it by running the old and new tests on my local machine, and see the relative timing difference.

I wouldn't even try to correlate it with numTryTestHelpers historical values.

You should be able to pick a goal time (5 or 6 minutes), subtract empirical make.bash time, and then that's the amount of time to you have to run all the tests. So find the sum of all the test durations for js-wasm and divide, and that's the shard count you want.