cc: @bcmills @rsc @ianlancetaylor

CC @jayconrod @matloob @ianthehat

Somewhat related: #29430

You can kind of already do this. If you run go list -test -f '{{if eq .Name "main"}}{{.GoFiles}}{{end}}' *.go, go list writes the testmain source to the build cache, and you can read it out from there. Kludgy though.

Something that complicates this change is that we generate the testmain source when loading packages, not during execution as you'd expect. The generated testmain is listed in .GoFiles, not .CompiledGoFiles because of this. If we were going to generate the testmain in a subprocess, we should move the code generation into a proper action first so that we can benefit from caching. Actually, we should do this anyway so go list can do less work.

Even if we did this, I'd be very surprised if the generated file could be used as-is. Bazel needs to do some other stuff in main before running tests, like changing directories, sharding, filtering, and coverage post-processing. Every build system is going to need some custom code, and I'm not confident we can factor all that out into a separate file.

That is pretty interesting. I didn't know it was stored in the cache. Digging through the cache seems like it could be more fragile than what we are dealing with here.

I'd be very surprised if the generated file could be used as-is. Bazel needs to do some other stuff in main before running tests, like changing directories, sharding, filtering, and coverage **post-processing.

In Buck we change directories , do test discovery, do sharding through a "test runner" which is just a wrapper around the compiled testmain which seems like a much cleaner solution with better abstractions, but that isn't really the topic here.

Does Bazel/Blaze do this for other languages as well like Python or C++?

Every build system is going to need some custom code, and I'm not confident we can factor all that out into a separate file.

There is very little that Buck does here intentionally as it's already too hard to maintain the testmain generation code with just changes to the testing package

The core problem we have had w/ Buck is that the testing package and the generated code that uses has changed with nearly every major Go release. We have been able to hack around this by adding reflection, implementing an interface, etc Even if some of the testmaingen code was available as an external library that could be imported it would help, but it would require a stable API between it and the testing package which may be less than ideal.

Here are some of the changes that has been required.

• facebook/buck@4271ef7#diff-d7a1345bc79619677ed4da43c65e7540
• facebook/buck@2b454ad#diff-d7a1345bc79619677ed4da43c65e7540

In fact when we upgrade to new Go versions this is really the only consistent blocker we have had over the years.

I think @jayconrod's answer here is the right one. There shouldn't be a separate API for generating testmain.go by itself. Even knowing that testmain.go is a thing is too much detailed knowledge about tests. What we've exposed is how to ask 'tell me the exact way the test binary should get built' instead. That's a much more general query.

Per discussion with @golang/proposal-review, looks like this is resolved. Please comment if that’s not the case.

I guess then the problem is there is no spec on how other build tools should interact with the testing package. It would be beneficial to have a spec on how that interaction would look so that other building tools could just build their own test main generation (or whatever).

The original problem is that the interaction between the the testing package and the tool is very much intertwined leading to a lot of breakage when you step slightly outside of it.