Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/test2json: support emitting JSON directly from the test binary #22996

Closed
petrhosek opened this issue Dec 5, 2017 · 3 comments
Closed

cmd/test2json: support emitting JSON directly from the test binary #22996

petrhosek opened this issue Dec 5, 2017 · 3 comments
Labels
FrozenDueToAge NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made.
Milestone

Comments

@petrhosek
Copy link

It is now possible to emit JSON from a test binary (#2981, #22789), but this still requires the go tool:

go tool test2json ./foo.test -test.v

This doesn't work in -c case which is really useful when cross-platform running tests, i.e. we build the tests binaries on one platform but run them on a different one as plain binaries without having to use the go tool. Would it be possible to support that scenario as well, e.g. as -test.json file flag?

@ianlancetaylor ianlancetaylor added this to the Go1.10 milestone Dec 5, 2017
@ianlancetaylor ianlancetaylor added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Dec 5, 2017
@ianlancetaylor
Copy link
Contributor

I believe you can capture the test output on one machine and run it through test2json on a different machine.

@petrhosek
Copy link
Author

It's certainly doable but it makes the CI pipeline more complicated because it requires additional post-processing.

@rsc
Copy link
Contributor

rsc commented Dec 13, 2017

We considered whether to do this originally, but it's very difficult to do completely. The test binary cannot reliably ensure that all its output goes into test2json. Maybe it crashes during init before test2json even starts up. Maybe it manages to print from a func init before replacing os.Stdout/os.Stderr with pipes into test2json. The binary also cannot reliably ensure that all output is processed by test2json. Maybe it crashes, in which case it could exit before test2json, so that the parent might not observe the final output processed by test2json. The output processing problem could be fixed by making the test binary exec test2json and have test2json then reexec the test binary as a subprocess, but then the test binary starts up twice, which may break other things.

Note that you don't need the whole go command for your CI system. You can do a cross-platform build of cmd/test2json once, install it onto your target system, and then run

path/to/test2json ./foo.test -test.v

That should be much more robust than anything we could do in the test binary itself. Please use that.

@rsc rsc closed this as completed Dec 13, 2017
@golang golang locked and limited conversation to collaborators Dec 13, 2018
@rsc rsc removed their assignment Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made.
Projects
None yet
Development

No branches or pull requests

4 participants