proposal: cmd/go: add a test flag to update runnable example output comments #51254
Comments
|
or maybe under |
|
That would require |
|
I have run into cases before where it was hard to get the output comment correct because there was some trivial whitespace that didn’t affect overall correctness but was hard to get right and copypaste through the terminal was slightly off (I forget the exact details but it was like some tabs mixed with spaces which weren’t copying correctly or something). Having this feature would have been very helpful. |
|
This example pattern is very similar to the NodeJS library, Jest, with its "Snapshot" pattern. https://jestjs.io/docs/snapshot-testing run tests to validate that the examples match: jestupdate the examples on disk: I'd be super excited to see this in Go! |
We have support for examples which can be run as part of
go test, as documented in https://pkg.go.dev/testing#hdr-Examples.Right now, if I change the example code in any way that changes its output, such as by tweaking the package API implementations, I'll get a failure when running
go testagain. With #41980, that failure will show me a diff, which is pretty neat.However, if I do intend to change the output of the example for good, I'll need to manually update the
// Output:comment lines in the example code, such as by copying the output text from the output ofgo test, pasting it in the editor, commenting out those lines, and placing them where the old ones are.It would be nice if we instead had a flag, like
go test -updateoutputbut with a better name, so that could happen in-place automatically. It could also work with unordered output lines, which could keep the source code deterministic by e.g. sorting the lines.The reason it would be a flag is because the existing default behavior should be kept. By default with
go test, a runnable example whose output doesn't match should result in a failure.It should be noted that @cznic has implemented an external version of this in https://pkg.go.dev/modernc.org/fe. It can certainly work outside of cmd/go, though I'd argue that it would be nice to have it built-in, akin to how we have other cmd/go commands to edit source in-place such as
go fixorgo mod edit.One question might be: how often does this come up? I personally run into this scenario relatively often, as I try to design end-user Go APIs by first writing one or two examples, often in runnable form. As I iterate on implementing the logic of the APIs, I want to verify that their output becomes what I expect, akin to re-running a test until it succeeds.
This may not need to be a proposal, but since the suggestion is to add a new
go testflag, I'm erring on the side of caution - it may be a controversial feature request :)cc @bcmills @matloob (Bryan and I briefly discussed this idea on Slack back in September 2021)
The text was updated successfully, but these errors were encountered: