Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: examples fail if output is colored #34284

Closed
GuillaumeLescure opened this issue Sep 13, 2019 · 11 comments
Closed

testing: examples fail if output is colored #34284

GuillaumeLescure opened this issue Sep 13, 2019 · 11 comments

Comments

@GuillaumeLescure
Copy link

GuillaumeLescure commented Sep 13, 2019

Issue :
go test fail if the output is colored.

Even if the tags are in the output (the comment line), it fails and seems to be litterally evaluated.

Exemple :
In myColored_test.go :

package main

import (
"fmt"
)

func Example_ColorTerm() {
fmt.Print("\033[31mHello World\033[0m")
//Output: Hello World
}

Expectation :
Be able to pass the test either :

  • as litteral "Hello World" (colors ignored)
  • as interpreted string "\033[31mHello World\033[0m" (colors are also checked)

I personally prefere the second solution because it allows more accurate tests.

Environment :

  • go version go1.12.9 linux/amd64
  • Manjaro (Linux 4.19.69-1)
@mvdan
Copy link
Member

mvdan commented Sep 13, 2019

I believe that example outputs are only intended for plaintext; I haven't seen anyone attempt to use non-printable or special characters with them.

I'm also not sure that such an example should be supported. What should godoc.org show for it? It's not a terminal, so it won't color the words.

@GuillaumeLescure
Copy link
Author

Agree but if you want to show the user what the output should look like (even without colors), you'll have to fill the "Output comment". But if you do so, it will automatically compare it to the "real" output.

If the colors are also displayed in the documentation, I would be ok with it, but that's not my point.
My point is : I cannot have a documentation with exemple and working test.

@ianlancetaylor ianlancetaylor changed the title Test fail for example if output is colored testing: examples fail if output is colored Sep 13, 2019
@ALTree
Copy link
Member

ALTree commented Sep 13, 2019

Agree but if you want to show the user what the output should look like (even without colors), you'll have to fill the "Output comment".

It doesn't have to be an Output comment... you can just use a normal comment and then it won't be matched with the output. What am I missing?

@ianlancetaylor
Copy link
Contributor

In my opinion examples should just match the output characters exactly. Anything else will just lead us into confusion. What if you want to use an example to show the difference between colorized and non-colorized output?

@GuillaumeLescure
Copy link
Author

GuillaumeLescure commented Sep 13, 2019

It doesn't have to be an Output comment... you can just use a normal comment and then it won't be matched with the output. What am I missing?

godoc_output
That's no the same result on godoc.

In my opinion examples should just match the output characters exactly. Anything else will just lead us into confusion. What if you want to use an example to show the difference between colorized and non-colorized output?

I completly agree, it's far better and simpler to keep the consistency between both. So the solution would be to colorize the output on the documentation and allow it in the Output comment. No ? Otherwise, it just can't be done.

@mvdan
Copy link
Member

mvdan commented Sep 13, 2019

If you're suggesting to make godoc.org interpret terminal escape codes, or become a terminal emulator, I doubt that's likely to happen :)

Edit: to clarify, that seems like a lot of added complexity for little gain.

@ALTree
Copy link
Member

ALTree commented Sep 13, 2019

@GuillaumeLescure yes, it doesn't look the same, it's obviously a workaround... My point is that there's a workaround that still somewhat allows you do what you want ("show the user what the output should look like (even without colors),") without changing godoc for everyone. It a corner case, after all.

@GuillaumeLescure
Copy link
Author

If you're suggesting to make godoc.org interpret terminal escape codes, or become a terminal emulator, I doubt that's likely to happen :)

That would be nice to keep consistency indeed but I understand the price it cost. That's why I didn't suggest that at first and just do the check without the interpretation.

But at this point we have 3 solutions :

  • do nothing and every terminal color tag will fail the test (could be very annoying for those that want a nice documentation but also use colored output like for logs)
  • make godoc.org an other terminal emulator, heavy work, little gain, add complexity to godoc, etc ...
  • allow to interpret string literally and ignore thoses tag in the render of the website (my original solution) but we lost consistency between input and output

I think all of them are bad and force a trade-off ... something I hate X)

@GuillaumeLescure
Copy link
Author

GuillaumeLescure commented Sep 13, 2019

@ALTree agree. I just want to show the issue, if it's already known and decision is made, I can live with it. But I think it's not corner case. It can happen very often for many people.

Ex : you use a package that have a dependence on a log system that colorize output but you cannot change the output (yeah a not-so-good package). So everytime you will use that package (maybe the whole project like Qt library) you will display colorized output in your shell. So after writting your app, you will not be able to write a documentation with exemple 😢

PS : and a workaround stands for "working around the issue" so there is an issue 😉

@ianlancetaylor
Copy link
Contributor

I think that a package that always colorizes output even when not writing to a terminal, and that has no way to disable that colorization, is, as you say, a not-so-good package. I don't see a real need to make the support for examples more complex in order to support not-so-good packages.

@GuillaumeLescure
Copy link
Author

I think that a package that always colorizes output even when not writing to a terminal, and that has no way to disable that colorization, is, as you say, a not-so-good package. I don't see a real need to make the support for examples more complex in order to support not-so-good packages.

Ok I understand. I think it's probably the best solution here.

@golang golang locked and limited conversation to collaborators Sep 13, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants