-
Notifications
You must be signed in to change notification settings - Fork 17.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
os,fmt: Documentation should mention that os.Stdout, and fmt.Print* functions are unbuffered #36619
Comments
@kohlrabi thanks for writing this up.
The flush behavior is documented for
I think this level of control is consistent with Go's API design of generally requiring the programmer to be explicit, and including tools like I'm not sure if I agree on explicitly calling out that /cc @ianlancetaylor |
I can't see how it makes sense to add any comment about buffering to the fmt package. For the os package a short sentence or two somewhere might be OK. |
agreed wasted hours of time debugging on this... |
It does not look to work as unbuffered. I launch the Go program as a subprocess with popen from a program written in another language and see it puts |
@Nakilon Unbuffered output means that concurrent output from multiple goroutines can be interleaved. |
@ianlancetaylor ah, ok, I thought it means it gets flushed immediately preventing the mess. But in my case I don't do anything concurrent, my program is pretty simple. It just accumulates JSON strings and then prints them out when asked. After this edit the issue appeared: diff --git a/satellite.go b/satellite.go
index baec58d..479e769 100644
--- a/satellite.go
+++ b/satellite.go
@@ -12,7 +12,7 @@ func main() {
var err error
var line []byte
var temp interface{}
- var array []interface{}
+ var array [][]byte
stdin := bufio.NewReader(os.Stdin)
for {
if c, err = stdin.ReadByte(); err != nil {
@@ -31,13 +31,10 @@ func main() {
if err = json.Unmarshal(line, &temp); err != nil {
panic(err)
}
- array = append(array, temp)
+ array = append(array, line)
case 103: // g print all and die
fmt.Println(len(array));
- for _, temp = range array {
- if line, err = json.Marshal(temp); err != nil {
- panic(err)
- }
+ for _, line = range array {
fmt.Println(string(line));
}
os.Exit(0) Initially I was parsing the JSON strings (for other needs) and storing them in UPD: other observations -- the corrupted line lacks leading two bytes, and it happens only when the |
@Nakilon That is best discussed on a forum, not on this issue. See https://golang.org/wiki/Questions. Thanks. |
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
I just wanted to print formatted elements of an array, in this case I wanted to prepend a
0x
to every byte in my array. The idiomatic C routine goes somewhere along these lines:print.c
:The performance measured by just running
time ./print
on my machine isNaively using the same approach in go lead to disastrous performance:
print_loop.go
:I
go build
, and then run the resulting binary withtime ./print_loop
:I realize that printing all elements of a byte array is possible by using
"% x"
as format string, and passing the array to that completely, likefmt.Printf("0x% x", n)
, but that is not exactly what I want, since it does not prepend0x
to every byte.I realized by "chance" and some googling (https://stackoverflow.com/questions/13422381/idiomatically-buffer-os-stdout, https://grokbase.com/t/gg/golang-nuts/158cgb3rfd/go-nuts-os-stdout-is-not-buffered=) that
os.Stdout
are unbuffered in go, so I immediately tried to use abufio.Writer
, and everything was "fine". This code is obviously much more verbose, and it is of utmost importance to not forget flushing the buffer, since it apparently does not happen automatically on leavingmain
(and thus exiting):print_buffer.go
:This results in:
The issue is that neither the documentation on
fmt.Print*
, nor the one onos.Stdout
mention that I/O is unbuffered, and people using idiomatic C-style approaches to printing formatted output are in for a huge surprise. I certainly was dumbfounded as to why my simple go code performed so much worse than the equivalent C code.What did you expect to see?
I expect the documentation to mention the buffering behaviour of I/O operations for functions in
fmt
, and foros.Stdin
andos.Stdout
. The documentation forfmt
could also link to theos
documentation, i.e. mention explicitly that the unbufferedos.Stdout
is used.What did you see instead?
Nothing.
The text was updated successfully, but these errors were encountered: