Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bufio.Reader#ReadLine() returns a wrong result when reading a long line with 1 million chars #65191

Closed
xuanswe opened this issue Jan 21, 2024 · 4 comments

Comments

@xuanswe
Copy link

xuanswe commented Jan 21, 2024

Go version

Go 1.21 (go.dev/play)

Output of go env in your module/workspace:

Go 1.21 (go.dev/play)

What did you do?

https://go.dev/play/p/W_LvqY2Qu84

func main() {
	reader := bufio.NewReader(strings.NewReader("abcdefghijklmnopqrstuvwxyz" + strings.Repeat("a", 1000000)))

	line, isPrefix, _ := reader.ReadLine()

	// "abcdefghijklmnopqrstuvwxyz" => correct
	fmt.Printf("Prefix of 'line' after the first call to ReadLine() %v\n", line[:26])

	for isPrefix {
		var tmp []byte
		tmp, isPrefix, _ = reader.ReadLine()
		line = append(line, tmp...)
	}

	// TODO: Why not "abcdefghijklmnopqrstuvwxyz" ???
	fmt.Printf("Prefix of 'line' after complete reading a line %v\n", line[:26])

	fmt.Println(len(line)) // 1000026
}

What did you see happen?

Prefix of 'line' after the first call to ReadLine() [97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122]
Prefix of 'line' after complete reading a line [97 97 97 97 97 97 97 97 97 97 97 97 97 97 97 97 97 97 97 97 97 97 97 97 97 97]
1000026

What did you expect to see?

Prefix of 'line' after the first call to ReadLine() [97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122]
Prefix of 'line' after complete reading a line [97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122]
1000026
@xuanswe
Copy link
Author

xuanswe commented Jan 21, 2024

ReadString() works correctly

https://go.dev/play/p/Y_PiQYJzEgZ

package main

import (
	"bufio"
	"fmt"
	"strings"
)

func main() {
	reader := bufio.NewReader(strings.NewReader("abcdefghijklmnopqrstuvwxyz" + strings.Repeat("a", 1000000)))

	// This only works on Windows (\r\n), Unix (\n)
	// TODO: how to handle Mac's EOL, which is '\r'?
	line, _ := reader.ReadString('\n')
	line, _ = strings.CutSuffix(line, "\n")
	line, _ = strings.CutSuffix(line, "\r")

	// "abcdefghijklmnopqrstuvwxyz" => correct
	fmt.Printf("Prefix of 'line' %v\n", line[:26])

	fmt.Println(len(line)) // 1000026
}

@xuanswe
Copy link
Author

xuanswe commented Jan 21, 2024

I am new to Go, so if I am doing smt wrong, please help me correct my code. Thanks!

@seankhliao
Copy link
Member

The returned buffer is only valid until the next call to ReadLine

Unlike many projects, the Go project does not use GitHub Issues for general discussion or asking questions. GitHub Issues are used for tracking bugs and proposals only.

For questions please refer to https://github.com/golang/go/wiki/Questions

@seankhliao seankhliao closed this as not planned Won't fix, can't repro, duplicate, stale Jan 21, 2024
@xuanswe
Copy link
Author

xuanswe commented Jan 21, 2024

The returned buffer is only valid until the next call to ReadLine

Oh, thanks. I thought that it's a bug. But it's not.

@golang golang locked and limited conversation to collaborators Jan 20, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants