Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bufio: Scanner.Scan() behavior changed for empty reader #66946

Closed
nathan-osman opened this issue Apr 21, 2024 · 2 comments
Closed

bufio: Scanner.Scan() behavior changed for empty reader #66946

nathan-osman opened this issue Apr 21, 2024 · 2 comments

Comments

@nathan-osman
Copy link

Go version

go version go1.22.2 linux/arm64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_arm64'
GOVCS=''
GOVERSION='go1.22.2'
GCCGO='gccgo'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3317454097=/tmp/go-build -gno-record-gcc-switches'

What did you do?

I have created a simple test program that reproduces the issue:

package main

import (
	"bufio"
	"fmt"
	"runtime"
	"strings"
)

func main() {
	fmt.Printf("Go version: %s\n", runtime.Version())

	s := bufio.NewScanner(strings.NewReader(""))
	s.Split(func(data []byte, atEOF bool) (advance int, token []byte, err error) {
		err = bufio.ErrFinalToken
		return
	})
	fmt.Println(s.Scan())
}

Playground link: https://go.dev/play/p/HXAIfzWJ_cN

What did you see happen?

Using Go 1.22.2, I get the following output:

Go version: go1.22.2
false

But earlier versions of Go have the following output:

Go version: go1.21.9
true

A bit of digging seems to point to this commit: bc2124d — however, the updated documentation really isn't clear about what return value should be expected under this scenario.

"[Scan] returns false when there are no more tokens"

"...and a nil token with ErrFinalToken immediately stops the scanning."

What did you expect to see?

I expected to see identical behavior between versions.
@seankhliao
Copy link
Member

given the context in #56381, the new behavior seems correct. is there any reason to believe otherwise?

@ianlancetaylor
Copy link
Contributor

This seems to be exactly the behavior described the release note, in the bufio section of https://go.dev/doc/go1.22#minor_library_changes. Closing as working as intended.

@ianlancetaylor ianlancetaylor closed this as not planned Won't fix, can't repro, duplicate, stale Apr 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@nathan-osman @ianlancetaylor @seankhliao