Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

os: (*File).ReadAt() would probably fail if file is opened with O_DIRECT flag #65135

Closed
hzzb opened this issue Jan 17, 2024 · 2 comments
Closed

os: (*File).ReadAt() would probably fail if file is opened with O_DIRECT flag #65135

hzzb opened this issue Jan 17, 2024 · 2 comments

Comments

@hzzb
Copy link
Contributor

hzzb commented Jan 17, 2024

Go version

go version go1.21.6 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/root/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/root/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.6'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='0'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2417799151=/tmp/go-build -gno-record-gcc-switches'

What did you do?

package main

import (
	"fmt"
	"os"
	"syscall"
)

func main() {
	filePath := "main.go"
	flags := os.O_RDONLY | syscall.O_DIRECT
	fil, err := os.OpenFile(filePath, flags, 0644)
	if err != nil {
		fmt.Printf("failed to open file: %s\n", err)
		return
	}

	buf := make([]byte, 10)
	_, err = fil.ReadAt(buf, 0)
	if err != nil {
		fmt.Printf("failed to read file: %s\n", err)
		return
	}
	fmt.Printf("succeed to read file\n")
}

What did you see happen?

root@host:/opt# go run main.go
failed to read file: read main.go: invalid argument

What did you expect to see?

succeed to read file that is opened using the O_DIRECT flag.

From the man page, it seems required to pass perfect arguments to ReadAt() so that buffer address, length, offset are all aligned.

As go is a easy-to-use language, is it worthy to handle alignment requirements inside the standard pkg, providing a more user-friend api.
@hzzb hzzb changed the title os: (*File).Pread() would probably fail if file is opened with O_DIRECT flag os: (*File).ReadAt() would probably fail if file is opened with O_DIRECT flag Jan 17, 2024
@seankhliao
Copy link
Member

ify you're using OpenFile then it's on you to set the right flags.

@seankhliao seankhliao closed this as not planned Won't fix, can't repro, duplicate, stale Jan 17, 2024
@hzzb
Copy link
Contributor Author

hzzb commented Jan 18, 2024 •
edited

Thanks for your quick reply. It's by intention to open file with the O_DIRCT flag because we want to bypass kernel page cache for performance consideration. The flags we set is right. It's possible to let user application provide strictly aligned arguments when calling ReadAt(). but that's a bit of trivial. We succeed to read file when pass strict-aligned args. Kindly consult go team if it is worthing to take that trivial arguments alignment work into the standard os pkg. By doing so, future users can use ReadAt() with less efforts. community may help contribute if this is worthing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@seankhliao @hzzb