Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

io: Seeker documentation does not match implementations #48316

Closed
bcmills opened this issue Sep 10, 2021 · 7 comments
Closed

io: Seeker documentation does not match implementations #48316

bcmills opened this issue Sep 10, 2021 · 7 comments
Labels
Documentation FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@bcmills
Copy link
Contributor

bcmills commented Sep 10, 2021

What version of Go are you using (go version)?

go1.17

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

N/A

What did you do?

Read https://pkg.go.dev/io@go1.17#Seeker. Compare the behavior it requires to the implementations of that interface in the standard library.

What did you expect to see?

Behavior matching the documentation (emphasis mine).

Seek sets the offset for the next Read or Write to offset, interpreted according to whence: SeekStart means relative to the start of the file, SeekCurrent means relative to the current offset, and SeekEnd means relative to the end. Seek returns the new offset relative to the start of the file and an error, if any.

The use of “and” here implies that implementations must return the current offset regardless of any error.

Seeking to an offset before the start of the file is an error. Seeking to any positive offset is legal, but the behavior of subsequent I/O operations on the underlying object is implementation-dependent.

To me, “is legal” implies “does not result in an error”.

What did you see instead?

Many implementations reject positive offsets that exceed the size of the entity being read:

It isn't obvious to me whether os.File.Seek ever rejects positive offsets, since it passes the offset through to a platform-specific system call.

On error (for example, when the offset overflows or is otherwise invalid, or when an underlying system call fails), most implementations return 0 instead of “the new offset relative to the start of the file”.

@bcmills
Copy link
Contributor Author

bcmills commented Sep 10, 2021

Given that nearly every implementation in the standard library deviates from the documented io.Seeker behavior in the same systematic way, I think we should just update the io.Seeker documentation. I would suggest this wording (changes in bold):

Seek sets the offset for the next Read or Write to offset, interpreted according to whence: SeekStart means relative to the start of the file, SeekCurrent means relative to the current offset, and SeekEnd means relative to the end. Seek returns the new offset relative to the start of the file and or an error, if any.

Seeking to an offset before the start of the file is an error. Seeking to any positive offset is legal may be allowed, but if the new offset exceeds the size of the underlying object the behavior of subsequent I/O operations on the underlying object is implementation-dependent.

@bcmills bcmills added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Sep 10, 2021
@gopherbot
Copy link

Change https://golang.org/cl/349054 mentions this issue: io: relax documented Seeker invariants that do not hold in practice

@bcmills
Copy link
Contributor Author

bcmills commented Sep 10, 2021

The current clause was added in https://golang.org/cl/11403043 (along with the inconsistent io.SectionReader overflow behavior).

(CC @bradfitz @robpike)

@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Sep 14, 2021
@bcmills bcmills added this to the Go1.18 milestone Sep 14, 2021
@gopherbot gopherbot removed the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Sep 14, 2021
@bcmills bcmills self-assigned this Sep 14, 2021
@andig
Copy link
Contributor

andig commented Sep 15, 2021

Seeking to any positive offset is legal

Seeking to any positive offset is legal may be allowed, but if the new offset exceeds the size of the underlying object the behavior of subsequent I/O operations on the underlying object is implementation-dependent.

Does that mean that seeking will never return an error and only error on the next io op? If that's the case- why not simply state so?

@bcmills
Copy link
Contributor Author

bcmills commented Sep 15, 2021

Does that mean that seeking will never return an error and only error on the next io op?

No. In practice (in ~all existing implementations), Seek may or may not return an error for an invalid positive offset — it depends on the specific implementation and on just how invalid the offset actually is.

Some implementations return an error from Seek for any offset past the end of the data. Some never return an error for a positive offset as long as there is a valid underlying file, but do return an error if the file is invalid. Some return an error if the offset is “too positive” (and would overflow an internal representation).

In practice, if you want to use Seek correctly you should check the error from Seek itself but also be aware that the next read may error out if the offset turned out not to be valid.

@andig
Copy link
Contributor

andig commented Sep 15, 2021

Seek may or may not return an error for an invalid positive offset — it depends on the specific implementation and on just how invalid the offset actually is.

In that case I could imagine the following to make it clearer:

If seeking to any positive offset and the new offset exceeds the size of the underlying object, then either Seek may error or the subsequent I/O operation depending on implementation.

@bcmills
Copy link
Contributor Author

bcmills commented Sep 15, 2021

As far as I can tell, the existing Seeker documentation does not require the subsequent I/O operation to error out: it says “the behavior … is implementation-dependent”, and by my reading that could allow any behavior as long as it is documented — from reasonable behaviors like returning io.EOF or some other error, to weird behaviors like looping around to read from some other offset, to downright-hostile behaviors like panicking the program.

For example, strings.Reader returns io.EOF, not a specific error indicating that the offset is out of bounds: https://play.golang.org/p/bCfLGZB3-Ul

Ideally, I think we would require Read operations that start out of bounds to return a non-nil error, and require Write operations that start out of bounds to either resize the underlying data or return a non-nil error — but we don't require either of those behaviors today, and doing so now would be a breaking change beyond the minimum required to make the API consistent with its longstanding implementations.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Documentation FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

3 participants