Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bufio: use underlying ReadFrom even when data is buffered #44815

Closed
neild opened this issue Mar 5, 2021 · 6 comments
Closed

bufio: use underlying ReadFrom even when data is buffered #44815

neild opened this issue Mar 5, 2021 · 6 comments

Comments

@neild
Copy link
Contributor

neild commented Mar 5, 2021

(*bufio.Writer).ReadFrom will use the underlying writer's ReadFrom method if present, but only when the bufio.Writer has no data buffered.

When the http package writes a request, it first writes the headers to a bufio.Writer and then uses io.Copy to write the request body. The buffered headers prevent the use of the underlying net.Conn's ReadFrom. Flushing the headers before the body may result in an unnecessary additional write, and possibly split a small request across multiple packets.

A similar problem may occur in other cases where a bufio.Writer has some amount of data written to it, followed by a large copy.

ReadFrom should instead fill out the current buffer, flush it, and then continue the operation with the underlying writer's ReadFrom (when present).

// ReadFrom implements io.ReaderFrom. This calls the underlying writer's ReadFrom method,
// if any. If b has buffered data, this fills the buffer before calling the underlying `ReadFrom`.
func (b *Writer) ReadFrom(r io.Reader) (n int64, err error) {
   ...
}
@neild neild changed the title bufio: use underlying ReadFrom even when data is buffered proposal: bufio: use underlying ReadFrom even when data is buffered Mar 5, 2021
@gopherbot gopherbot added this to the Proposal milestone Mar 5, 2021
@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) Mar 5, 2021
@rsc
Copy link
Contributor

rsc commented Jul 14, 2021

This seems like a completely reasonable optimization.
We might find a surprise, of course, but I don't see anything wrong with it a priori.

@rsc
Copy link
Contributor

rsc commented Jul 28, 2021

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc rsc moved this from Incoming to Active in Proposals (old) Jul 28, 2021
@rsc
Copy link
Contributor

rsc commented Aug 4, 2021

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

@rsc rsc moved this from Active to Likely Accept in Proposals (old) Aug 4, 2021
@gopherbot
Copy link

Change https://golang.org/cl/340530 mentions this issue: bufio: use underlying ReadFrom even when data is buffered

@rsc rsc moved this from Likely Accept to Accepted in Proposals (old) Aug 11, 2021
@rsc
Copy link
Contributor

rsc commented Aug 11, 2021

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

@rsc rsc changed the title proposal: bufio: use underlying ReadFrom even when data is buffered bufio: use underlying ReadFrom even when data is buffered Aug 11, 2021
@rsc rsc modified the milestones: Proposal, Backlog Aug 11, 2021
@neild neild self-assigned this Oct 6, 2021
@cristaloleg
Copy link

Looks like it's done but not merged? @neild

@dmitshur dmitshur modified the milestones: Backlog, Go1.18 Oct 25, 2021
@rsc rsc unassigned neild Jun 23, 2022
@golang golang locked and limited conversation to collaborators Jun 23, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
No open projects
Development

No branches or pull requests

5 participants