New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proposal: bufio: add a new method "Handover" for Writer and Reader #46758
Comments
What you are asking for is not race safe. |
If you want to implement this logic, insert your own type between bufio and net Conn. Then you have the hooks to reopen network connections as you desire. |
That results in one extra data copy which I'm trying to avoid. |
Btw, I don’t think this will be reliable because data may be lost between the sender and receiver in such a way that you cannot correctly determine the number of bytes received so you can correctly rewind the writer. This problem of distributed coordination is considered non trivial. |
I don’t believe this is correct. io.Writer does not require intermediaries to buffer data. |
The thing is, there is already some unflushed buffered data in bw.buf, how can I migrate it to a new |
I don’t think think you need to module bufio to do what you’re asking. I also think what you are asking is unsound because tcp will lie to you. I think you can prove this to yourself, or prove me wrong, with an straight net.Conn. If you can build a wrapper type that can handle the underlying net.Conn going away and being reconnected, then wrapping a bufio Writer around that whole thing for convenience can be done afterwards. |
Let's just put the TCP aside, I still think that Discarding all buffered data is also unsound to me from where I stand. |
Maybe, but why should the ability to switch underlying writers be restricted to just bufio? That feels like the domain of a different type which itself implements io.writer |
In case you didn't notice, actually |
It just really bothers me that bufio allows users to switch the underlying |
I agree with @davecheney : write your own type that does what you need. There is nothing particularly special about |
My case might be specific, but it should be all-purpose for a new method that is like Reset() but retains buffered data. I only brought my case out for illustrating the requirement and I believe there are more scenarios. |
Change https://golang.org/cl/329569 mentions this issue: |
As others have noted, the easy way to do this is to make your own Reader/Writer implementation that you pass to bufio.
There is literally no code required, just a |
Thank you @rsc for the solution which is actually similar to my current implementation, I just come here to ask for opinions from the Go team about adding this ability to Thanks again for your advices and time. |
This proposal has been added to the active column of the proposals project |
Based on the discussion above, this proposal seems like a likely decline. |
No change in consensus, so declined. |
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
bufio.Writer:
I set up a
net.Conn
c as the underlyingio.Writer
wr ofbufio.Writer
bw and replaced bw.wr with abytes.Buffer
bb (by callingbufio.Reset(bb)
) when the TCP socket disconnected caused by any network problem, so other goroutines can continue callingbufio.Write()/Flush()
to write/flush data into the local buffer bb (bw.wr), not knowing the network problem, I will create a newnet.Conn
and replace bw.wr with it, then copy all data in bb to bw.buf once the TCP socket resumes.Because the bw was shared by multiple goroutines (goroutine-safe along with mutex), some of goroutines might have called
bufio.Write()
writing data into the bw.buf beforebufio.Reset(bb)
was called, which means those data will be lost, when I reconnected to the remote server, I want to send all buffered data (including data was in bw.buf beforebufio.Reset(bb)
was called) to the remote server.What did you expect to see?
All buffered data was sent to the remote server after a reconnection.
What did you see instead?
I lost some data by calling
bufio.Reset(bb)
.So I propose to add a new method "Handover" for Writer and Reader, Handover is the same as Reset, except it retains all unflushed buffered data and migrates them to the new underlying
io.Writer
.It would be something like this:
I can call
bw.Handover(bb)
to retain all unflushed buffered data and flush it tobytes.Buffer
bb instead of callingbw.Reset(bb)
to discard all unflushed buffered data in bw.buf.The text was updated successfully, but these errors were encountered: