New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bufio: the way to determine bufio.Reader
size
#21343
Comments
@gobwas could you give a small example of a program that would show the need for this? I can't picture why knowing the available (i.e. unused, not total) number of bytes in a buffer would help you reuse it. |
@mvdan no, I want to get exactly the total number of bytes: var pool = map[int]*sync.Pool{
16: &sync.Pool{},
32: &sync.Pool{},
// ...
}
func GetReader(r io.Reader, n int) (br *bufio.Reader) {
// round n to power of two somehow.
if p, ok := pool[n]; ok {
br, _ = p.Get().(*bufio.Reader)
}
if br == nil {
br = bufio.NewReaderSize(r, n)
}
return br
}
func PutReader(br *bufio.Reader) {
n := ... // get the size of br's underlying buffer.
if p, ok := pool[n]; ok {
br.Reset(nil)
p.Put(br)
}
} |
The only way I could get the size is this workaround: type optimisticReader struct{}
func (optimisticReader) Read(p []byte) (int, error) {
return len(p), nil
}
func readerSize(br *bufio.Reader) int {
br.Reset(optimisticReader{})
br.ReadByte()
return br.Buffered() + 1
} |
You said you need something in You can set the size when you create the buffer. Can you not keep track of it? Exporting fields and methods is always an option, but it always comes at the cost of extra complexity and maintenance work. The fact that I don't recall anyone (nor me) saying that they needed this before makes me wonder if the tradeoff would be worth it. |
I'm pretty sure I had a TODO in some repo to expose the bufio.Reader size but grepping around, I can't find it. But this is also something I've wanted a few times now. |
Sorry for missing details in my first message, @mvdan. I meant |
@mvdan asked above why you need this. I haven't seen anyone answer that. |
@gobwas I think the request is not unreasonable, but before we add more to this API, is there any reason why b.Reset(nil) followed by b.Buffered() is not an option? In other words, since this is presumably about re-use of buffers (I guess to avoid problems with excessive allocation), is there a problem with resetting the buffer to get to the number? And if so, what is the problem? Finally, since you are maintaining the buffers anyway, couldn't you just keep track of the sizes as well (as @mvdan already asked)? |
@griesemer what do you expect to get after |
@gobwas Never mind - b.Buffered() wouldn't work of course. But as a work-around, an additional wrapper would work, or would it not? (ignoring the undesirability for a moment). |
@griesemer this workaround would work too without wrappers. But I want it to be a native method for |
FWIW this works:
but it does rely on Peek not setting the capacity of the returned buffer, which I'm not sure is covered by the compatibility guarantee. |
@rogpeppe thanks =) Yes, this looks like much shorter but still workaround. |
Ping ) |
Sorry, this isn't a case where pinging will make anything happen faster. We need to decide whether and how to expand the API. The issue is marked to be resolved for the 1.10 release, and we'll do our best. |
This looks OK. I'll send a CL.
|
Change https://golang.org/cl/75150 mentions this issue: |
I needed a warm-up task so I sent off https://go-review.googlesource.com/#/c/go/+/75150 |
Hello!
For reusing purposes, I need to determine the
bufio.Reader
underlying buffer size.That is, for
bufio.Writer
there isbw.Available()
method. Is there (or planning to be) something similar forbufio.Reader
?The text was updated successfully, but these errors were encountered: