Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bufio: textproto's bufio design causes half closed idle connections on TCP connection closed by a remote server #53930

Closed
kayrus opened this issue Jul 18, 2022 · 6 comments

Comments

@kayrus
Copy link

kayrus commented Jul 18, 2022

Long living (e.g. keepalive, connection reuse) textproto client connections don't get an event when a remote server closes a connection causing CLOSE_WAIT half closed TCP connections on the client side.

See an example: https://gist.github.com/kayrus/31023f2a1cc30b1744b3c45cd3557d21

$ go run .
220  ESMTP Service Ready
EHLO client
250-Hello client
250-PIPELINING
250-8BITMIME
250-ENHANCEDSTATUSCODES
250-CHUNKING
250-AUTH PLAIN
250 SIZE
MAIL FROM:<test@test.com> BODY=8BITMIME
250 2.0.0 Roger, accepting mail from <test@test.com>
RCPT TO:<test@test.com>
250 2.0.0 I'll make sure <test@test.com> gets this
DATA
354 2.0.0 Go ahead. End your data with <CR><LF>.<CR><LF>
test
.
250 2.0.0 OK: queued
221 2.4.2 Idle timeout, bye bye
2022/07/18 10:51:43 tcp       34      0 127.0.0.1:59336         127.0.0.1:1025          CLOSE_WAIT  22336/smtp-test

I see that for an HTTP client go lang has a workaround. IMHO this solution is too complicated and I wonder whether it's possible to implement something general and reuse it for HTTP/SMTP/etc.

see also emersion/go-smtp#185

@seankhliao
Copy link
Member

The CLOSE-WAIT is from your client never closing the connection.
Since you never interact with the connection after the close event, I don't see how this could be handled. The same goes for plain net.Conn / net.TCPConn connections.
If you want to maintain a connection pool, you'd be responsible for ensuring that an application level healthcheck/keepalive can still go through. HTTP/SMTP are very different protocols in that regard.

I don't see anything to do here.

@seankhliao seankhliao closed this as not planned Won't fix, can't repro, duplicate, stale Jul 18, 2022
@kayrus
Copy link
Author

kayrus commented Jul 19, 2022

@seankhliao thanks for your reply. Nevertheless I disagree with you. Without bufio it is quite easy to detect that the connection is closed. But with bufio you always need to Peek(1) in advance. I don't know how exactly it could be done, I'm not an expert in OS TCP stack or bufio package, but I'd suggest to introduce some internal channel, which can detect in advance that the TCP connection is closed and an interface that notifies about the closed connection.

@ianlancetaylor
Copy link
Contributor

The socket API does not provide any mechanism for detecting that a TCP socket is closed without reading from it.

@kayrus
Copy link
Author

kayrus commented Jul 20, 2022

HTTP/SMTP are very different protocols in that regard

@seankhliao generally they are quite similar, they both use textproto wrapped around bufio. And you can see that http transport implementation peeks bufio in a loop (Peek(1)) to detect the closed connection.

@ianlancetaylor, my aim is to avoid reinventing the wheel and have a general implementation to detect a closed connection.

@ianlancetaylor
Copy link
Contributor

@kayrus If you can find a general mechanism, then, great. I don't know of one.

@kayrus
Copy link
Author

kayrus commented Aug 31, 2022

with my latest change to go-smtp where I'm checking for a connection + buffering what is read + write the buffer down the test looks better: the connection is closed.

the recent test (see https://gist.github.com/kayrus/31023f2a1cc30b1744b3c45cd3557d21) closes the connection properly.

I'm sure that the similar logic can be done in bufio. @ianlancetaylor what do you think?

@golang golang locked and limited conversation to collaborators Sep 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants