Related Issues and Documentation

• encoding/gob: panic decoding into local type, received remote type #62117 (closed)
• encoding/gob: roundtrip fail for a struct embedding a field implementing Gob{Decoder,Encoder} #22172 (closed)
• encoding/gob: reusing Encoder results in corrupt encoded data #49844 (closed)
• encoding/gob: Decoder cannot be reused if a composite types are involved #29766 (closed)
• encoding/gob: Weird errors when using gob in more than one goroutine #1238 (closed)
• package GOB "GobEncode()" looping #41496 (closed)
• gob decode nil value occur EOF error #16147 (closed)
• encoding/gob: oom in Decode #53369 (closed)
• encoding/gob: decoding []byte into other uint sizes results in a panic #2662 (closed)
• encoding/gob: incorrect decoding of slices when they are reused across multiple decode iterations #64071 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

Can you show us a small, complete, standalone program that demonstrates the problem? Thanks.

I’m very sorry, but it's difficult to reproduce it as a small, complete, standalone program, so I can't provide a demo.
It only appears after running for some time in the production environment, and the likelihood of reproduction may depend on factors such as packet size and network conditions.

Stack:
https://github.com/golang/go/blob/master/src/syscall/zsyscall_linux_amd64.go#L736
https://github.com/golang/go/blob/master/src/syscall/syscall_unix.go#L183
https://github.com/golang/go/blob/master/src/internal/poll/fd_unix.go#L161
https://github.com/golang/go/blob/master/src/net/fd_posix.go#L55
https://github.com/golang/go/blob/master/src/net/net.go#L189
https://github.com/golang/go/blob/master/src/bufio/bufio.go#L227
https://github.com/golang/go/blob/master/src/io/io.go#L335
https://github.com/golang/go/blob/master/src/io/io.go#L354
https://github.com/golang/go/blob/master/src/internal/saferio/io.go#L37
https://github.com/golang/go/blob/master/src/encoding/gob/decoder.go#L103
https://github.com/golang/go/blob/master/src/encoding/gob/decoder.go#L91
https://github.com/golang/go/blob/master/src/encoding/gob/decoder.go#L148
https://github.com/golang/go/blob/master/src/encoding/gob/decoder.go#L227
https://github.com/golang/go/blob/master/src/encoding/gob/decoder.go#L204

My thoughts:
When reading data(https://github.com/golang/go/blob/master/src/internal/poll/fd_unix.go#L161), syscall.Read sometimes returns syscall.EAGAIN.
At this point, ReadAtLeast(https://github.com/golang/go/blob/master/src/io/io.go#L335) will return a network error 'i/o timeout'.
Therefore, ReadData(https://github.com/golang/go/blob/master/src/internal/saferio/io.go#L37) seems illogical.
When returning a network error 'i/o timeout', it does not handle the partially read data but instead discards it!!!
It seems that there are many similar usages throughout Go, such as (_, err := io.ReadFull(r, buf))...

What I did:
Because packages like strings and net cannot be used, which would cause 'import cycle not allowed,'.
In ReadAtLeast, after the Read, I added:
if err != nil && contains(err.Error(), "i/o timeout") {
err = nil
continue
}
ps:
func contains(s, substr string) bool {
if len(substr) == 0 {
return true
}
if len(s) < len(substr) {
return false
}
for i := 0; i+len(substr) <= len(s); i++ {
if s[i:i+len(substr)] == substr {
return true
}
}
return false
}
It may look ugly, but it works.
If possible, please give me some more elegant suggestions.

Change https://go.dev/cl/609759 mentions this issue: encoding/gob: gob.(*Decoder).Decode failed #69131

Therefore, ReadData(https://github.com/golang/go/blob/master/src/internal/saferio/io.go#L37) seems illogical.
When returning a network error 'i/o timeout', it does not handle the partially read data but instead discards it!!!
It seems that there are many similar usages throughout Go, such as (_, err := io.ReadFull(r, buf))...

There are similar usages because there is nothing wrong with that usage. A call to io.ReadFull means that if we don't get all the data we expect then we have an error. In this context an I/O timeout error is an error. If you are in a situation where it's OK to receive some bytes and then an I/O timeout error, you should not call io.ReadFull.

The encoding/gob package does not set any timeouts, and it does not expect to operate over a connection that can have timeouts and then recover.

I think we can close as working as intended for encoding/gob

I'll note that you could write a custom implementation of io.Reader that ignores timeout errors, and use that with encoding/gob.

There are similar usages because there is nothing wrong with that usage. A call to io.ReadFull means that if we don't get all the data we expect then we have an error. In this context an I/O timeout error is an error. If you are in a situation where it's OK to receive some bytes and then an I/O timeout error, you should not call io.ReadFull.

I understand that io.ReadFull works fine on its own; if an error occurs, it returns the data read up to that point. When there is an I/O timeout, it might be better to handle the partial data in readMessage rather than just using saferio.ReadData.

I'll note that you could write a custom implementation of io.Reader that ignores timeout errors, and use that with encoding/gob.

In fact, I tried using a custom io.Reader and set it to ignore timeouts, but the problem still persists. What’s puzzling is that I found the first syscall.Read encountering an syscall.EAGAIN(I/O timeout) occurs within the same second.

I think we can close as working as intended for encoding/gob

Sorry for the late reply.

It's entirely normal for syscall.Read to return EAGAIN. In Go network connections are in non-blocking mode, and managed through epoll or kqueue. That works by calling syscall.Read until it returns EAGAIN, and then pausing the outer Go Read call until the network connection has data available.