net: use EDNS to increase DNS packet size #51153
Comments
|
CC @mdempsky |
|
Although this is late in cycle, this is a small, focused change that we intend to backport regardless, which makes sense to accept. We've discussed a bit and are supportive of merging this before the RC. |
toothrot
added
the
NeedsFix
|
I agree that compliant DNS servers shouldn't have any issues with the use of EDNS. But the rationale for this CL is to accommodate non-compliant servers that send invalid responses, so it seems like we should also be mindful about non-compliant servers/firewalls that break EDNS. Will there be more release candidates before Go 1.18? If so, I think including in Go 1.18 seems reasonable. I'm not sure why this is important to backport though? |
|
Backporting is justified to me by the comments in "Miscellany" in #51127. This tooling is critical for some users, and forcing them to upgrade to (as of yet unreleased 1.18) isn't a great workaround. I assume this impacts the Go command as well. |
|
The go command almost always links to libc, which means using the platform C resolver by default and not the netgo one. |
When compiled with libc, the net package decides at runtime whether to use cgo or go resolver |
|
@gopherbot Please open backport issues. |
|
Backport issue(s) opened: #51161 (for 1.16), #51162 (for 1.17). Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases. |
|
@ianlancetaylor This will cause new failures because |
|
@ianlancetaylor your CL https://go-review.googlesource.com/c/go/+/385035 landed 10 hours ago to the master branch. Shall we thus close this issue? |
|
@fweimer-rh Do you think that we need to worry about those possibilities in practice? I don't see anything in glibc, although it's true that that case is different because the user must explicitly opt-in to EDNS in the resolv.conf file. If those are real possibilities then I will revert the CL. It's too late in the release cycle to add that kind of complexity. Thanks. |
|
@ianlancetaylor I expect some problems. How widespread they are I don't know. I'm not aware of any stub resolver implementations that use EDNS0 by default and without any fallback. Go's implementation might be the first one with wide deployment. This change seems rather risky to me at this point. |
|
An alternative would be to merge #51128 or a variant thereof, which does not advertise EDSN0 support in the request but accepts a larger (safe) buffer size by default. This is akin to what glibc does, though reading C and executing it are often two different things, it looks to me like glibc may behave better here solely because it allocates a larger buffer, without advertising EDSN0 support. A common cause of Allocating a ~1200 byte buffer by default, or perhaps 1024 byte is all that's needed, would mitigate this issue without additional protocol implications. Glibc defaults to 1024 bytes, I think? |
|
@AaronFriel The initial buffer sizes vary on the NSS functions used (1024 and 2048, see |
|
Fair, I also wouldn't use glibc as a reference. I'm merely offering a compromise to maximize compatibility with real world DNS resolvers without advertising full EDNS0 support. |
|
After discussion, we currently plan to do this:
|
|
Change https://go.dev/cl/386015 mentions this issue: |
|
Change https://go.dev/cl/386016 mentions this issue: |
|
Change https://go.dev/cl/386014 mentions this issue: |
This reverts https://go.dev/cl/385035. For 1.18 we will use a simple change to increase the accepted DNS packet size, to handle what appear to be broken resolvers that don't honor the 512 byte limit. For 1.19 we will restore CL 385035 to make a proper EDNS request, so that it has more testing time before it goes out in a release. For #6464 For #21160 For #44135 For #51127 For #51153 Change-Id: Ie4a0eb85ca0a6a73bee5cd4cfc6b7d2a15ef259f Reviewed-on: https://go-review.googlesource.com/c/go/+/386014 Trust: Ian Lance Taylor <iant@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Damien Neil <dneil@google.com>
|
Change https://go.dev/cl/386034 mentions this issue: |
|
Change https://go.dev/cl/386035 mentions this issue: |
ianlancetaylor
changed the title
|
OK, changes made. Moving this issue to 1.19 to use EDNS. |
The existing value of 512 bytes as is specified by RFC 1035. However, the WSL resolver reportedly sends larger packets without setting the truncation bit, which breaks using the Go resolver. For 1.18 and backports, just increase the accepted packet size. This is what GNU glibc does (they use 65536 bytes). For 1.19 we plan to use EDNS to set the accepted packet size. That will give us more time to test whether that causes any problems. No test because I'm not sure how to write one and it wouldn't really be useful anyhow. Fixes #6464 Fixes #21160 Fixes #44135 Fixes #51127 For #51153 Change-Id: I0243f274a06e010ebb714e138a65386086aecf17 Reviewed-on: https://go-review.googlesource.com/c/go/+/386015 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
ianlancetaylor
added
the
early-in-cycle
…1232 bytes The existing value of 512 bytes as is specified by RFC 1035. However, the WSL resolver reportedly sends larger packets without setting the truncation bit, which breaks using the Go resolver. For 1.18 and backports, just increase the accepted packet size. This is what GNU glibc does (they use 65536 bytes). For 1.19 we plan to use EDNS to set the accepted packet size. That will give us more time to test whether that causes any problems. No test because I'm not sure how to write one and it wouldn't really be useful anyhow. For #6464 For #21160 For #44135 For #51127 For #51153 Fixes #51162 Change-Id: I0243f274a06e010ebb714e138a65386086aecf17 Reviewed-on: https://go-review.googlesource.com/c/go/+/386015 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> (cherry picked from commit 6e82ff8) Reviewed-on: https://go-review.googlesource.com/c/go/+/386035 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
…1232 bytes The existing value of 512 bytes as is specified by RFC 1035. However, the WSL resolver reportedly sends larger packets without setting the truncation bit, which breaks using the Go resolver. For 1.18 and backports, just increase the accepted packet size. This is what GNU glibc does (they use 65536 bytes). For 1.19 we plan to use EDNS to set the accepted packet size. That will give us more time to test whether that causes any problems. No test because I'm not sure how to write one and it wouldn't really be useful anyhow. For #6464 For #21160 For #44135 For #51127 For #51153 Fixes #51161 Change-Id: I0243f274a06e010ebb714e138a65386086aecf17 Reviewed-on: https://go-review.googlesource.com/c/go/+/386015 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> (cherry picked from commit 6e82ff8) Reviewed-on: https://go-review.googlesource.com/c/go/+/386034 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
|
Change https://go.dev/cl/389634 mentions this issue: |
For #51153 Change-Id: I4374c63498b62ba7a08f146eebd034cbd50623f6 Reviewed-on: https://go-review.googlesource.com/c/go/+/389634 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
|
Change https://go.dev/cl/415579 mentions this issue: |
We use EDNS(0) by default. No need to fall back to netdns=cgo if we see a explicit request for EDNS(0) in resolv.conf. For #51153 Change-Id: I135363112e3de43ce877aad45aba71d1448068b7 Reviewed-on: https://go-review.googlesource.com/c/go/+/415579 Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Damien Neil <dneil@google.com>
Issue #51127 lists a bunch of cases where the Go DNS resolver has trouble because it follows RFC 1035 and only accepts 512 byte packets. https://go.dev/cl/385035 uses EDNS(0) to advertise that the resolver accepts a larger packet size, and accepts packets up that increased size (1232 bytes per https://dnsflagday.net/2020/). The CL is reported to fix the immediate problem found in #51127, that the Go resolver fails on WSL: #51127 (comment).
I would like to request a freeze exception to include https://go.dev/cl/385035 in the 1.18 release. If acceped for 1.18 I'll also open backport issues to earlier releases.
The CL should be safe. Accepting a larger packet size does not cause harm. EDNS(0) was designed to be backward compatible with DNS servers that don't understand it; per RFC 1035, released in 1987, they should just ignore it. EDNS(0) was first specified in RFC 2671 in 1999, and restated and refined in RFC 6891 in 2013. It is unlikely that there any DNS servers out there that can't at least ignore it.
CC @golang/release
The text was updated successfully, but these errors were encountered: