New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
net: /etc/hosts parsing truncates to 4KB (os.GetPagesize) #21674
Comments
Interesting! Thank you @alexzorin for reporting this issue. Just for easier lookup in the future, here is the permalink to the code you referred to Lines 67 to 72 in 93471a1
|
@rsc I suppose that the seemingly arbitrary os.Getpagesize() limit was set because we wanted a performance hit from reads on page boundaries or that we thought that size was sufficient? I don't see a file limit for a hosts file in either of the below:
|
Change https://golang.org/cl/65671 mentions this issue: |
Hello, I've been tracking the CL and I'm not sure how to add my input, sorry if I'm doing it wrong. I saw a comment by @rsc that worries me a little:
I originally ran into this issue with customers exactly because of entries on specific lines that exceeded 4kB (big NAT-ted cPanel servers). If the system resolver deals with it without a problem I would not expect Go to introduce such a surprising and arbitrary limitation here. I apologise if the original issue made it look like the only concern was the remainder of the file - properly parsing long lines is equally important to my application. |
Change https://golang.org/cl/79516 mentions this issue: |
Due to the way
net/file,open
is implemented, pure-Go DNS resolution cannot handle /etc/hosts files that are larger than the OS page size, which is 4KB by default on Linux platforms.Go silently truncates the data after 4KB and does not parse lines on the threshold, since EOL is never reached for them.
The cgo equivalents for this functionality handle these files properly, and so should Go.
We often see these large hosts files in the web hosting environment (e.g. cPanel), when servers are behind NAT and require large hosts files.
Env: Go1.9, linux/amd64
I have also attached a Dockerfile tarball that reproduces the issue fully - go-4kb-hosts.tar.gz
The text was updated successfully, but these errors were encountered: