New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
net: udp packets not load balanced by kubernetes service #31427
Comments
Looks like your expectation on the distribution relies on k8s. Why do you think this is an issue of Go's standard library? |
Yeah it can be on either sides, of course - but if I use netcat, it gets distributed correctly, so I guess it has something to do with the go net library. |
Can you please provide a self-contained small example only using Go? Also, please make sure the detail of your "traffic distribution"; what are the targets, just packets are a bit vague, UDP packets? and the endpoints? UDP sockets? To me, your example makes a UDP socket just listening to "0.0.0.0:3333", all available unicast and anycast addresses on the node w/ transport port # 3333, and doesn't have any traffic distribution magic. |
This is not an issue of package The script calls Your Go code just creates one socket and send many packets through it. All the packets have the same local port. Generally, If some UDP packets have the same local port and remote port, they will be assumed to be belong to one "connection". The packets of one "connection" won't be routed to different backends. |
@crvv this is possibly the answer I was looking for, thank you! What in the stack is making that assumption, is it Kubernetes proxy or just Linux networking? Can I affect it - can I change the client port in net library for each call or do I have to reconnect it every time? (that might be expensive) |
The attached files in the bug are the self contained small example and they only use G, updated comments to add some of the details - thanks! |
So made more investigation - does not look like the host iptables are involved, this is something I need to bring to the Kubernetes network stack - will open an issue there |
Opened this issue in kubernetes: kubernetes/kubernetes#76517 |
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
sudo apt-get update && sudo apt-get install golang
tells me I am on the latest version.What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
I have a simple udp server that just receives packets deployed as a Kubernetes Service with 3 instances.
Then with a simple udp client I call
conn = net.Dial
and in a forever loop I callconn.Write
What did you expect to see?
UDP packets are randomly distributed across the three pods.
What did you see instead?
UDP packets ALWAYS go to the same server until I stop the client.
If I try to use netcat, the packets correctly go to the right server.
Server:
Client:
Kubernetes deployment:
Kubernetes service:
launch the client with:
verify that the packets go to the same server until you stop and restart the client
run a test from bash with:
verify that the packets end on different servers
The text was updated successfully, but these errors were encountered: