New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
runtime: unexpected fault address runtime.memhash16 #33635
Comments
This is exceedingly strange. Could you disassemble your binary and show us exactly the disassembled code for The fault address is also |
Below is the output from
|
Yes, that's weird. The instruction that's loading the G pointer from the TLS area is faulting, and on the address of the instruction itself, not the address it's loading from. |
We are indeed using cgo libraries: https://github.com/confluentinc/confluent-kafka-go/ and https://github.com/DataDog/zstd would be involved here but the scope of the Unfortunately, I don't think we can hand over a binary for this particular service. Are there any debug steps that we can take on our end to gather more data for you? As I mentioned in the initial post, this service has been running for quite a long time and has only crashed twice with this error, I'm not quite sure how we could reproduce it. |
If you can catch it crash in a debugger, a dump of the memory regions with their permissions would be nice. Try running on a different machine. We've seen memory corruption issues generate weird bugs before. Those cgo libraries don't seem particularly dangerous. |
Interesting that you bring this up. We are running this application in our k8s environment but I just looked and both of these pods happened to be scheduled on the same underlying VM. |
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
What did you expect to see?
What did you see instead?
Our application is occasionally crashing with the following error:
The code in question looks like:
This particular application is processing messages and calling this once per message at a rate of ~75k / second. We've seen this happen twice over the past week or so.
The text was updated successfully, but these errors were encountered: