-
Notifications
You must be signed in to change notification settings - Fork 18k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
x/mobile/bind/java: buffer overflow in the Java/Go parameter serialization #9251
Comments
Wouldn't you want to continue following the same methodology of doubling, instead of introducing a more arbitrary size? ...
if (m->off+size > m->cap) {
uint32_t cap = 2*m->cap;
while (cap < m->off+size) {
cap = 2*cap;
}
m = mem_resize(m, cap);
}
uint8_t *res = m->buf+m->off;
m->off += size;
m->len += size;
return res; |
Yes, always doubling is more in keeping with the original code. This is a bit off the topic of the issue, but it is interesting to consider what's a good trade-off here, especially since function parameter signatures and parameter sizes may vary across usage of this facility. Our app passes JSON data as strings from Java to Go. In the specific case of our Java->Go function calls, always doubling isn't an optimal strategy since there's only one "big" parameter. Allocating ~2x the necessary size might be a concern on a memory constrained Android device (although native code isn't as constrained as the Java VM; and, of course, once the data gets to be a certain size we'd no longer be passing it around as strings). For what it's worth, code such as Java's ArrayList and Android Java's AbstractStringBuilder (see enlargeBuffer) appear to allocate just enough memory when trying to add additional capacity that's not within the next factor size. I think both of these cases use a 1.5x factor as well. |
Thanks for the detailed bug report, fix coming soon. There are a lot of possible strategies for allocation and I agree it is not obvious what is optimal. However I suspect it is the wrong problem to focus on: when I wrote gobind I was talking to Go as a subprocess, so everything needed to be serialized. It seems entirely possible we can now avoid a lot of allocations, and write directly from seq_android.c into Go memory. Especially for tasks like passing JSON around, where almost certainly the Java encoder is capable of producing UTF-8. The ideal type signature there would be []byte in Go and byte[] in Java, with the bytes written directly from Java to Go. (And avoiding the UTF16 -> UTF-8 conversion.) |
The new testLongString triggers the bug without this change. Fixes golang/go#9251. Change-Id: I463e2897b5b08f53801f151c7311d591546c0719 Reviewed-on: https://go-review.googlesource.com/1373 Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
There appears to be a bug in the buffer resize logic in
mem_write
inmobile/bind/java/seq_android.c
? The buffer is resized to at most 2x its previous capacity, even if the new data to be appended is much longer. Ultimately the data is written to the buffer even though the buffer is too small, and the heap is corrupted and the program crashes. In the case of a serialized string parameter,mem_write
is called fromJava_go_Seq_writeUTF16
which is also the function that actually writes to the undersized buffer.To reproduce, pass a long string to Go code using the bind parameter serialization mechanism. For a function that takes a single string, the default buffer size of a
Seq
is 64 bytes, andmem_write
will at most double it to 128 bytes, and each character requires 2 bytes, and there's a 4 byte length prefix, so any string over ~62 characters will corrupt the heap (but only significantly longer strings will crash the program consistently, a 300 character string in my test case).In the
gobind
generated Java code, immediately after the_in.writeUTF16(s)
, for example, you can callSeq.log()
to log the buffer state and observe that thelen
>cap
. At some random point after the buffer overflow, Android logcat displays "heap corruption detected by dlmalloc" and the process terminates.This is the current resize logic (https://github.com/golang/mobile/blob/551dbd1f07f5826104e50893b453bcf6273fbeaa/bind/java/seq_android.c#L81):
This seems to fix it (with this change the
Seq.log()
output looks correct and the process doesn't crash):The text was updated successfully, but these errors were encountered: