debug/elf: uncompress .zdebug sections in (*Section).Open #58254
Comments
|
Right now if you use the Would that fix your problem? |
seankhliao
changed the title
|
Just to reiterate you're suggesting to essentially move the offset handling done in the |
|
OK, taking this out of the proposal process. |
ianlancetaylor
changed the title
ianlancetaylor
added
help wanted
NeedsFix
|
This would be backwards incompatible and require a GODEBUG setting per #56986, right? |
|
It would be backwards incompatible, but not every single backward incompatibility requires a GODEBUG setting. We only need a GODEBUG setting if the change is likely to break a reasonable number of real programs. In this case that seems to me to be unlikely. There may well be programs that are prepared to uncompress the ELF section data themselves, but I would expect those problems to work correctly if the receive uncompressed data. Do you know of examples where that would not work? |
I do not, delve for example will assume the section is not compressed if it doesn't start with ZLIB: https://github.com/go-delve/delve/blob/4303ae45a8e2996b30d2318f239677a771aef9c1/pkg/dwarf/godwarf/sections.go#L88. However I don't think it would be too strange if some program existed that simply errored in that circumstance. The string ZLIB at the start of the section exists to determine the type of compression, if you didn't see ZLIB you could also assume that a different compression is being used (no other compression algorithms are allowed at the moment but someone could write that to future-proof their code). The counterpoint to this is that moving decompression from debug/dwarf to debug/elf just provides a small convenience, users could always write the decompression themselves. Also does the auto-decompression only go in debug/elf or also debug/macho and debug/pe? Compilers other than go do not produce compressed zdebug sections in PE and Mach-O executables. |
|
The .zdebug compression is obsolete, as all new files should be using As far as I can see the suggested change doesn't affect debug/dwarf at all. The code that would change, besides Given that |
The method (*Section).Open automatically decompresses section contents when they are marked with SHF_COMPRESSED. This is the recommended approach in the ELF standard. Some tools use the older mechanism of prefixing the sections with a z as in .zdebug_str for example, this case was unhandled. The new implementation will look for SHF_COMPRESSED as well as sniffing for ZLIB magic bytes and for sections starting with .z. Fixes golang#58254
The method (*Section).Open automatically decompresses section contents when they are marked with SHF_COMPRESSED. This is the recommended approach in the ELF standard. Some tools use the older mechanism of prefixing the sections with a z as in .zdebug_str for example, this case was unhandled. The new implementation will look for SHF_COMPRESSED as well as sniffing for ZLIB magic bytes and for sections starting with .z. Fixes golang#58254
|
Change https://go.dev/cl/513875 mentions this issue: |
|
Already fixed by https://go.dev/cl/480895. |
I would like to propose adding an API to the debug/elf package to fully read sections (compressed and uncompressed alike). Current APIs only allow reading uncompressed sections, or in select cases may handle compressed sections, but do not allow accessing the decompressed bytes of a compressed section.
Why?
I have recently come across the
dwztool, which some Linux distributions (eg. Fedora and Debian) use to optimize debuginfos (as in debuginfos that have already been split off of a binary usingobjcopy --only-keep-debug $FILE $FILE.debug).dwzdeduplicates as much data as possible and subsequently splits the original debuginfos into two files, a primary debuginfo file which has a.gnu_debugaltlinksection added, which links to the second file, which is also a valid elf file, where the deduplicated data lives, the primary debuginfo can then have references to point to where to read deduplicated data from in the supplementary debuginfo file. An example: The primary debuginfo DWARF DIEs, let's say aDW_AT_namecould not be a string, but instead an offset into the supplementary debuginfo file's.debug_stror decompressed.zdebug_strsection, directing us to read a null-terminated string at that offset.Once the bytes of the section are available writing the code to read null-terminated strings at an offset is simple and could have been done separately. If it was just uncompressed
.debug_strwe could just use thedebug/elfpackage's functionality to read the section, however, the issue arises when the section is compressed, as the unexported fieldcompressionOffsetneeds to be set appropriately to read its content successfully.For background: I happen to work on the open-source continuous-profiler, Parca, which uses the
debug/elfanddebuf/dwarfpackages for symbolizing profiling data, and we'd like to support symbolizing these cases.I could see a couple of different APIs for me to achieve my goal, but the ability to read compressed sections would have a minimal API surface and would allow any very special cases that the go runtime itself doesn't need, to be handled separately.
Alternatively, I could imagine an API within the
debug/dwarfpackage that exposed a reader to read strings at a given offset, but that would be easy to misunderstand and misuse I feel.The text was updated successfully, but these errors were encountered: