I'm filing this bug to track discussion and activity related to a problem in Go's debug/dwarf package related to how DWARF stores information about bitfields (in C structs) the way in which debug/dwarf presents this info to users/clients.

Background:

During Go 1.18 development, a change was made to the debug/dwarf package to add support for the DW_AT_data_bit_offset DIE attribute. The tracking issues for that work is #46784, and the CL eventually submitted was https://go-review.googlesource.com/c/go/+/328709.

The CL in question treats DW_AT_data_bit_offset (added in DWARF 4) and DW_AT_bit_offset (present in DWARF 3 and prior) as equivalent; it changes Go's debug/dwarf package to look first for s DW_AT_data_bit_offset attr (if it is available) and assign that value to the BitOffset field of the StructField object, and if DW_AT_data_bit_offset is not present, it falls back and looks for DW_AT_bit_offset. The resulting StructField is then eventually is passed back to the client as a result of a ReadType call.

This CL was in fact problematic/buggy, due to the fact that the DW_AT_data_bit_offset attribute is not just a clone of DW_AT_bit_offset; the two have different definitions (the former records a bit offset within the enclosing struct itself, whereas the latter records an endian-sensitive offset within the containing storage unit for the bitfield). During the 1.18 release freeze Dmitry discovered this problem and pointed it out (see https://go-review.googlesource.com/c/go/+/328709/comments/edf0619d_daec236f).

Where to go from here:

A first option is just to revert CL 328709 and restore things to the state they were in prior to 1.18. The drawback of doing this is that clients using debug/dwarf to analyze code generated by compilers that are using the more modern attribute (DW_AT_data_bit_offset) will see zero values for BitOffset when making ReadType calls (this is why the bug was filed in the first place).

A second option is to leave StructField unchanged (a single BitOffset value) and then provide some sort of hook that users can call that will tell them how to interpret it (e.g. whether the compilation unit is using DW_AT_data_bit_offset or DW_AT_bit_offset). This might look something like

  // UsesV4DataBitOffset returns TRUE if the current compilation unit
  // makes use of DW_AT_data_bit_offset, FALSE if the unit does not
  // use DW_AT_data_bit_offset.
  func (r *Reader) UsesV4DataBitOffset() bool {
    ...
  }

This has the disadvantage that it requires changes to existing clients: people who use debug/dwarf and actually look at bitfields will need to modify their code. It is also a bit complicated to implement in that it's not possible to determine whether DW_AT_data_bit_offset is in use only by looking at the DWARF version (since some versions of GCC still use the older attribute even when "-gdwarf-4" is requested). This could also be attacked by adding a new field to StructField itself (perhaps a boolean indicating which interpretation to use).

A third option is to look for the new attribute and then use the new attr values to compute the old DWARF-3 values, and present the DWARF 3 values to the user. This would be the best solution in terms of backwards compatibility, but seems unpleasant given that we're effectively undoing what was intended to be a good change made in DWARF 4 (since the new attribute is endian-insensitive and is deemed to be easier for clients to use). It also seems tricky to implement, which makes it less attractive given we are fairly late in the 1.18 freeze.

A fourth option is to simply change the meaning of the field-- just assign the new attr value to BitOffset and expect that clients will interpret it correctly. For compilation units that contain the old attribute, we could just ignore the value (seems unfriendly) or possibly try to compute the new DWARF 4 value using the old attr values. This option obviously has problems in that it requires clients to update their code, and if writing code to compute new attr values from old attr values seems problematic (again given we are fairly late in the 1.18 freeze).

A related question is: how many uses of BitOffset are there out in the wild?

I did an informal audit of Go code on github (using the new/beta "cs.github.com" search feature) looking for uses of "BitOffset" in Go code (e.g. query of the form 'language:go ".BitOffset"'). There were surprising few hits.

Most of what turned up were clones of the Go and GCC repos. There is a reference to BitOffset in Delve (and all the various clones of Delve) but it turns out that they've copied "type.go" from the main Go repo's debug/dwarf package and hacked it up for their own purposes.

The only "real" references I can find are in syzcaller (hence the interest from Dmitry) and its various clones, and then a use in a package https://github.com/volatilityfoundation/dwarf2json, which does look at BitOffset and interprets it according to DWARF 3 rules.

Of course, this is only Github public repos, there may be private repos that use BitOffset and there will be code elsewhere outside of github.