Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

debug/pe: add APIs to support reading COMDAT info for sections #51868

Closed
thanm opened this issue Mar 22, 2022 · 8 comments
Closed

debug/pe: add APIs to support reading COMDAT info for sections #51868

thanm opened this issue Mar 22, 2022 · 8 comments

Comments

@thanm
Copy link
Contributor

thanm commented Mar 22, 2022

This is a proposal to add additional APIs to the "debug/pe" package to
enable reading of auxiliary symbol information, specifically to enable
extraction of info about COMDAT sections, needed by the Go linker.

Background + Motivation

The debug/pe package currently provides hooks for reading symbol table
information from PE object files, notably the "COFFSymbols" slice
provided in pe's "File" type:

  // A File represents an open PE file.
  type File struct {
	FileHeader
	OptionalHeader any // of type *OptionalHeader32 or *OptionalHeader64
	Sections       []*Section
	Symbols        []*Symbol    // COFF symbols with auxiliary symbol records removed
	COFFSymbols    []COFFSymbol // all COFF symbols (including auxiliary symbol records)
	StringTable    StringTable

	closer io.Closer
  }

The COFFSymbols array is something of a fiction, in that the slice
includes entries for "regular" symbols, but also for entries in the
file's symbol array that correspond to aux symbols. Both regular
symbols and aux symbols are 18 bytes in size, but the format of aux
symbols varies depending on the type of regular symbol it precedes.
Picture to illustrate:

   ...
   k+0:  regular sym k
   k+1:    1st aux symbol for k
   k+2:    2nd aux symbol for k
   k+3:  regular sym k+3
   k+4:    1st aux symbol for k+3
   k+5:  regular sym k+5
   k+6:  regular sym k+6
   ...

More can be found in the Microsoft PE format documentation.

The existing debug/pe APIs list the number of aux symbols for each
regular symbol, making it possible to skip aux entries when iterating
through the main symbol array, but there is no way to look inside the
aux symbols.

One class of aux symbol is the section definition
aux symbol, which holds additional bits of info about symbols
corresponding to defined sections in the object file.

One part of the section definition aux symbol is material related to
COMDAT sections; the Go linker needs to be able to access this COMDAT
info in order to support more modern clang-based C toolchains on
Windows (see CL stack at
https://go-review.googlesource.com/q/topic:modernize_windows_c_compiler
and associated issue at https://golang.org/issue/35006).

Proposal details

The proposal is to add these new APIs to debug/pe:

// Section characteristics flags.
const (
	IMAGE_SCN_CNT_CODE               = 0x00000020
	IMAGE_SCN_CNT_INITIALIZED_DATA   = 0x00000040
	IMAGE_SCN_CNT_UNINITIALIZED_DATA = 0x00000080
	IMAGE_SCN_LNK_COMDAT             = 0x00001000
	IMAGE_SCN_MEM_DISCARDABLE        = 0x02000000
	IMAGE_SCN_MEM_EXECUTE            = 0x20000000
	IMAGE_SCN_MEM_READ               = 0x40000000
	IMAGE_SCN_MEM_WRITE              = 0x80000000
)

// COFFSymbolAuxFormat5 describes the expected form of an aux symbol
// attached to a section definition symbol. The PE format defines a
// number of different aux symbol formats: format 1 for function
// definitions, format 2 for .be and .ef symbols, and so on. Format 5
// holds extra info associated with a section definition, including
// number of relocations + line numbers, as well as COMDAT info. See
// https://docs.microsoft.com/en-us/windows/win32/debug/pe-format#auxiliary-format-5-section-definitions
// for more on what's going on here.
type COFFSymbolAuxFormat5 struct {
	Size           uint32
	NumRelocs      uint16
	NumLineNumbers uint16
	Checksum       uint32
	SecNum         uint16
	Selection      uint8
	_              [3]uint8 // Padding
}

// These constants make up the possible values for the 'Selection'
// field in an AuxFormat5.
const (
	IMAGE_COMDAT_SELECT_NODUPLICATES = 1
	IMAGE_COMDAT_SELECT_ANY          = 2
	IMAGE_COMDAT_SELECT_SAME_SIZE    = 3
	IMAGE_COMDAT_SELECT_EXACT_MATCH  = 4
	IMAGE_COMDAT_SELECT_ASSOCIATIVE  = 5
	IMAGE_COMDAT_SELECT_LARGEST      = 6
)

// COFFSymbolReadSectionDefAux returns a blob of axiliary information
// (including COMDAT info) for a section definition symbol. Here 'idx'
// is the index of a section symbol in the main COFFSymbol array for
// the File. Return value is a pointer to the appropriate aux symbol
// struct. For more info, see:
//
// auxiliary symbols: https://docs.microsoft.com/en-us/windows/win32/debug/pe-format#auxiliary-symbol-records
// COMDAT sections: https://docs.microsoft.com/en-us/windows/win32/debug/pe-format#comdat-sections-object-only
// auxiliary info for section definitions: https://docs.microsoft.com/en-us/windows/win32/debug/pe-format#auxiliary-format-5-section-definitions
//
func (f *File) COFFSymbolReadSectionDefAux(idx int) (*COFFSymbolAuxFormat5, error) {

...
}

When a client discovers a section definition symbol that has the IMAGE_SCN_LNK_COMDAT bit set in its characteristics flags, it can call into COFFSymbolReadSectionDefAux to obtain a pointer to the aux info for the section, allowing it to read the info about the COMDAT section.

@thanm thanm added the Proposal label Mar 22, 2022
@gopherbot gopherbot added this to the Proposal milestone Mar 22, 2022
@ianlancetaylor
Copy link
Contributor

Adding to proposal project for discussion, but my personal feeling is that we don't need a formal proposal to add simple accessors and constants to the debug/* packages.

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) Mar 22, 2022
@thanm
Copy link
Contributor Author

thanm commented Mar 22, 2022

I agree, it seems a bit heavyweight, but I am just following the instructions as laid out in https://go.googlesource.com/go/+/96567fb3ccdc8d8c052606112db7b39abd725b59/api/README

@gopherbot
Copy link

Change https://go.dev/cl/394534 mentions this issue: debug/pe: add APIs for reading section def aux info

@alexbrainman
Copy link
Member

@thanm

Your proposal LGTM.

Thank you.

Alex

@rsc
Copy link
Contributor

rsc commented Mar 30, 2022

This seems clear enough and unobjectionable enough that I think we can just move it to accept. Thanks!

@rsc rsc moved this from Incoming to Accepted in Proposals (old) Mar 30, 2022
@rsc
Copy link
Contributor

rsc commented Mar 30, 2022

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

@rsc rsc changed the title proposal: debug/pe: add APIs to support reading COMDAT info for sections debug/pe: add APIs to support reading COMDAT info for sections Mar 30, 2022
@rsc rsc modified the milestones: Proposal, Backlog Mar 30, 2022
@thanm
Copy link
Contributor Author

thanm commented Mar 31, 2022

https://go.dev/cl/394534 has been +2'd and is ready to submit; I am going to go ahead and close this issue out (this is now required in order for the API check test to pass during all.bash).

@thanm thanm closed this as completed Mar 31, 2022
gopherbot pushed a commit that referenced this issue Mar 31, 2022
Add hooks to support reading of section definition symbol aux data
(including COMDAT information) from the aux symbols associated with
section definition symbols. The COFF symbol array made available by
"pe.File" includes entries for aux symbols, but doesn't expose their
structure (since it varies depending on the type of aux symbol). This
patch adds a function for returning a specific class of aux symbol
("type 5") that immediately follows a COFF symbol corresponding to a
section definition.

Updates #35006.
Updates #51868.

Change-Id: I21fcc057150f7a3c64f01a5961aabca0fa43399e
Reviewed-on: https://go-review.googlesource.com/c/go/+/394534
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Trust: Alex Brainman <alex.brainman@gmail.com>
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
@gopherbot
Copy link

Change https://go.dev/cl/414515 mentions this issue: api: correct debug/pe issue number for Go 1.19 changes

gopherbot pushed a commit that referenced this issue Jun 28, 2022
It was #51868 not #51686.

For #53310.

Change-Id: I2cf28ca4de65e7030fdbd05e7f32fe42c8f3ca0a
Reviewed-on: https://go-review.googlesource.com/c/go/+/414515
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
jproberts pushed a commit to jproberts/go that referenced this issue Aug 10, 2022
It was golang#51868 not golang#51686.

For golang#53310.

Change-Id: I2cf28ca4de65e7030fdbd05e7f32fe42c8f3ca0a
Reviewed-on: https://go-review.googlesource.com/c/go/+/414515
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
@golang golang locked and limited conversation to collaborators Jun 27, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
No open projects
Development

No branches or pull requests

5 participants