The names. For example, “structlayout”, versus “typelayout”? If we decide that this is a good place for 0-width signal types, some of them (NoCopy) aren’t about type layout which means whatever-layout isn’t quite right.

How about typetag, e.g. typetag.Platform and typetag.NoCopy?

Working old code will continue to work properly.

Will layout changes be guarded behind the module's Go version? I would find that important for the unsafe case.

Will layout changes be guarded behind the module's Go version? I would find that important for the unsafe case.

This proposal is just part 1, adding the structlayout.Platform type and implementing its semantics. That is completely backwards-compatible, if you never see that type everything works as before.

Part 2, actually changing the layout of unadorned structs, is not part of this proposal. Of course, this proposal has less motivation if part 2 never happens.

Being part of the type system is a bit weird. Can i throw it in in a interface{}? What happens when i take a pointer to it and it's set to nil. Can i make it generic with type parameters?

@nemith All of those will work fine. This proposal just changes the layout of the fields within a struct. Field layout is already fully described by the reflect.Type value associated with the type. Nothing else cares.

@nemith Part of the reason for putting it into the type system is so that all the other Go tools understand it, from the point-of-view of type comparison, identity, etc.

If/when additional signal types get added, what are the semantics of including more than one in the same struct? Will it be required that all signal types have orthogonal semantics, will it be a compile time error if they are incompatible? What if they are only incompatible on one platform/OS pair, would there be a vet check to alert someone that doesn't explicitly try that combination to the potential issue?

@ChrisHines

I see no reason to require orthogonal semantics ("PlatformLayout" overlaps with "DeclaredLayout", which is not yet proposed but I can imagine it) but I do think that incompatible combinations should be diagnosed at compile time. And looking at a plausible interaction with alignment specification (the other/next layout tag I expect to someday see) I can construct plausible examples that would use both.

I think for this particular tag it would be reasonable to require that it precede any other field that has non-zero size.

My goal is to have as few of these tags as possible, motivated by real problems, so hopefully there will not be many combinations that apply, I think it is fine for the compiler to reject any combinations that are problematic. Even one of these tags should be a niche case; two should be niche-squared.

HOWEVER:

This might get messy for special C-hardware-specific types, for example, those used to talk about xmm and ymm registers, that currently have no Go equivalent. There are several approaches to that problem and I am not sure which is best; the existence/names of the very-wide data types is platform-dependent for C compilers, but Go could decide to just generally support 128 and 256-bit integers. Or, we could add per-field type tags for alignment that would precede 128-bit or 256-bit fields. So, something like:

type Ymms struct {
    _ typetag.PlatformLayout
   Inactive bool
   Reg  [32]uint256 // These are aligned to a platform-appropriate boundary
}

or

type YmmPart uint64
type Ymms struct {
    _ typetag.PlatformLayout
   Inactive bool
    _ typetag.Align256 // I hope the right alignment is 32-bytes / 256 bits.
    Reg  [32][4]YmmPart
}

(I added the boolean field just to make it clear that the hypothetical Align256 tag precedes a particular field.)

I prefer the choice where the programmer doesn't need to go read documentation to figure out what the C compiler is doing. On the other hand, after quickly checking what the internet to see what the YMM alignment rules are (and discovering a mess with annoying special cases), I can understand needing to be able to specify a platform order yet also be very picky about the alignment. Because of that, I think that specifying both platform layout and specific (increased) alignment for certain fields should be allowed. Specifying reduced alignment is probably a compile-time error, certainly taking the address of a field with reduced alignment is a compile-time error.

(Why is taking the address of a reduced-alignment field a likely error? The compiler would prefer to use the fast, assume-that-integers-are-aligned, instructions for dereferencing a *int64, and most people want the compiler to do that because it is faster. Taking the address of an unaligned field breaks that assumption.)

I am not sure if this is right for a vet check or not; vet would need to know a lot about different architecture and OS combinations and their C compilers. A different and slightly interesting question is what should happen if a struct tagged with "platform" layout contains a type that is inherently Go-oriented, like slice or map, or one for which Go has its own alignment assumptions. My inclination is to say that right now vet isn't checking any of this; platform interchange types are already a niche, and weird combinations of type tags (that don't exist yet) and/or Go types is a hypothetical niche of that niche.

@mvdan proposed improved naming, included at the top for anyone coming upon this later and not wanting to slog through comments:

After reflecting on the discussion below, I would modify this to:

package structs 

// PlatformLayout, as a field type, signals that the size, alignment,
// and order of fields conform to requirements of the GOOS-GOARCH
// target platform and may not match the Go compiler’s defaults.
type PlatformLayout struct {}

the rationale for the name change is that structs is one word, and parallels strings, bytes, and slices, and is generic enough to include other (future) tags specifying "nocopy" or alignment. Furthermore, such type-modifying tags only work within structures; the package name strongly hints at this.

Platform is a bit odd since Go is a platform too, and "the GOOS-GOARCH target platform" sounds like Go too.
What about structs.HostLayout, and don't mention GOOS-GOARCH in the docs?
(For the record, the main need for this is to write structs that match Windows and WASM, not Cgo.
Cgo can always do something magical and unexposed.)

Would this also affect the alignment of the struct on the stack?

@ydnar - it depends. On some architectures the stack is not very aligned, and so extra-aligned data is heap-allocated instead. Otherwise, yes, probably.

And I am fine with HostLayout, will edit the top proposal to reflect this.

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

Have all remaining concerns about this proposal been addressed?

The proposal is to add

package structs
type HostLayout struct{}

that can be added as a field named _ in a struct. This would have no significant effect in most tools, but a compiler could use it as a hint about laying out the struct. Of course there is an effect for type equality, since a struct with one of these fields is different from a struct without, but that’s exactly what we want if the compiler is using it to hint a different layout.

Would like to additionally require that this directive-typed field must appear before any field with greater-than-zero size, if that's okay? (This should not be requirement for some imagined other directives, e.g., explicit next-field alignment.)

What advantage do we get from imposing such a requirement?

Simplifies the implementation ever-so-slightly (avoids a pre-pass over structure fields), also makes its use more uniform, and I don't see much harm in the restriction (which could be relaxed later if I turn out to be wrong in "not much harm").

My take on it is that the advantages we get from the restriction aren't worth the cost of complicating the spec by adding the restriction.

Would the field be zero sized if it's the last _ field in a struct?

@iant I am okay with that also.

@ydnar _ fields can have width. A zero-width (field) type is either an array of zero elements, a struct of zero fields, or a struct/array built entirely of zero width types.

Change https://go.dev/cl/578355 mentions this issue: cmd/compile: layout changes for wasm32, structs.HostLayout

@ydnar _ fields can have width. A zero-width (field) type is either an array of zero elements, a struct of zero fields, or a struct/array built entirely of zero width types.

I’m referring specifically to 6f07ac2:

cmd/gc: pad structs which end in zero-sized fields

For a non-zero-sized struct with a final zero-sized field,
add a byte to the size (before rounding to alignment).  This
change ensures that taking the address of the zero-sized field
will not incorrectly leak the following object in memory.

reflect.funcLayout also needs this treatment.

Fixes https://github.com/golang/go/issues/9401

@ydnar Yes, that would kick in if you choose to put this field last in a struct. I don't see any reason to treat it differently. For all purposes other than it's special purpose, it's just a field.

Change https://go.dev/cl/581316 mentions this issue: cmd/compile: wasm32-specific structs.HostLayout changes

Would the field be zero sized if it's the last _ field in a struct?

We should probably document that "By convention, this field should be placed first in a struct." That's a good convention to have regardless.

@cherrymui pointed out that, while it's important that an addressable zero-sized field at the end of a struct must have non-zero size, that we could carve out zero-sized _ fields from this rule. Algorithmically, we would first strip zero-sized _ fields at the end of the struct, and then append a byte if the remaining final field is zero-sized. This is an implementation detail, and is something we could decide to implement in the future. It would remove a minor foot-gun.

@cherrymui pointed out that, while it's important that an addressable zero-sized field at the end of a struct must have non-zero size, that we could carve out zero-sized _ fields from this rule.

This seems to imply a change to reflection: https://go.dev/play/p/TfYcdEwAACm

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

The proposal is to add

package structs
type HostLayout struct{}

that can be added as a field named _ in a struct. This would have no significant effect in most tools, but a compiler could use it as a hint about laying out the struct. Of course there is an effect for type equality, since a struct with one of these fields is different from a struct without, but that’s exactly what we want if the compiler is using it to hint a different layout.