For language change proposals, please fill out the template at https://go.googlesource.com/proposal/+/refs/heads/master/go2-language-changes.md .

When you are done, please reply to the issue with @gopherbot please remove label WaitingForInfo.

Thanks!

I don't quite see the point of this proposal because it seems to be you can already do what you want in Go, with a few supporting methods. Here is your "entire point" example as it works today:

https://play.golang.org/p/GaaBp-vMUU1

package main

import (
	"fmt"
)

type Point struct{ X, Y float64 }

func (p Point) String() string {
	return fmt.Sprintf("Point{%f,%f}", p.X, p.Y)
}

func PointAt(X, Y float64) Point {
	return Point{X, Y}
}

type Box struct {
	Point
	W, H float64
}

func BoxAt(X, Y, W, H float64) *Box {
	return &Box{PointAt(X, Y), W, H}
}

type Circle struct {
	Point
	R float64
}

func CircleAt(X, Y, R float64) *Circle {
	return &Circle{PointAt(X, Y), R}
}

func (p *Point) Move(delta Point) {
	p.X += delta.X
	p.Y += delta.Y
}

func MoveCirclesAndBoxes() {
	boxes := []*Box{BoxAt(1.0, 1.0, 1.0, 1.0), BoxAt(0.0, 1.0, 1.0, 1.0)}
	circles := []*Circle{CircleAt(1.0, 1.0, 1.0), CircleAt(0.0, 1.0, 1.0)}
	fmt.Printf("boxes: %v\ncircles: %v\n", boxes, circles)

	// Essentially, this is the entire point of this proposal, for me at least:
	delta := Point{1.0, 2.0}
	for _, box := range boxes {
		box.Move(delta)
	}
        // Can also iterate like this: 
	for i := 0; i < len(circles); i++ {
		circles[i].Move(delta)
	}

	fmt.Printf("boxes: %v\ncircles: %v\n", boxes, circles)
}

func main() {
	MoveCirclesAndBoxes()
}

This uses no interfaces and the calls to Point.Move are statically dispatched.

@gopherbot please remove label WaitingForInfo

@beoran, yeah. I did spend a significant chunk of that dealing with what alternatives exist today, and after expressing the analogous unsafe expression that is the target of this functionality, an example as above was literally my first example. I'm not claiming that it's not something that can be done currently. Quite the opposite. This is most effectively done with raw pointers, and several less drastic solutions also exist.

I did give a few reasons why such a solution would better than above, even if it is all a hypothetical to your concrete example, but this isn't some sob story about how Go doesn't have the tools I need. This isn't some demand that I'm gonna take my custom to Python, but rather "Hey. This seems like it'd slot nicely in with Go's memory model. What'd ya think?"

Example from proposal:
converted := struct{One,Two,Three int}{10,20,30}.{[3]int}

One thought I had while quickly glancing over the proposal is if those conversions are to be made without copying to new memory then it will introduce a memory ordering on the struct fields in go. This is not currently guaranteed as far as I remember by the go spec as a thought was the go compiler could optimise this for the user under the hood in the future (unless there is some annotation to tell the compiler not to).

Ok, now that you filled in the template I have a better idea of what you are trying to do.

The unsafe code you use now,
subber := (*SubType)(unsafe.add(unsafe.Pointer(&super), unsafe.OffsetOf(&super.X))) is similar to C code with C's offsetof(), where we can create a pointer to somewhere in the middle of an existing struct and treat that as if it is an embedded struct. In C we do this because we do not have struct embedding like we have in Go, and also for performance. Of course in Go this is unsafe and in C it may break strict aliasing, so in both cases it is not really safe to use.

But in Go we do have struct embedding, so I would say the first reason for this feature is debatable. The main problem you seem to face is performance. Rather than proposing a rather complex new language feature, I think we should probably try to improve the performance of the compiled code for embedded structs, so it works better in your use case.

All-in-all, I appreciate the attention given thus far, and if I must walk away being told by the Go team, "We care a lot more about X API working well, and we think your suggestion is a distraction from it. Try using X, it'll get you close but Safely.", I could accept that. I am hoping to push the issue a bit, if there's willingness to consider it.

@martisch, honestly I was thinking that the possibility that memory might have multiple owners was a bigger deal. I didn't realize that field-ordering wasn't a stable thing. I generally see the same data in the same order, and while explicitly not covered by a compatibility guarantee, I assumed this was fairly stable. If you meant from an API perspective, matching the precise order and type between structs absolutely would be the biggest barrier to a user.

Either way, I was writing this proposal with the assumption that field-order was stable, even if it didn't carry a guarantee. Field Number doesn't correspond to offset, embedded interfaces are uinptr width, and embedded structs also have their own subsection of the total offset. If this is a misunderstanding, I would love a pointer to reference/source that would help me get a better head-model of allocations. If you simply meant that you don't think the feature compelling enough to officially support a feature which depends on that, I would find it frustrating but wholly acceptable.

@beoran, I appreciate your interest in skipping past any XY problems posed. I find embedding an inappropriate tool for my use on account of behavior composition versus structural composition. Again, I will readily admit that I'm comparing Go to Rust, and I am in want of a specific functionality that may very well be of non-interest for the language design of Go. As above, it'd be an obnoxious finding, but I could accept that as outcome.

Embedding is currently the only effective way to compose behaviors, because it is the only way to compose structure. If that proposal for struct fields holding a function reference being able to satisfy interfaces would help me somewhat. The core problem domain is performance pathways over types that aren't mutually convertible. Possibly saying more about our ingestion phase and data design than any deficiency in Go, I'm interested in a means of expressing that each of those types satisfies a contract based on fields, while being non-exclusive with other contracts which include those same fields. Embedding is a way to accomplish this, but it fails on the non-exclusion point.

I understand that this is what interfaces are literally for, but they have an implementation I find unsuitable as a whole in my domain. Today, there are absolutely ways to accomplish the pattern I want, a contract constraining the reduction of instances of various types to a common interchange form, and where behaviors can be defined which will apply to instance in that form, where mutations to instances in that form carry over to the instance in original form, and where overlapping contracts are non-exclusive. The further goal of limiting allocations, ideally keeping all allocations localized and/or on the stack, is satisfied by each of the below strategies.

Given that my problem is one of reducing varying types to the subset on which they overlap, I could totally see this being handled by the Generics system at some point in the future. The subtle issue of defining field-based constraints aside, and the API by which a constraint could be defined based on types from other packages, I still have the issue of type-mixing, and then of migrating changes back to the original, after a series of mutations. If X as field of generic pointer to T, (*T)(t).X was permissible at some point, and if there were not overhead in selecting that function (mitigated to some extent by ([]T)[i].X), I could become content with that. Even with any possible issues, that at least guarantees compile-time checking.

The solution that I'm playing with as a viable, production-ready approach to refactor to is to use a slice of fixed arrays of uint8, serializing using a fixed-field schema, and passing those chunks around as datasleds with a husky-dog of a function having en-closure incrementing index and a reference to the sled, iteratively returning the fields needed for that pipeline stage. You may have noticed my lobster claws, and I do know I'm intending a pattern that's not really idiomatically Go.

Having done a fair bit of hacking in Go, and I observed that my solution above resembled the Go memory model, and that I could get similar behavior as above by using a raw pointer to convert the array of uint8 to an array of struct. In my test model, it worked, but I observe the comments from @martisch to indicate that as a happy accident.

The part that embedded structs can't accomplish, by design, is mutations outside of their fields. So if you want, say that Point example, to operate on Point with height, you have to define that on the next highest unit above a Point, rather than on a slightly different definition of a Point that still satisfies a hypothetical contract around, yeah, field ordering. Instead, to merge pipelines having different element types, even if they all satisfy a common hypothetical contract, I'm back at reallocation as a new type, or else refactoring to wrap Point in a dimensioned other type, and then embed that. I get that in this contrived example, you could absolutely handle arbitrarily dimensioned drawables as a struct with Point and slice of Point with enum type, but the point is rather that Go doesn't yet have a compile-checked performance path that offers a contract that supports that use case -- it's obviously not my place to even claim it should, but the observation stands.

My second alternative was to create a type where each of the fields internally are pointers. Because I don't have any guarantee that len(datasled)%Sizeof(item) == 0, it could not be viable unless I create that guarantee with a runtime panic at the start. With those allowances, that pattern kind of almost works, until you want to merge pipelines, and then there's the type-mixing problem. A possible solution is for that type to be packaged with a closure which mutates its internal pointers on its behalf, so that the type itself can be monomorphic. This is a solution that actually works for me, mostly, as a (mostly) ergonomic way to diversely monomorphize a collection without reallocation.

Finally, not suggested above, because I think it's more to do with my use case than with the original problem, is a struct wrapper having an empty interface, where the interface is treated as a multipurpose pointer, and where each method is a return or mutation on the type-switched field on interface.(type). I haven't tested that, and once generic structs are up, I'm expecting that would be better on all counts, excepting possibly the case of type-mixing.

First of all, I am not on the Go team, I'm just a regular punter here who likes to comment and evaluate Go issues.

Well, I thought about what you are trying to do, and reading the generics proposal I felt it should be possible to do something like this with the go generics. I tried it out but it didn't work, so I filed this: #48522.

I would like to kindly advise you to "forget" Rust and other C-like languages when programming Go. Your previous knowledge is actually likely to lead you astray, since Go is semantically much more of a Wirth-style language like Oberon or Modula 3. I would try to figure out first how to write your code in a Go-like and readable way, and then think of how to improve the performance.

Edit: a classic trick would be to turn your structs inside out and use separate fixed length arrays of fields in stead.

I actually just saw that, and I went "Huh, this person must have read my propsal -- Oh! Hey, it's that guy with the name that sounds like the bear-dude from LoTR that got totally left out of the movies!" I guess I disregarded the possiblity that there was interest in supporting that API. Not having dug into the stenciling machinery, unless different versions of a generic function are created for each type that is used with an example, the use of fields on a generic is fairly likely to remain off the table.

A function call as assembly produced by the Go Compiler, having a struct parameter, any use of fields on that struct are rendered as pointers+offset, from the struct pointer. I could see that being handled in assembly, most obviously, by rendering a different function for each distinct type used as call argument, but it could also be done with, in the assembly, a pointer to an array of the offsets to be used, one for each type which would be passed as call argument, and the array being of the length of distinct fields referenced in the function body. Now that I think about it, it seems like that could be OK in the end, but I'd have to do a mockup, ensuring that it resembles what I just described, and compare it to some static calls.

And you speak truth on the "Forget that stuff" bit. One of my favorite patterns in Go as of late has been the slice of array with driving closure iterator. It isn't idiomatic Go, but it's generally fairly close to the for i;i<l;i++ {} pyramid equivalents, while being more fun to write, and emphatically easier to refactor or repurpose. Plus, most of the utility of what I'm suggesting is had by swapping iterators, anyway, and it's covered by static analysis up until the point where I do the raw-pointer cast.

Nonetheless, a facility for structured type division and supervision, with static analysis from core tooling, would be a benefit to the language, cut use of empty interface, and provide a decent alternative to unsafe. And, maybe you're right, and enough of the possible applications are already covered by struct embedding and now generics that it's just not advantageous enough to introduce memory model constraints over.

Well, I think that allowing access to fields in generics is probably going to be a good way to help solve your problem, and actually I am surprised that it is not supported yet, since it doesn't really make sense for it not to be supported. I don't care too much about the implementation details, rather I think we should push to keep it "on the table" , as it were, and get it implemented somehow. Then if that implementation were to have some performance downsides, we can always suggest improvements. "First make it right. Then make it fast." (Jeffrey Palermo).

And yes, Go also makes closures easier than most C-like languages do, which allows them do be used in various creative ways. So if it's easier to read, write and refactor, then by al means, go for it.

And I fully agree that Go needs a new language feature to move away from using the empty interface and unsafe as much as possible. But, generics is that language feature, if fully implemented as is now documented in the proposal. Also, the feature you are proposing seems to be of too limited use, focused only on your own use case. That is a problem that allowing field access in generics doesn't have. So If you agree, I'd like your support for #48522, especially by providing examples from your current projects in how allowing access to struct fields in generic types would help you.

@sammy-hughes
The toy examples in your proposal are not hard to read, but in the real world it is not uncommon for structs to have way more than 3 fields. Could you discuss what your examples would look like if I wanted to destructure this struct like so: fields LowerC ... LowerV into a subtype Lower, fields Percent and CompilingRuntime into a subtype SpecialChars? In particular, would I be supposed to create dummy struct types for the preceeding/succeeding fields (relative to each desired subtype) with no other purpose than to avoid writing out ~5 dozen fields, and whenever I change the main struct, make sure I also update the dummies?

With the current facilities that Go provides, something like

(*Lower) (unsafe.Add(
    unsafe.Pointer(super),
    unsafe.Offsetof(super.LowerC),
))

can be hidden inside a method with a descriptive name, e.g. func (super *CmdFlags) AsLower() *Lower, and I think use cases that would utilize more than a handful of the n*(n+1)/2 possible destructurings of a struct with n top-level fields are not common, so you won't need many of these methods. All is pretty readable and maintanable, imo.

So, do you think there will be an overall gain in readability and maintainability for real world programs, than what can be achieved presently?

As to the final question, @3bodar, I do. My last professional position was maintaining the data-delivery portion of a web application, and my current position is with a data warehousing team, part of an Ag/Chem org. Not using Go, I worked with a team responsible for ingesting assessor data for a company that offers a suite of financial projection products. I will agree that 2 of 3 of those positions wouldn't have seen the workload impacted.

In the first case, my last position working with Go, I would completely agree with @beoran's commendation of struct embedding, as the need was simply to represent data for purposes of CRUD operations between a remote client and the persisted data layer, on RDS/MySQL. We did some reporting, but a lot of the actual logic in the transforms involved were done in SQL, at the data layer. This feature would provide limited benefit, and any polymorphism needed would be completely served by struct embedding. We dealt with concretes like "Location" embedded in "Structure". Relationships between the concepts represented generally had clearly-defined relationships in persistence representation and view, needing no redefinition, none being possible.

The latter mentioned position, using a slew of Microsoft GUI tools, with C# as the main language for actual code, we had an internal flat file that would be compatible with the "datasled" technique I mentioned in an earlier comment, where we're essentially just marshalling into an array of byte. Were I to rewrite in Go, I would not reach for anything more complicated than a custom serializer framework, writing struct types for each distinct region, with serialization logic being well established, and then write the transform logic from original source (database dumps, fixed-length formats like our own, horrifying messes of GB-sized CSV files with no text delimiter), again, with likely no possible use-case for a feature such as I've proposed above.

My current position, we've had a stable data model in persistence, of the "General enough that it sucks only a little, for everything," but we routinely get new output definitions for existing transforms, and the number of transform stages a set undergoes will vary depending on the nature of the subject/project. Some transforms are universally applied, and some only apply to a specific experiment, but having a fixed persistence/input surface, but with data serializing into a range of distinct forms, varying by the value domain of the measurement, how self-referential the data is, the weighting of associated observations, the experiment constraints that apply, etc. In the current form, we have over 40 fields on the primary entity, with several lists of associated entities. Given that we are processing and delivering GB's of data per request, and that the specific form of the serialized data is based on the request spec, being one of many, our options are

1. Write each stage for a reduced, interpreted "interchange" form, and then loop through the experimental finding, then any applicable related findings, loading them into a slice of that reduced form, and execute the battery of contiguous stages which use that form, nil the slice after looping, then move on to the next transform. If two transforms are separated by one or more transforms on a different interchange form, having to choose whether you can afford to retain the transformation, or if you have to manually free it to ensure sufficient memory for the next transform.
2. Just eat the cache locality issues, soak the constant cache-line fills, and write the thing to properly select logic for a primary finding type or the appropriate related finding type, on every stage, for every distinct operation in that stage.
3. Optimize using struct embedding. Work up a new common model to use every time a new spec requires a change, wrapping some types, expanding others, and just changing methods on still more. This presents risk to compatibility across existing pipelines, and requires just accepting that, while you can optimize for throughput, you're still stuck in a constant maintenance phase.
4. Beg, borrow, or bribe management into allowing unsafe in production code, having sufficient tests to guarantee the output for the conditions you forsee, but knowing that you're giving up compiler assistance in the process, and as unlikely as a memory-model change is, risking an update to Go invalidating your entire waterworks.

With what I'm proposing there are still a number of headaches and problematic logistical elements, but a lot of those problems become much simpler to handle.

1. The proposed functionality comes from the interest of alternatively expressing the existing data in a form optimized for records-per-cache fill, without paying for it by allocating more memory than the necessary pointers, and without two copies per field included in the interchange re-expression.
2. The proposed functionality allows interchange forms to be mutually compatible, provided they each satisfy a contract around field-ordering on the common model. And changes to that model carries the risk of invalidating contracted interchange types, but because the proposal stipulates type-checking, with the same rigidity of assignability as in any other assignment in Go, such invalidations would fail, even without unit tests for that specific eventuality, because pre-compile static analysis can find the assignment as invalid.
3. The proposed functionality would allow the common model to continue unchanged, and adjustments for each new spec that comes through necessitates at most, definition of new re-expressing interchange types, and will require no change as often as solution strategy 3 above would have permitted such.
4. The proposed functionality, supported by clear expectations on the part of any user of such syntax, and any restrictions on expression-combination as are needed by the Go toolchain to guarantee safety, would allow changes which otherwise might have caused outages to instead fail the build at compile. Additionally, having the support of the Go toolchain, such projects as used the, hypothetically accepted and implemented, feature proposed, over unsafe, would be easier to work with and debug, not having a blindspot in static-analysis on every call to unsafe.

So yes, the final result, assuming it becomes accepted as a supported feature of the language, would be that vectorized calculations on datasets too large to fit in cache for a single-processor become much easier to accomplish in Go, and some of the routine optimizations by such projects, currently using unsafe, gain the support of Go's excellent compile toolchain.

@sammy-hughes
Thanks for the detailed reply, but except for the first sentence, I couldn't understand much of what you are saying. Not even whether you are trying to answer my other questions (the essential ones for me to better understand how the proposal would affect the language). But nevermind.

I would just add one thing - the feature as described doesn't appear to me as fitting smoothly into Go's design philosophy of simplicity and clarity of design, and, I think, would be of no more than niche use, so I'd predict it's going to be rejected. It's very possible I don't know what I am talking about, as I consider myself a novice gopher (having written < 5k lines of Go code in all) and I've yet to fully read and understand the spec.

If you prove me wrong, more power to you. Good luck.

@3bodar, yeah, no worries.

I don't aim to convince you that this is not complex as far as the backing machinations, but I do object to the suggestion that it would be arcane to use. The syntax is rigid, but it follows, essentially, this core rule: account for all fields/elements, in exactly the correct order. Other rules, yes, but mainly that.

Such a syntax would have to be that rigid if it is intended for inclusion outside of the package "unsafe". I believe it to be very simple for the Gopher using it, even if it takes fiddling to arrive at a proper invokation. That fiddling would be adjusting the call to confirm to a static, readable, exists-in-code requirements list, e.g. the struct or array (not slice) being destructured.

Meanwhile the syntax itself borrows from existing syntax elements. First, it was directly inspired by the type-union clause of a constraint declaration: x.{w|y|z}, itself quite similar to the syntax for asserting the underlying type of an interface: x.(y). As a type expression list, it matches such syntax roughly.

I am proposing a feature and syntax which is a combination of special cases for the following Go features:

1. Slices. I'm proposing a feature and syntax to slice on instances of structured types, using an intentionally rigid API. I expect my proposal to assist mostly research and computational workloads, where it offers optimizations over what is already safely possible, and replaces the need for unsafe in most of the cases I'm thinking of for what is unsafely possible. The API does need to be rigid enough to allow compile-time guarantees to be possible, and the core functionality likely is not served by anything less rigid.
2. Embedding: In the above comments, I pointed out why embedding is cumbersome in my specific workload, and I would expect that extends to anyone dealing with the following issues: A, constantly evolving output specification, and B, extremely demanding performance expectations. Outside of that, I expect that existing machinery completely satisfies the list of applications I can imagine, this proposed feature possibly not providing significant benefit outside of that.

As far as who will benefit from it, the following cases are examples of use-cases completely supported by the existing Go ecosystem, without resorting to unsafe, or needing any new syntax.

• Front-end for a data-persistance layer (database, redis, filestore, whatever): Design for Code-gen consumiung the database DDL.
• Data processing in a concrete domain (like property value data from county assessors): Special cases exist, but "structure" is inherent to the domain.
• Developing a rendering engine (using a structured, planned approach, with good management): Struct embedding is likely all you'll likely need.

Contrarily, the following are examples of application-targets that will benefit from such a syntax, already often resorting to unsafe, for raw pointers to get the performance optimizations described in this proposal.

• Developing a rendering engine, except taking an agile approach
• Experimental modeling for a research project or natural resource discovery
• Embedded application development
• Any project with management that throws new requirements at you every time they watch CSI
• Tool development having rigid goals, but with an appetite for any possible incremental performance gain

That said, and acknowledging that this could be a bit of a hassle to implement (mostly in the improvements to precompile analysis tools like go-vet), and that the demands of the syntax proposed could be difficult to satisfy in certain cases, it would be a feature that is quite accessible. The proposal contains the possibility of consolidating dozens of lines of runtime-safety around raw pointers into a single, compiler-checked assignment.

@3bodar, I failed to answer a specific question not once but twice.

TLDR:

1. The feature as proposed requires all fields to be explained, so yeah, either 5 dozen fields or a couple dump structs. It's intended to allow safety at speed, not for convenience.
2. Your unsafe example gives a pointer which is potentially nilled by the GC, if either SuperType instance or SubType instance goes out of scope. The proposal as written is for that, but safe.
3. This proposal is a win if the tricks I can already use unsafe to accomplish can happen with tooling support at linter, validation, compiler, and GC.

I am going to cover as discussion the question you cared about, include some snippets based on the example you suggested, and while offering profuse apology along the way, briefly catch why I thought I was answering your question, while apparently quite spectacularly missing the point.

Paraphrasing for clarity (I'm one to talk):

Could you discuss what your examples would look like if I wanted to destructure this struct ? Does the creation of "spacer" struct types, exclusively to provide structural offsets constitute an anti-pattern?

Yeah. I didn't answer that. I answered twice in good faith, but I didn't address a specific question, and for that I am sorry. After rereading this thread, I went "Aw, Py**on!" (Sorry for the colorful language. Bit of a sailor).

The chief end of this proposal should be considered, I, performance improvement, II, static-analysis and safety improvement, and III, very distantly an ergonomics improvement. As the goal is, crassly put, to push "unsafe hacks" the realm of safe and dependable code, supported by Go's tooling, by provisions for non-ambiguous contracts, I and II or II and I is the same.

I. The proposal is intended to gain a facility for a developer to write a contract around the structure of a compound type, either to divide or to supervise, whereby the garbage collector may be aware of the effective lifetime of component fields, the compiler can be aware of incorrect type conversions, and linters/syntax formatting can be aware of the progressively asserted structure of values subject to the feature being proposed.

II. Considered as a safety improvement, the syntax proposed intends that an utterly unambiguous contract may be established, that is executable by the compiler, and subsequently by the runtime GC. Further, it would help simplify static analysis tooling, versus using raw pointers, to trace values across their various identifiers, recognizing the value across its various identifiers being used in context of concurrency in a manner resulting in race conditions, and various quality-of-life matters such as linter context highlighting and in-editor tracing.

III. The syntax, as proposed whatever other form, must be suitable to provide guarantees that are better not made than if offered ambiguously. Still,

• if an external library, having a stable API, has struct types which are cumbersome to use, the feature as proposed would permit almost-free slicing on the internal structure, with some preparation, but with full tooling support.
• As proposed, the feature allows writing types to meet external surfaces, without needing a dozen different custom serializers for different services.
• It could allow skipping the odd extra line for operations wherein some combination or separation occurs.
• It could possibly be used to enable patterns like collecting a struct out and error out into a field on the out struct (mildly senseless, but possibly enabled).

There are some examples where I would rather like to have this syntax, for reasons that are purely ergonomic. For most such cases, interfaces and generics are suitable and effective.

The struct referenced, the CmdFlags struct from the Go compiler codebase, was suggested as having 2 ranges of interest, LowerC to LowerV, and the two adjacent fields Percent and CompilingRuntime. With defs to follow, example would be one of:

func MakeCmdFlagsA(
  in0 *cmdFlagsSpacerPrelogue, 
  in1 *Lower, 
  in2 *SpecialChars, 
  in3 *cmdFlagsSpacerEpilogue,
) CmdFlags {
    var out CmdFlags.{cmdFlagsSpacerPrelogue, Lower, SpecialChars, cmdFlagsSpacerEpilogue} = *in0, *in1, *in2, *in3
    return out
}

func MakeCmdFlagsB(
  in0 *cmdFlagsSpacerPrelogue, 
  in1 *Lower, 
  in2 *SpecialChars, 
  in3 *cmdFlagsSpacerEpilogue,
) CmdFlags {
    //This is hypothetical, and does not represent a part of my proposal, specific syntax here being mentioned nowhere.
    //I simply observed the question "How might that look?", and suggest a possiblity here.
    out.{CmdFlags.{cmdFlagsSpacerPrelogue, Lower, SpecialChars, cmdFlagsSpacerEpilogue}} = *in0, *in1, *in2, *in3
    return out
}

func MakeCmdFlagsC(
  in0 *cmdFlagsSpacerPrelogue, 
  in1 *Lower, 
  in2 *SpecialChars, 
  in3 *cmdFlagsSpacerEpilogue,
) CmdFlags {
    //This is hypothetical, and does not represent a part of my proposal, specific syntax here being mentioned nowhere.
    //I simply observed the question "How might that look?", and suggest a possiblity here.
    out = CmdFlags.{cmdFlagsSpacerPrelogue, Lower, SpecialChars, cmdFlagsSpacerEpilogue}(*in0, *in1, *in2, *in3)
    return out
}

func ProcessCmdFlags(in *CmdFlags) {
    _, lower, specialChars, _ := (*in).{cmdFlagsSpacerPrelogue, Lower, SpecialChars, cmdFlagsSpacerEpilogue}
    //use things as destructured. Did not need the prelogue or epilogue, so dumped into placeholders.
}

For reference, that's based on the following struct definition, from the Go compiler flags:

type cmdFlagsSpacerPrelogue struct {
    _, _ CountFlag //alternatively [2]CountFlag
    _ string
    _, _ CountFlag //alternatively [2]CountFlag
    _ func(string)
    _, _, _, _ CountFlag //alternatively [4]CountFlag
    //I didn't chase that long enough to know if 1. A different form has V, 2. it's a derived property, or 3. A form derived from this has V
    _ CountFlag
}
type cmdFlagsSpacerEpilogue struct {
    _, _, _, _, _  string
    _, _, _, _ bool
    _, _, _ *bool
    _, _ func(string)
    _, _, _, _ string
    _ CountFlag
    _ bool
    _ string
    _ int64
    _ string
    _, _, _, bool
    _ *bool
    _ bool
    _ string
    _ bool
    _, _, _ string
    _ bool
    _ struct { // I am unclear if listing these fields flattened to the outside struct would have the same meaning
        _ struct {
            _ map[string][]string 
            _ map[string]string
        }
        _ []string
        _, _ map[string]string
        _, _ bool 
    }
}
type Lower struct {
    LowC int
    LowD func(string)
    LowE, LowerH, LowerJ, LowerL, LowerM CountFlag
    LowO string
    LowP *string
    LowR CountFlag
    LowT bool
    LowW CountFlag
    LowV *bool
}
type SpecialChars struct {
    Prozent int
    CompilingRuntime bool
}
type CmdFlags struct {
	// Single letters
	B CountFlag    "help:\"disable bounds checking\""
	C CountFlag    "help:\"disable printing of columns in error messages\""
	D string       "help:\"set relative `path` for local imports\""
	E CountFlag    "help:\"debug symbol export\""
	G CountFlag    "help:\"accept generic code\""
	I func(string) "help:\"add `directory` to import search path\""
	K CountFlag    "help:\"debug missing line numbers\""
	L CountFlag    "help:\"show full file names in error messages\""
	N CountFlag    "help:\"disable optimizations\""
	S CountFlag    "help:\"print assembly listing\""
	// V is added by objabi.AddVersionFlag
	W CountFlag "help:\"debug parse tree after type checking\""

	LowerC int          "help:\"concurrency during compilation (1 means no concurrency)\""
	LowerD func(string) "help:\"enable debugging settings; try -d help\""
	LowerE CountFlag    "help:\"no limit on number of errors reported\""
	LowerH CountFlag    "help:\"halt on error\""
	LowerJ CountFlag    "help:\"debug runtime-initialized variables\""
	LowerL CountFlag    "help:\"disable inlining\""
	LowerM CountFlag    "help:\"print optimization decisions\""
	LowerO string       "help:\"write output to `file`\""
	LowerP *string      "help:\"set expected package import `path`\"" // &Ctxt.Pkgpath, set below
	LowerR CountFlag    "help:\"debug generated wrappers\""
	LowerT bool         "help:\"enable tracing for debugging the compiler\""
	LowerW CountFlag    "help:\"debug type checking\""
	LowerV *bool        "help:\"increase debug verbosity\""

	// Special characters
	Percent          int  "flag:\"%\" help:\"debug non-static initializers\""
	CompilingRuntime bool "flag:\"+\" help:\"compiling runtime\""

	// Longer names
	AsmHdr             string       "help:\"write assembly header to `file`\""
	Bench              string       "help:\"append benchmark times to `file`\""
	BlockProfile       string       "help:\"write block profile to `file`\""
	BuildID            string       "help:\"record `id` as the build id in the export metadata\""
	CPUProfile         string       "help:\"write cpu profile to `file`\""
	Complete           bool         "help:\"compiling complete package (no C or assembly)\""
	ClobberDead        bool         "help:\"clobber dead stack slots (for debugging)\""
	ClobberDeadReg     bool         "help:\"clobber dead registers (for debugging)\""
	Dwarf              bool         "help:\"generate DWARF symbols\""
	DwarfBASEntries    *bool        "help:\"use base address selection entries in DWARF\""                        // &Ctxt.UseBASEntries, set below
	DwarfLocationLists *bool        "help:\"add location lists to DWARF in optimized mode\""                      // &Ctxt.Flag_locationlists, set below
	Dynlink            *bool        "help:\"support references to Go symbols defined in other shared libraries\"" // &Ctxt.Flag_dynlink, set below
	EmbedCfg           func(string) "help:\"read go:embed configuration from `file`\""
	GenDwarfInl        int          "help:\"generate DWARF inline info records\"" // 0=disabled, 1=funcs, 2=funcs+formals/locals
	GoVersion          string       "help:\"required version of the runtime\""
	ImportCfg          func(string) "help:\"read import configuration from `file`\""
	ImportMap          func(string) "help:\"add `definition` of the form source=actual to import map\""
	InstallSuffix      string       "help:\"set pkg directory `suffix`\""
	JSON               string       "help:\"version,file for JSON compiler/optimizer detail output\""
	Lang               string       "help:\"Go language version source code expects\""
	LinkObj            string       "help:\"write linker-specific object to `file`\""
	LinkShared         *bool        "help:\"generate code that will be linked against Go shared libraries\"" // &Ctxt.Flag_linkshared, set below
	Live               CountFlag    "help:\"debug liveness analysis\""
	MSan               bool         "help:\"build code compatible with C/C++ memory sanitizer\""
	MemProfile         string       "help:\"write memory profile to `file`\""
	MemProfileRate     int64        "help:\"set runtime.MemProfileRate to `rate`\""
	MutexProfile       string       "help:\"write mutex profile to `file`\""
	NoLocalImports     bool         "help:\"reject local (relative) imports\""
	Pack               bool         "help:\"write to file.a instead of file.o\""
	Race               bool         "help:\"enable race detector\""
	Shared             *bool        "help:\"generate code that can be linked into a shared library\"" // &Ctxt.Flag_shared, set below
	SmallFrames        bool         "help:\"reduce the size limit for stack allocated objects\""      // small stacks, to diagnose GC latency; see golang.org/issue/27732
	Spectre            string       "help:\"enable spectre mitigations in `list` (all, index, ret)\""
	Std                bool         "help:\"compiling standard library\""
	SymABIs            string       "help:\"read symbol ABIs from `file`\""
	TraceProfile       string       "help:\"write an execution trace to `file`\""
	TrimPath           string       "help:\"remove `prefix` from recorded source file paths\""
	WB                 bool         "help:\"enable write barrier\"" // TODO: remove

	// Configuration derived from flags; not a flag itself.
	Cfg struct {
		Embed struct { // set by -embedcfg
			Patterns map[string][]string
			Files    map[string]string
		}
		ImportDirs   []string          // appended to by -I
		ImportMap    map[string]string // set by -importmap OR -importcfg
		PackageFile  map[string]string // set by -importcfg; nil means not in use
		SpectreIndex bool              // set by -spectre=index or -spectre=all
		// Whether we are adding any sort of code instrumentation, such as
		// when the race detector is enabled.
		Instrumenting bool
	}
}

If you end up reading this, @3bodar, I do quite hope you find this a better answer. As a quick explanation of why I think I was answering you, accepting that you clearly didn't agreee, consider my perspective on my two replies to youi:

I initially heard from you, "Hey your thingy looks complicated. Why? Would it even be worth it in the real world?" My response was intended to nicely say, "Yeah. That kind of means I don't care about you," but in a way that made it clear that I also don't care about past me, for much of my career. I wanted to show how it actually doesn't make sense for two real-world examples, and what made them different from now.

After you replied, I replied in context of my memory of your first reply and then the text of your second reply. The core of it hit me as "Meh. Doesn't look useful," which was a conclusion I support you arriving at, and challenged my use jargon (sorry. if we were friends, it would have been a "Lets stop texting and phone call" thing). In my reply, I set out to plead "no contest" on those points, as I agree on the spectrum of applicability, but not on the scheme of the syntax and on strength of appeal. I absolutely think it "looks like Go", and I fully intend that this proposal feel like Go, too.

Meanwhile, thanks for expressing interest. I hope it was all at least food for thought! One of the many things we agree on is the likely outcome of this proposal, but if I've done nothing other than spun some mental gears, I'd be quite content.

@sammy-hughes
Now that's to the point, and I appreciate you taking the time to write it. And it's what I expected - with the "Pr[o]logue", "Epilogue", etc. And it's the segue I needed in order to add a couple more thoughts.

The essential problem I see is the syntax. The notions of simplicity and clarity being a contentious matter, I feel pretty adamant in my assessment that the code in those "MakeCmdFlags..." is horrendous. Just imagine coming accross that snippet without having a clue and trying parse the purpose of it.

And the fundamental reason for that complexity is, I believe, that you are trying to achieve something out of thin air without having a proper abstraction for it. Similar problems have already been solved in Go - you're basically in need of something that to a struct, is what a slice is to an array. (Or, in a similar vein, what the underlying []byte is to a string, ignoring issues of immutability and conversion semantics.) I wouldn't call that something a "struct slice" as I think it would be natural to allow for holes in it (contributing to sizeof, otherwise inaccessible), so let's call it a "struct view". So, just for illustration, if you could make up some syntax for creating a "struct view" type based on an existing struct type, e.g.

type MyStruct struct {
    A int
    ...
    K float64
    L float64
    M float64
    ...
    Z string
}

type MyStructView struct(MyStruct) {
    // only MyStruct fields allowed to the right, optional new names to the left
    F
    C    Circle(K, L, M) // aggregation 
    NewZ Z
}

then you could write simple and elegant code that feels Go-like:

// downcasting:
myStructView = MyStructView(myStruct)
// upcasting:
myStruct = MyStruct(myStructView)
// field access:
var c *Circle = &myStructView.C
z := myStruct.NewZ

The compiler would have all the info it needs - parent struct type, field types and offsets - to ensure type safety, similar to the array/slice case, and would know that the conversions are to/from a "struct view" type, and act appropriately, i.e. no need for new syntax for the destructuring.

My point here is just to provide an example of what I would consider a feature that is not obviously unworthy of Go's "ideals" - I have no idea whether something like the above satisfies all your requirements, or whether it would hold any water once you've analyzed all the intricacies related to it's precise specification and interaction with the rest of the language.

This whole time, I wasn't implying there's no value in your proposal, only that it's of little use even if it's just one aspect of it that feels like a not-so-pretty patch on the face of Go.

Cheers. 🍺 🍺

var out CmdFlags.{cmdFlagsSpacerPrelogue, Lower, SpecialChars, cmdFlagsSpacerEpilogue} = *in0, *in1, *in2, *in3

Why would I write this instead of just using a composite literal?

I understand that you have a use case for this, but it seems quite specialized to your code. Can you point to any places in the Go standard library, or in popular Go packages, where this new syntax would be desirable? Thanks.

@ianlancetaylor
A key point of @sammy-hughes's is to not make a new value, similar to the way slicing an array reuses the underlying array. So a composite literal is not an option. And he wants type safety too, so unsafe is not an option either. In my previous post I expounded on what I think is the best he can realistically hope for - a "struct view" type such that the "struct view"/struct pair mirrors the semantics of the slice/array pair.

In the var out CmdFlags example, we are creating a new value: CmdFlags. Using a composite literal is going to wind up producing exactly the same executable code.

@ianlancetaylor
I see. But from what @sammy-hughes has written so far, I've understood that his "destructuring" idea is all about using type safe sub-structs without creating copies. I hope he chimes is on that, but I suspect my idea about "struct views" might serve his purposes.

And just for what it's worth, could you comment briefly on whether you think such a feature could be generally useful - a "struct view" type, tied to a struct type, whose value is a "struct view" header (similar to a slice header) made up of just a pointer to the actual struct - basically doing for structs what slices do for arrays? Would it be worth it to open an separate proposal issue, or would you outright dismiss it?

From my perspective slices are basically safe pointers. For every case where in C you can safely use pointer addition, in Go you can use a slice. (For cases where in C you can unsafely use pointer addition, in Go you can use unsafe.Pointer.) The connection between slices and arrays is that the array is the range safely covered by the slice-as-pointer. (With this perspective we see that Go append is basically C realloc.)

My understanding of the struct view idea is that it is not that. A struct view is a way of picking out certain fields of a struct and accessing them directly. It's a form of narrowing and aliasing. And to some extent it's a form of renaming, although in practice, due to the way that pointers work in Go, that can only work for very limited cases, namely when the layout of the fields in the underlying struct precisely matches the layout of the fields in the renaming struct view.

The struct view is an interesting idea but I don't see a lot of benefit for real code. If we look back at https://go.dev/blog/go2-here-we-come we see that the first criteria for a language change is "address an important issue for many people." I don't myself see a lot of code that would benefit from struct views, and I don't see people expressing a need for anything like struct views. So I don't see it as a likely language addition. Sorry.

@ianlancetaylor
That's right - the struct case is different from the array case as structs don't ever change in size. What you're saying makes perfect sense. Thanks.

So, looong weekend dogsitting some very active Australian Shepards. A few quick points:

1. @ianlancetaylor, I would like a quick yes/no if examples from other languages would be considered as support. Meanwhile, I do observe your poo-pooing (Not derizive or dismissive). What I'm proposing is a very low-level optimization, and while I'll definitely dig for some candidates, I don't expect to find much in the Go ecosystem. I do believe such a feature would be prolifically useful in projects where it is applicable, at best that would be in niche domains, for few projects..
2. A performance possibility that could possibly completely satisfy me would be some way to guarantee that methods/functions are inlined, potentially even that guarantee with the addition of "of arbitrary complexity." I described a solution I'm presently using to maximize cache-ops, but which relies on a closure.
3. @3bodar, ignoring the sounds-like-a-verdict from Ian for sec, specifying that the compiler can infer segments requested, whenever it is ambiguous which fields you intended, a compile-time error would be fitting. I agree with you that such syntax would be more convenient if that behavior were possible, however, it means A, a big chunk of additional complexity, and B, it becomes less clear what is occurring. That said, your proposed syntax is as good as mine, only with the exception that I'm not clear on how one would define a view across multiple types, being effectively the same type.
4. I find the comments above generally show an understanding of what I'm suggesting, and there's nothing I feel warrants specific correction. (You're good, @3bodar)

Examples of other languages that provide similar features would be considered as support.

(If you mean examples of other languages that use something like unsafe pointer casting to do a similar operation, that is less interesting. Other languages have different idioms and requirements, and in particular C and C++ specify plain-old-data structure layouts more precisely than Go does, when combined with relevant processor ABI docs. And one can do the unsafe pointer casting in Go, too.)

@ianlancetaylor, the context being explicitly "compiler-guaranteed safe aliasing of structured types based on contract-in-code", and examples of how that's used.

@sammy-hughes

...how one would define a view across multiple types, being effectively the same type.

The straightforward way to achieve something like that would be to explicitly define views per struct type and factor out the common part into a new pair of struct/view:

type A struct { ... }
type ViewA struct(A) { ... }
type B struct { ... }
type ViewB struct(B) { ... }
type Common struct { ... }
type CommonView struct(Common) { Common } // the view is the whole struct

In this setting the compiler would have no problem doing the narrowing conversions from ViewX to CommonView if those are "effectively the same type", so a func can accept a CommonView. Going in the other direction using just a conversion is obviously impossible as the original type is lost, but I'm not sure that's a problem.

Anyway, I've practically lost interest in this issue as I think Go simply isn't the right language for what you're trying to do, so I see myself bailing out of the discussion. But still, good luck...

@3bodar, I also am mostly at your conclusion. I'll be spending the weekend satisfying the request for examples of projects which could/do benefit from such a feature, but after that I think I'll be ready to close the issue.

If ViewA and ViewB are both described by a field list which is convertible to Common, then I think most of my wishlist could be signified by such. One would still need to cast from ViewA/ViewA to Common, but there is no more difficulty suggested by that than by my suggested syntax, and given a little more complexity in tooling changes, significantly less difficulty is presented to the user of your suggested syntax.

I think the only real difference in capability would be whether the syntax could be used to alias field-ranges from types in other packages, and I'm not clear on how easily non-contiguous ranges of fields would be described, and whether any reduction in vectorized cache-stored values could be achieved--granted, I'm not 100% on whether that's possible as a benefit to be had, without copy, with my version of such syntax either.

That said, I expect to still be writing my own static-analysis tools for these kinds of projects, approval or no, for quite a while.

Based on the discussion above, this is a likely decline. Leaving open for four weeks for final comments.

@ianlancetaylor, I did indicate that I planned to dig for some examples. I haven't been able to spend meaningful time on that yet. I accept the "Proposal-FinalCommentPeriod" tag.

I know of several structures that are like what I'm talking about, in C# and Rust, but I'm not clear how much of what I'm proposing is actually included there, and time I've spent has mostly been there. Again, though, I haven't really added meaningful research into existing features of other languages and uses of such, and I have spent no time on projects in the Go ecosystem.

I recognize that this is still in final comment period, but nearly a month after my initial intent to explore, i still haven't gotten to this. I have found that Rust's traits and C++'s "concepts" have some overlap here, but they don't hit all the points I'd want. While I still think my proposal feasible, if not insignificant labor to implement, I recognize the concerns over broad applicability.

I'm going to close this ticket, accepting that the proposal is already in "Likely decline" status. Should I feel continued need, I may develop the idea into a distinct code-gen tool, but as of now, I agree with the consensus that it does not fit naturally as a Go feature...yet.

Thank you @ianlancetaylor and @griesemer for the due consideration, and to @beoran and @3bodar, I enjoyed the pushback and suggestions around this feature proposal.

As a post-close update, this exact functionality was implemented in Rust as a library. It is healthy, being actively maintained for over a year, and it is used by one other package. This is a point, yes, for usefulness, but it's also a positive case of functionality not being directly suitable (at least yet) for core language implementation, being implemented independently.

Further, the approach taken is one which requires that views are implemented in the same package (Go parlance for Rust's "mod" keyword) as the type subject to the view. To implement this in Go, a function-call is still required, as Go has yet to adopt a feature like Rust's Trait construct. Additionally, as this implementation depends on declarative, privileged prep-work, a responsible implementation would likely follow that same pattern; as @3bodar suggested, with the view being declaratively prepared in the same owning package as a type subject to view.

This was implemented using Rust's unsafe keyword, and Rust has a relaxed view of "unsafe" usage as compared to Go, having syntax rules that control scope of "unsafe"ness, and specific community conventions around "unsafe" use. Go currently has a stop-gap approach, wherein some "unsafe" functionality simply requires that "unsafe" is imported, even if as _, and applies silently to the entire file. I believe this provides more confirmation that Go is not yet at a state that would support core tooling support for such library or special syntax for that functionality, without a preceding declarative element.

In other words, one of the more progressive languages I can think of has a similar functionality that only fits as a library, and then requires specific preparation work and direct ownership of types being subject to such a view. This is solid confirmation that "Decline" was the correct choice.

The referenced crate (in go, equivalent to a module) is structview: https://crates.io/crates/structview/1.1.0