proposal: runtime: add AlignedN types that can be used to increase alignment #19057

ianlancetaylor · 2017-02-13T16:27:49Z

The sync/atomic packages have this in the docs in the "Bugs" section: "On both ARM and x86-32, it is the caller's responsibility to arrange for 64-bit alignment of 64-bit words accessed atomically. The first word in a global variable or in an allocated struct or slice can be relied upon to be 64-bit aligned." This makes it difficult to use atomic operations in types that may not necessarily be at the beginning of an allocated struct or slice. For example, sync.WaitGroup does this:

	// 64-bit value: high 32 bits are counter, low 32 bits are waiter count.
	// 64-bit atomic operations require 64-bit alignment, but 32-bit
	// compilers do not ensure it. So we allocate 12 bytes and then use
	// the aligned 8 bytes in them as state.
	state1 [12]byte

and this:

func (wg *WaitGroup) state() *uint64 {
	if uintptr(unsafe.Pointer(&wg.state1))%8 == 0 {
		return (*uint64)(unsafe.Pointer(&wg.state1))
	} else {
		return (*uint64)(unsafe.Pointer(&wg.state1[4]))
	}
}

Further, on x86 there are vector instructions that require alignment to 16 bytes, and there are even some instructions (e.g., vmovaps with VEC.256), that require 32 byte alignment. While those instructions are not currently generated by the gc compiler, one can easily imagine using them in assembler code, which will require the values to be appropriately aligned.

To permit programmers to force the desired alignment, I propose that we add new types to the runtime package: runtime.Aligned2, runtime.Aligned4, runtime.Aligned8, runtime.Aligned16, runtime.Aligned32, runtime.Aligned64, runtime.Aligned128. (We could also use bit values, giving us runtime.Aligned16 through runtime.Aligned1024, if that seems clearer.)

These types will be identical to the type struct{} except that they will have a the alignment implied by the name. This will make it possible to write a struct as

type vector struct {
    vals [16]byte
    _ runtime.Aligned16
}

and ensure that instances of this struct will always be aligned to a 16 byte boundary.

It will be possible to change sync.Waitgroup to be

type WaitGroup struct {
    noCopy noCopy
    _ runtime.Aligned8
    state uint64
    sema uint32
}

simplifying the code.

Although this functionality will not be used widely, it does provide a facility that we need today without requiring awkward workarounds. The drawback is the addition of a new concept to the runtime, though I think it is fairly clear to those people who need to use it.

Another complexity is that we will have to decide whether the size of a value is always a multiple of the alignment of the value. Currently that is the case. It would not be the case for the runtime.AlignedN values. Should it be the case for any struct that contains a field with one of those types? If the size of a value is not always a multiple of the alignment, we will have to modify the memory allocator to support that concept. I don't think that will be particularly difficult, but I haven't really looked.

The text was updated successfully, but these errors were encountered:

josharian · 2017-02-13T18:41:09Z

A few minor comments:

If we do this, we should probably do go/types: add GCSizes #17586 too. It's not cmd/vet that relies on go/types for information about what cmd/compile will do alignment-wise. Another example is wasted.
Your struct vector above will have a byte of padding at the end to avoid GC problems. You probably want the struct{} field as the first field. And we'll want to carefully document that recommendation. I mention this only because if you missed this detail, others definitely will.
This could also be done with a //go: annotation on the type. We already have other //go: annotations on types. Don't panic. I'm not suggesting we do this. (Even the mere mention of annotations tends to raise banshee-level howling.) But it does raise the general question about when and why to use magic embedded types vs annotations vs perhaps some other general mechanism that doesn't exist yet.

ianlancetaylor · 2017-02-13T19:18:26Z

It's true that that we do have go:nointerface and go:notinheap annotations on types, but neither is documented and the latter is clearly only for the runtime. I suppose my general reaction to support //go: annotations on types can be summed up as https://www.youtube.com/watch?v=hulm_T_xnwY .

cespare · 2017-02-13T19:59:41Z

@ianlancetaylor when you wrote

(We could also use runtime.Aligned16, etc., if that seems clearer.)

I don't see how this differs from the previous sentence.

ianlancetaylor · 2017-02-13T20:03:28Z

@cespare I meant to imply using bit values rather than byte values. Updated original comment to clarify. Those names would mean that the AlignedN names would correspond to intN names.

rsc · 2017-02-13T20:30:09Z

I agree we should fix this problem. I am less certain about how to fix it. Perhaps to start we should just align 64-bit integers to 64-bit addresses on 32-bit platforms. It's called out in sync/atomic because it's basically a bug on our side, one that we've just not fixed.

The solution proposed here essentially assumes the compiler will not reorder fields, at least not if these tags are present. I don't think we've fully closed the door on that (#10014). In that issue (two years ago), I argued that it is important for the programmer to have control over locality, so wholesale reordering of fields is not great (for example, sort by size and then lay out would give optimal packing but I think be too invasive).

At the same time, I am getting tired of looking for uint32-sized or bool-sized holes when adding fields to existing structs, and even more I am getting tired of being forced to choose between "understandable struct definition" and "small-in-memory struct definition". I do wonder if the compiler should be able by default to sift individual small fields up into gaps that would otherwise go unfilled, but not otherwise reorder the definitions. This is getting off-topic for this issue, except that any such scheme would need an override annotation for cgo and so that might give a mechanism for expressing alignment as well; of course any reordering would need to keep alignment in mind. I don't have any good ideas.

Also not every variable that needs alignment is a field in a struct. I'm not sure what to do about that either. Code might declare 'var x [16]byte' and want to pass it to something that requires 16-byte alignment, for example. Maybe that's getting too far ahead of ourselves, but it's worth keeping in mind.

I had hoped that alignment would be a property of a type, not a specific declaration. Are there cases where that's not tenable?

mdempsky · 2017-02-13T20:55:42Z

runtime.AlignedCache (which could just alias an appropriate AlignedN type) would help address #19025.

ianlancetaylor · 2017-02-14T00:05:48Z

I did not mean to imply that this approach meant that the compiler could not reorder fields. The language spec does not state that the alignment of one field in a struct implies anything at all about the alignment of subsequent fields in the struct. I envision any uses of this as being of the form

type Vec struct {
    _ runtime.Aligned16
    b [16]byte
}

Here we know that any instance of Vec is aligned to a 16-byte boundary.

I agree that alignment should be a property of a type, and I believe that that satisfies all alignment needs in Go. The question is how you specify that alignment, and whether it can be done without using a magic comment. This proposal is one approach: in effect, you can only specify the alignment of struct types, and you do so by adding a field of type runtime.AlignedN.

For comparison, in C (with GCC extensions) you specify alignment of a type by writing

typedef T ... __attribute__((aligned(n)));

You can also specify alignment of a specific variable in the same way. Or, you can implement alignment for a specific memory allocation by using memalign or posix_memalign (since C's memory allocator does not understand types, it is generally necessary to use these functions when allocating memory for an aligned type).

Clearly for Go it would be nicer to be able to declare the alignment of any type, rather than this proposal which in effect only permits you to declare the alignment of a struct. If we can figure out a way to do that, we should. But I'd really rather not do it via a magic comment.

minux · 2017-02-14T00:16:06Z

For sync/atomic, the biggest problem is usually not the alignment of the whole structure (because we have at least 8 byte alignment on struct larger than 8 byte), but ensuring that the next uint64 will be aligned correctly within the struct. A nice solution is to just increase alignment of 64-bit types on 32-bit architectures. For vectors, I think once we figure out how to use SIMD intrinsics, we will have types of 128-bit, 256-bit and 512-bit alignment.

ianlancetaylor · 2017-02-14T00:44:10Z

For sync/atomic, the biggest problem is usually not the alignment of the
whole structure (because we have at least 8 byte alignment on struct larger
than 8 byte), but ensuring that the next uint64 will be aligned correctly
within the struct.

Yes, understood. With this proposal you write "the next uint64" as a field of type alignedUint64, a struct defined as

type alignedUint64 struct {
    _ runtime.Aligned8
    v uint64
}

Vectors and sync/atomic are not the only uses of aligned types, so I think a more general solution would be a good idea if we can find one.

cespare · 2017-02-14T00:46:26Z

Here's a slight variation on the original proposal:

Add runtime.Align2, ... runtime.Align128. (Note: "align", not "aligned")
These are just like struct{}, except that if a runtime.AlignN, is used as a struct field, it specifies the alignment of the following field (in declared order).
If a runtime.AlignN is declared as the last struct field, it specifies the alignment of the entire struct.

This gets to @minux's point about aligning particular struct fields. You could even use multiple AlignNs to align multiple fields in a single struct.

(AlignN is more like a magic comment than AlignedN is, but at least it isn't a comment.)

rsc · 2017-02-14T01:15:39Z

@ianlancetaylor Thanks for the clarification. I was slightly confused by the fact that in your original post 'struct vector' (really should be 'type vector struct') puts the alignment after the field, not before.

minux · 2017-02-14T01:45:02Z

On Feb 13, 2017 7:46 PM, "Caleb Spare" ***@***.***> wrote: Here's a slight variation on the original proposal: - Add runtime.Align2, ... runtime.Align128. (Note: "align", not "aligned") - These are just like struct{}, except that if a runtime.AlignN, is used as a struct field, it specifies the alignment of the *following* field (in declared order). - If a runtime.AlignN is declared as the last struct field, it specifies the alignment of the entire struct.

I think any AlignN will set the alignment for the whole structure. The logic is this: In order to ensure N byte alignment for a particular field, we must first ensure at least N byte alignment for the whole structure and then make sure the field has N byte aligned offset within the structure. I understand this proposal diverges from existing C convention, but I think it's more intuitive to the programmer. And we can also add a magical AlignCacheLine type to force at least cache line alignment in order to reduce false sharing (i think this is better than #19025). In order to not make the types too magical from the language specification's perspective, I suggest that we make them like this: type alignedbyte byte // the only magical and unexported type, only allowed as underlying type of an array, the resulting array has the alignment as the size of the array type aligned16 [16]alignedbyte // a 16-byte aligned type which takes 16-byte of space. type Align16 [0]aligned16 // zero size, can be embedded into other structs to force alignment of the next field (and also the whole structure) This takes advantage of recently clarified spec #18950, regarding alignment of [0]T.

josharian · 2017-02-14T01:57:02Z

If a runtime.AlignN is declared as the last struct field, it specifies the alignment of the entire struct.

Zero-sized final fields cause cmd/compile to insert a padding byte at the end. I imagine that is unacceptable in many of the cases for which increasing alignment is important.

rsc · 2017-02-14T02:22:02Z

As Minux pointed out (hard to tell with the Github-mangling of the email reply), AlignN declared anywhere would end up specifying a min alignment for the entire struct, since a struct can't have alignment less than any of its fields.

josharian · 2017-02-14T02:56:07Z

Thanks, Russ, I missed that.

Clever, Minux. Seems like it might be worth accepting it is a language change and defining unsafe.AlignedByte and letting folks take it from there.

ianlancetaylor · 2017-02-14T04:31:28Z

alignedbyte is clever but I'm a little uncomfortable with the idea that [13]alignedbyte has an alignment requirement of 13 bytes.

josharian · 2017-02-14T04:44:01Z

Perhaps it could specify a minimum alignment, and compilers could choose to round up to the nearest power of two.

cespare · 2017-02-14T05:24:01Z

Regarding Minux's idea:

type Align16 [0]aligned16 // zero size, can be embedded into other structs
to force alignment of the next field (and also the whole structure)

To clarify, isn't there "magic" required to guarantee this relationship with the next field (same as in my AlignN version), if the compiler can reorder fields?

minux · 2017-02-14T05:28:20Z

The reason my original comment didn't export the alignedbyte type is because I only intend the runtime package to instantiate power-of-two sized array of it and then expose zero sized array of those types. I don't think we need to fully expose the magical alignedbyte type to the user (as a language extension.)

rsc · 2017-02-14T05:31:49Z

@josharian "minimum alignment" is not a well-defined concept unless the space of possible requests are all multiples or divisors of each others. A multiple of 16 is not a multiple of 13.

I tend to agree with Ian that alignedbyte is a little too much rope.

minux · 2017-02-14T05:37:27Z

I think the compiler definitely cannot reorder arbitrary structures. Perhaps we can embed a unsafe.PackedStruct to signify that the compiler can rearrange a struct, but I still think packing a struct is better done by another program, not automatic by the compiler. One question for the runtime.AlignN idea: could it be used to reduce the alignment? I.e. can it be used to pack a struct? type packed struct { // size 5 (instead of 8) uint8 _ runtime.Align1 uint32 // at offset 1 (instead of 4 and leave 3 byte padding) } Packing a struct is an occasionally requested feature (and it will actually help cgo w.r.t. complex types), but supporting that on some architectures means non-atomic accesses to non-naturally-aligned fields.

josharian · 2017-02-14T05:37:51Z

Ack.

The thing I am struggling with is that once you start introducing magic, it is unclear which the right magic is. A magic field type in package runtime? A comment-based type annotation? An interpreted field tag (or type tag)? A magic interface in package runtime (check whether a type has an Aligned2 method)?

This does feel a bit like a language change, though, and unsafe does seem like the right home for manually messing with alignments. Maybe there's an alternative unsafe formation that provides less rope? Here's a terrible idea to start: unsafe.AlignedShift: [n]unsafe.AlignedShift has alignment 1<<n. :)

rsc · 2017-02-14T06:10:27Z

I think the compiler definitely cannot reorder arbitrary structures.

What you think is only a little interesting; telling us why is much more interesting.

Perhaps we can embed a unsafe.PackedStruct to signify that the compiler can rearrange a struct, but I still think packing a struct is better done by another program, not automatic by the compiler.

That's fine for structs that people don't look at. What bothers me most about packing structs explicitly (by hand or with a program) is that doing so rewrites the source code to be less readable.

bcmills · 2017-02-14T14:14:15Z

Allowing the compiler to reorder structures would be tricky: we may need to distinguish between unoptimized layouts (which the compiler should obviously fix) and hand-optimized layouts.

If the author has intentionally adjusted cache-line locality or packed the struct to match a kernel or C data structure, how do we tell the compiler not to break that?

rsc · 2017-02-15T15:01:21Z

@bcmills, struct field reordering is #10014. When I mentioned it above I wrote:

This is getting off-topic for this issue, except that any such scheme would need an override annotation for cgo and so that might give a mechanism for expressing alignment as well.

minux · 2017-02-15T15:08:13Z

@rsc, it seems you used to oppose the idea of compiler reordering the fields. why you come to think that field reordering is beneficial now? #10014 (comment) And I still agree with your comment that packing the struct is solving the 1970 problem, and the 2010 problem can't be solved by the compiler, at least not alone.

bcmills · 2017-02-15T15:49:06Z

I would guess that most uses of alignment fall into one of two categories:

Passing a pointer to an unexported field to an atomic function.
Passing a struct to a C function (via cgo or a syscall).

We normally let the compiler figure out details of allocation and layout. Perhaps we could do the same for alignment most of the time.

We could do something akin to escape analysis to see which fields need to be aligned:

If a pointer to a field is passed as a function parameter that requires a particular alignment, then both the field and the struct require at least that alignment. Pointers passed to functions in package atomic must be aligned to their element size.
If a pointer to a field or struct is passed to a cgo function call, syscall, or converted to an unsafe.Pointer which may be passed to a cgo function call or syscall, then the field and the struct require C-compatible alignment.

For the few remaining cases (are there any?), perhaps we could add a no-op function call (akin to runtime.KeepAlive):

package runtime

// Align marks its argument as requiring the given alignment.
// The ptr argument must be a pointer to a variable of a struct type,
// or a pointer to a field on a variable of a struct type. 
// The alignment argument must be a compile-time constant.
func Align(ptr interface{}, alignment int)

The only situation I'm aware of that would require explicit calls to Align would be if a pointer is allocated and returned from a function in one package but the alignment constraints occur only in the calling package. I cannot think of any examples of such usage at the moment.

josharian · 2017-02-15T15:54:18Z

@bcmills a remaining case: argument to user-written assembly routine using vector instructions. (Minux mentioned this above too.)

rsc · 2017-04-10T15:53:24Z

On hold per discussion above.

gopherbot · 2017-04-20T00:57:23Z

CL https://golang.org/cl/41143 mentions this issue.

On 32-bit x86 a uint64 variable by itself is aligned to an 8-byte boundary. A uint64 field in a struct is aligned to a 4-byte boundary. The runtime.ticks variable has a uint64 field that must be aligned to an 8-byte boundary. Rather than rely on luck, split up the struct into separate vars so that the required alignment happens reliably. It would be much nicer if issue golang/go#19057 were fixed somehow, but that is for another day. Change-Id: If609ed165fb9cb1ea89fd351268c9984aa5df20d Reviewed-on: https://go-review.googlesource.com/41143 Reviewed-by: Than McIntosh <thanm@google.com>

GCC release freeze. * go-backend.c: Include "go-c.h". * go-gcc.cc (Gcc_backend::write_export_data): New method. * go-gcc.cc (Gcc_backend::Gcc_backend): Declare __builtin_prefetch. * Make-lang.in (GO_OBJS): Add go/wb.o. commit 884c9f2cafb3fc1decaca70f1817ae269e4c6889 Author: Than McIntosh <thanm@google.com> Date: Mon Jan 23 15:07:07 2017 -0500 compiler: insert additional conversion for type desc ptr expr Change the method Type::type_descriptor_pointer to apply an additional type conversion to its result Bexpression, to avoid type clashes in the back end. The backend expression for a given type descriptor var is given a type of "_type", however the virtual calls that create the variable use types derived from _type, hence the need to force a conversion. Reviewed-on: https://go-review.googlesource.com/35506 commit 5f0647c71e3b29eddcd0eecc44e7ba44ae7fc8dd Author: Than McIntosh <thanm@google.com> Date: Mon Jan 23 15:22:26 2017 -0500 compiler: insure tree integrity in Call_expression::set_result Depending on the back end, it can be problematic to reuse Bexpressions (passing the same Bexpression to more than one Backend call to create additional Bexpressions or Bstatements). The Call_expression::set_result method was reusing its Bexpression input in more than one tree context; the fix is to pass in an Expression instead and generate multiple Bexpression references to it within the method. Reviewed-on: https://go-review.googlesource.com/35505 commit 7a8e49870885af898c3c790275e513d1764a2828 Author: Ian Lance Taylor <iant@golang.org> Date: Tue Jan 24 21:19:06 2017 -0800 runtime: copy more of the scheduler from the Go 1.8 runtime Copies mstart, newm, m0, g0, and friends. Reviewed-on: https://go-review.googlesource.com/35645 commit 3546e2f002d0277d805ec59c5403bc1d4eda4ed9 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Jan 26 19:47:37 2017 -0800 runtime: remove a few C functions that are no longer used Reviewed-on: https://go-review.googlesource.com/35849 commit a71b835254f6d3164a0e6beaf54f2b175d1a6a92 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Jan 26 16:51:16 2017 -0800 runtime: copy over more of the Go 1.8 scheduler In particular __go_go (aka newproc) and goexit[01]. Reviewed-on: https://go-review.googlesource.com/35847 commit c3ffff725adbe54d8283c373b6aa7dc95d6fc27f Author: Ian Lance Taylor <iant@golang.org> Date: Fri Jan 27 16:58:20 2017 -0800 runtime: copy syscall handling from Go 1.8 runtime Entering a syscall still has to start in C, to save the registers. Fix entersyscallblock to save them more reliably. This copies over the tracing code for syscalls, which we previously weren't doing, and lets us turn on runtime/trace/check. Reviewed-on: https://go-review.googlesource.com/35912 commit d5b921de4a28b04000fc4c8dac7f529a4a624dfc Author: Ian Lance Taylor <iant@golang.org> Date: Fri Jan 27 18:34:11 2017 -0800 runtime: copy SIGPROF handling from Go 1.8 runtime Also copy over Breakpoint. Fix Func.Name and Func.Entry to not crash on a nil Func. Reviewed-on: https://go-review.googlesource.com/35913 commit cc60235e55aef14b15c3d2114030245beb3adfef Author: Than McIntosh <thanm@google.com> Date: Mon Feb 6 11:12:12 2017 -0500 compiler: convert go_write_export_data to Backend method. Convert the helper function 'go_write_export_data' into a Backend class method, to allow for an implementation of this function that needs to access backend state. Reviewed-on: https://go-review.googlesource.com/36357 commit e387439bfd24d5e142874b8e68e7039f74c744d7 Author: Than McIntosh <thanm@google.com> Date: Wed Feb 8 11:13:46 2017 -0500 compiler: insert backend conversion in temporary statement init Insert an additional type conversion in Temporary_statement::do_get_backend when assigning a Bexpression initializer to the temporary variable, to avoid potential clashes in the back end. This can come up when assigning something of concrete pointer-to-function type to a variable of generic pointer-to-function type. Reviewed-on: https://go-review.googlesource.com/36591 commit c5acf0ce09e61ff623847a35a99da465b8571609 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 1 17:57:53 2017 +0100 libgo: build tags for aix Build tags for the libgo source files required to build libgo on AIX. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37633 commit 67ed19616898ea18a101ec9325b82d028cd395d9 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 2 15:41:31 2017 +0100 libgo: handle AIX tag in match.sh and gotest Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37638 commit 83ea2d694c10b2dd83fc8620c43da13d20db754e Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 1 17:48:16 2017 +0100 libgo: add AIX support in configure and Makefile - support for GOOS=aix - CFLAGS/GOCFLAGS/LDFLAGS for AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37632 commit 35d577fe22ffa16a3ccaadf5dae9f6f425c8ec8c Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Mon Mar 6 15:00:15 2017 +0100 runtime: adapt memory management to AIX mmap On AIX: * mmap does not allow to map an already mapped range, * mmap range start at 0x30000000 for 32 bits processes, * mmap range start at 0x70000000_00000000 for 64 bits processes Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37845 commit 4e49e56a5fd4072b4ca7fcefe4158d6885d9ee62 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Mon Mar 6 13:42:26 2017 +0100 runtime: add getproccount implementation for AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37844 commit ff626470294237ac664127894826614edc46a3d0 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Mon Mar 6 17:31:21 2017 +0100 runtime: handle ERESTART errno with AIX's wait4 On AIX, wait4 may return with errno set to ERESTART, which causes unexepected behavior (for instance, go build may exit with the message "wait: restart system call" after running a command, even if it was successfull). Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37846 commit 37daabbfc83d533b826ef9ab10e2dee7406e7198 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Mon Mar 6 11:02:58 2017 +0100 runtime: support for AIX's procfs tree On AIX, the process executable file is available under /proc/<pid>/object/a.out Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37842 commit a0275c039d56acf4bf48151978c1a4ec5758cc2c Author: Ian Lance Taylor <iant@golang.org> Date: Wed Mar 8 07:00:05 2017 -0800 libgo/Makefile.am: don't use nonportable \n or \t in sed expression The resulting zstdpktlist.go is less pretty, but it works. Reviewed-on: https://go-review.googlesource.com/37940 commit 29b190f76105aafa2b50b48249afdafecc97a4be Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 9 16:02:34 2017 +0100 runtime: netpoll and semaphores for AIX semaphore implementation based on Solaris implementation in libgo/go/runtime/os_solaris.go netpoll is just a stub to avoid build failure on AIX. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37966 commit 55ca6d3f3cddf0ff9ccb074b2694da9fc54de7ec Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 9 15:38:30 2017 +0100 libmain: ensure initfn is called when loading a go library AIX does not support .init_array. The alterative is to export the __go_init function and tell the linker it is an init function with the -Wl,-binitfini:__go_init option. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37965 commit 349a30d17d880ac8bc1a35e1a2ffee6d6e870ae9 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Fri Mar 10 11:15:08 2017 +0100 libgo: use an import list for missing symbols libgo depends on symbols provided by Go programs at runtime. On AIX, this requires either to build libgo with -Wl,-berok linker option and the programs with -Wl,-brtl, or to provide a list of imported symbols when building libgo. The second options seems preferable, to avoid requiring an additional option for every Go program. There are also some symbols that are specific to GNU ld and do not exist when linking with AIX ld (__data_start, __edata, __etext and __bss_start). Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37969 commit 91db0ea1ff068ca1d97b9c99612100ea5b96ddb2 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 8 15:34:45 2017 +0100 crypto/x509: add certificate files locations for AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37952 commit 92e521c854e91709b949548c47e267377850f26a Author: Ian Lance Taylor <iant@golang.org> Date: Fri Mar 10 14:10:11 2017 -0800 compiler: fix check for pointer in Temporary_reference_expression The check for an unrepresentable pointer in Temporary_reference_expression::do_get_backend was incorrectly translated from C to Go in https://golang.org/cl/14346043. Fix the check to use points_to rather than has_pointer and deref. This should not make any difference in practice as either way the condition will only be true for a pointer to void, but points_to is correct and more efficient. Reviewed-on: https://go-review.googlesource.com/38009 commit 9a0b676e59e7171a630c48fdc3d4de6712bad0ca Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 16 16:51:53 2017 +0100 libgo: add missing _arpcom struct to *sysinfo.go This struct is filtered due to having a field of type _in6_addr, but other types exported to *sysinfo.go are depending on it. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38251 commit 61262a757bdd3d9a595ab6a90f68c0c4ebed7bc1 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 16 18:27:46 2017 +0100 syscall: raw_ptrace stub for AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38253 commit 8029632b50880fd9b5e39299c738b38e3386595f Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 15 16:58:37 2017 +0100 libgo: adapt runtime.inc to AIX * Two AIX types are wrongfully exported to runtime.inc as their names make them look like a Go type. * The sigset go type conflicts with a system sigset type. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38192 commit 25f3a90d14bc268479369ecc0eada72791612f86 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 15 16:58:37 2017 +0100 libgo: update Makefile.in, accidentally omitted from last change Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38310 commit d52b4895616b66f93b460366527e74336829aaa5 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 16 18:39:26 2017 +0100 syscall: TIOCSCTTY does not exist on AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38254 commit ff1ec3847a4472008e5d53a98b6694b1e54ca322 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 16 18:07:34 2017 +0100 syscall: syscall does not exist on AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38252 commit c1ee60dabf0b243a0b0286215481a5d326c34596 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Fri Mar 17 17:18:18 2017 +0100 net: EAI_OVERFLOW does not exist on AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38266 commit ad4ad29aed9f70b14b39b488bfeb9ee745382ec4 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Fri Mar 17 17:23:56 2017 +0100 net: sockopt/sockoptip stubs for AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38267 commit 5d7db2d7542fe7082f426d42f8c2ce14aad6df55 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Fri Mar 17 16:35:05 2017 +0100 os/user: add listgroups stub for AIX This is required to build os/user. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38263 commit 4e57a7973e9fa4cb5ab977c6d792e62a8f7c5795 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 22 11:11:30 2017 +0100 os: fix readdirnames for AIX Largefile implementation should be used on AIX. readdir64_r function returns 9 and sets result to NULL when reaching end of directory, so this return code should not always be considered as an error. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38359 commit b34036967d1ec57b25e3debe077439b4210a1d4a Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Fri Mar 17 17:39:31 2017 +0100 libgo: adapt sigtab.go to AIX On AIX, _NSIG is not directly defined to its integer value in gen-sysinfo.go. The real value is _SIGMAX32+1 or _SIGMAX64+1, depending if we are building a 32bit ligbo or a 64bit libgo, so we need to read one of those constants to set nsig value in mksigtab.sh This change also ensures that all signal numbers from 0 to nsig-1 are referenced in sigtable. Reviewed-on: https://go-review.googlesource.com/38268 commit 20991c32671a183ec859b4f285df37fdd4634247 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 23 17:28:09 2017 +0100 syscall: missing import in socket_bsd.go Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38369 commit c34754bd9adf5496c4c26257eaa50793553c11e8 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 22 17:57:01 2017 +0100 sycall: WCOREDUMP macro is not defined on AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38363 commit 4f38813482227b12ea0ac6ac1b981ff9ef9853ef Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 23 17:44:43 2017 +0100 libgo: additional build tags for AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38510 commit d117ede6ff5a7083e9c40eba28a0f94f3535d773 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 23 17:48:46 2017 +0100 go/build: add AIX to "go build" command known OS Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38511 commit 7b0ddaa6a6a71f9eb1c374122d29775b13c2cac5 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Mar 23 09:57:01 2017 -0700 compiler: don't crash if imported package imports this one When building a test it's OK if test code imports a package that imports this one. The go tool is supposed to catch cases where this creates an impossible initialization order. The compiler already has code to permit this in Gogo::add_import_init_fn. This CL avoids a compiler crash on a similar case when writing out the export data. I have no test case for this. Basically it pushes a compiler crash into an error reported elsewhere. Problem was reported by Tony Reix. Reviewed-on: https://go-review.googlesource.com/38462 commit 925636975d075e3e3353823b09db3f933f23cb03 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Mar 29 14:14:18 2017 -0700 runtime: copy finalizer support from Go 1.8 runtime Reviewed-on: https://go-review.googlesource.com/38794 commit 1ccb22b96cb3b1011db0e427877d9ddecb577fa9 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 30 15:21:06 2017 +0200 runtime: initcontext and setcontext stubs for AIX Further investigations are required to understand the clobbering issue and implement a proper fix. Until then, those stubs are required to allow the build to complete. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38930 commit 27db481f369b54256063c72b911d22390c59199c Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 29 18:07:25 2017 +0200 os: fix Readlink failure on AIX AIX readlink routine returns an error if the link is longer than the buffer, instead of truncating the link. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38700 commit c93babbf48eddd0bc34d4179ffb302dc60087299 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 29 17:26:35 2017 +0200 compiler: implement support for reading AIX big archives This is required to read go export from a Go library. Code courtesy of Damien Bergamini from Atos Infogérance. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38698 commit 930dd53482bdee3a9074850d168d0b9d7819c135 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Apr 6 18:50:11 2017 -0700 compiler: fix whether conversions are static initializers The compiler was incorrectly treating type conversions from string to int or vice-versa as static initializers. That doesn't work, as those conversions are implemented via a function call. This case may never actually arise but it seems like the right thing to do. Reviewed-on: https://go-review.googlesource.com/39872 commit f02691e4195728dbf06f4dde0853c6bccc922183 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Apr 6 17:24:08 2017 -0700 compiler, runtime: don't let slices point past end of memory block When Go code uses a slice expression like [:len(str)] or [:cap(slice)], it's natural for the resulting pointer to point just past the end of the memory block. If the next memory block is not live, we now have a live pointer to a dead block, which will unnecessarily keep the block alive. That wastes space, and with the new Go 1.8 GC (not yet committed) will trigger an error when using GODEBUG=gccheckmark=1. This changes the implementation of slice expressions to not move the pointer if the resulting string length or slice capacity is 0. When the length/capacity is zero, the pointer is never used anyhow. Reviewed-on: https://go-review.googlesource.com/39870 commit 17527c35b027e1afcc318faf5563909e1e9d44a6 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Apr 6 15:30:11 2017 -0700 compiler: emit write barriers The Go 1.8 concurrent GC requires optional write barriers for all assignments that may change pointer values in the heap or in a global variable. For details see https://blog.golang.org/go15gc. This changes the gofrontend code to emit write barriers as needed. This is in preparation for future changes. At the moment the write barriers will do nothing. They test runtime.writeBarrier.enabled, which will never be non-zero. They call simple functions which just do a move without doing any of the other operations required by the write barrier. Reviewed-on: https://go-review.googlesource.com/39852 commit c0b00f072bf34b2c288e1271ec8118b88c4f6f6f Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Tue Apr 11 17:47:29 2017 +0200 libgo: allow building gox files from PIC objects libtool builds non-PIC objects in the same directory as .lo files and PIC objects in a .libs subdirectory. BUILDGOX rule uses the non-PIC objects to build the gox files, but on AIX only the PIC objects are built. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/40355 commit ea0f3da174c5503a209043f14ddda34871cfec52 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Apr 6 19:06:14 2017 -0700 compiler: add code to generate a ptrmask for a type The Go 1.8 garbage collector uses a ptrmask for all types below a certain size. A ptrmask is simply a bit vector with a single bit for each pointer-sized word in the value. The bit is 1 if the type has a pointer in that position, 0 if it does not. This change adds code to the compiler to generate a ptrmask. The code is not used by anything yet, it is just compiled. It will be used when we switch over to the Go 1.8 garbage collector. The new Array_type::int_length method, and the new memory_size methods, will also be used by other patches coming later. Reviewed-on: https://go-review.googlesource.com/39873 commit 3029e1df3be3614d196a03c15e50e68ff850aa4c Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 7 10:31:39 2017 -0700 compiler: add code to generate a gcprog for a type The Go 1.8 garbage collector uses a gcprog for all types above a certain size. A gcprog describes where the pointers are in the type, using a simple bytecode machine that supports repeating bits. The effect is to permit using much less space to describe arrays. The format is described in runtime/mbitmap.go in the docs for runGCProg. This is not yet added to the gofrontend, but can be seen in the gc sources. This change adds code to the compiler to generate a gcprog. The code is not used by anything yet, it is just compiled. It will be used when we switch over to the Go 1.8 garbage collector. Reviewed-on: https://go-review.googlesource.com/39923 commit 8b01ef1e9176d20f4c9e667972fe031069a4d057 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Apr 13 07:00:35 2017 -0700 compiler: add ptrdata computations and expressions For the upcoming Go 1.8 GC we need to compute the "ptrdata" of a type: the number of bytes in the type that can contain pointers. For types that do not contain pointers this number is zero. For many types it is a number larger than zero but smaller than the total size of the type. The Go 1.8 GC uses this number to make loops looking for pointers run faster by not scanning the suffix of a value that can not contain a pointer. Unfortunately there are two subtly different definitions of ptrdata, and we need both. The first is the simple one: the prefix that can contain pointers. The second is the number of bytes described by the gcprog for the type. Recall that we describe the exact position of pointers in a type using either a ptrmask or a gcprog. The ptrmask is simpler, the gcprog uses less space. We use the gcprog for large types, currently defined as types that are more than 2048 bytes. When the Go 1.8 runtime expands a gcprog, it verifies that the gcprog describes exactly the same number of bytes as the ptrdata field in the type descriptor. If the last pointer-containing portion of a type is an array, and if the elements of the array have a ptrdata that is less than the size of the element type, then the simple definition of the ptrdata will not include the final non-pointer-containing bytes of the last element of the array. However, the gcprog will define the array using a repeat count, and will therefore include the full size of the last element of the array. So for a type that needs a gcprog, the ptrdata field in the type descriptor must be the size of the data described by the gcprog, and that is not necessarily the same as the simple ptrdata. It might seem that we can always use the gcprog version of the ptrdata calculation, since that is what will appear in a type descriptor, but it turns out that for global variables we always use a ptrmask, not a gcprog, even if the global variable is large. This is because gcprogs are handled by expanding them into a ptrmask at runtime, and for a global variable there is no natural place to put the ptrmask. Simpler to always use the ptrmask. That means that we need to describe the size of the ptrmask, and that means that we need an expression for the simple form of the ptrdata. This CL implements the ptrdata calculation. This code is not actually used yet. It will be used later when the Go 1.8 GC is committed. Reviewed-on: https://go-review.googlesource.com/40573 commit 7a37331303b572412179a08141f1dd35339d40c8 Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 14 06:55:48 2017 -0700 compiler: zero length arrays never contain pointers Reviewed-on: https://go-review.googlesource.com/40696 commit c242f0508a64d3d74a28d498cbaeda785ff76258 Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 14 07:26:54 2017 -0700 bytes: disable allocations test on gccgo It turns out that testing.AllocsPerRun has not been producing correct results with the current gccgo memory allocator. When we update to the Go 1.8 memory allocator, testing.AllocsPerRun will work again, and this test will fail due to lack of escape analysis. Reviewed-on: https://go-review.googlesource.com/40700 commit 0dc369f1d63376a36bfb0999a1b0377fd444bfab Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Tue Apr 11 16:22:38 2017 +0200 os: alternative way to find executable path, using Args[0] AIX does not provide a proper way to find the original executable path from procfs, which contains just an hardlink. Executable path can be found using Args[0], Getcwd and $PATH. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/40353 commit f9bad1342569b338e3b2ea9f12ffc6d3d3fa3028 Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 14 08:01:19 2017 -0700 compiler: don't write struct with multiple sink fields to C header file When writing a struct to the C header file used by the C runtime code, a single sink field is fine: it will be called "_", which is valid C. There are structs with single sink fields that we want to write out, such as finblock. As it happens, though, the Go 1.8 runtime has a struct with two sink fields, gcControllerState, which will produce a C definition with two fields named "_", which will fail. Since we don't need to know that struct in C, rather than fix the general case, just punt if the struct has multiple sink fields. After the conversion to the Go 1.8 GC, we may be able to get rid of the C header file anyhow. I'm not sure yet. Reviewed-on: https://go-review.googlesource.com/40701 commit cfc28901a572aeb15b2f10a38f79eec04c64dfb2 Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 14 10:07:23 2017 -0700 runtime: disable allocations test on gccgo It turns out that testing.AllocsPerRun has not been producing correct results with the current gccgo memory allocator. When we update to the Go 1.8 memory allocator, testing.AllocsPerRun will work again, and these tests will fail due to lack of escape analysis. Reviewed-on: https://go-review.googlesource.com/40703 commit 36fedd76edaa48b9ec09709a70d9e4abaddf0caf Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 14 10:47:06 2017 -0700 runtime: remove unused size argument from hash/equal fns The size argument was removed from hash and equal functions in CL 34983. Somehow I missed removing them from three of the predefined functions. Reviewed-on: https://go-review.googlesource.com/40770 commit 90f6accb48d2e78cad8955b9292933f6ce3fe4c8 Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 14 13:23:05 2017 -0700 runtime: remove unused stack.go We're never going to use stack.go for gccgo. Although a build tag keeps it from being built, even having it around can be confusing. Remove it. Reviewed-on: https://go-review.googlesource.com/40774 commit befa71603fc66a214e01ac219f2bba36e19f136f Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 14 13:18:34 2017 -0700 runtime: build fastlog Take out the build tags which were preventing fastlog2 from being built. It's used by the upcoming Go 1.8 GC. Reviewed-on: https://go-review.googlesource.com/40773 commit b7e19e9be4ab4c3cd8f4c9506d79a8cd56bace40 Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 14 10:04:23 2017 -0700 runtime: add tests from Go 1.8 Some runtime package tests never made it into the gofrontend repo for some reason. Add them now. Reviewed-on: https://go-review.googlesource.com/40869 commit 1feef185aebd71bc2a09b9a04287461806096610 Author: Ian Lance Taylor <iant@golang.org> Date: Mon Apr 17 16:26:11 2017 -0700 runtime: change mcall to take a Go function value For future work in bringing in the Go 1.8 GC, change the mcall function to take a Go function value, which means that mcall can take a closure rather than just a straight C function pointer. As part of this change move kickoff from C to Go, which we want to do anyhow so that we run the write barriers that it generates. Reviewed-on: https://go-review.googlesource.com/40935 commit c3db34f4efc2d610f74a01dd2ad7775f48889b29 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Tue Apr 11 16:11:26 2017 +0200 runtime: netpoll implementation for AIX Code courtesy of Damien Bergamini from Atos Infogérance. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/40352 commit f5634dff40e53ad9ce61afd67fd07334e3af9d1f Author: Ian Lance Taylor <iant@golang.org> Date: Tue Apr 18 22:06:07 2017 -0700 runtime: move mstart from Go to C The assignments done in mstart must be done without write barriers, as mstart is running without an m or p. In the gc toolchain the equivalent code to intialize g and g->m is written in assembler; on GNU/Linux, it's in the clone function. Reviewed-on: https://go-review.googlesource.com/40989 commit 671d7c74592f4b6fe3665af279482ba0ea47ca2d Author: Ian Lance Taylor <iant@golang.org> Date: Tue Apr 18 17:47:28 2017 -0700 compiler: varargs slices do not escape in runtime Also, don't try to allocate an empty slice on the stack, as it will confuse the GCC backend. Also add a few trivial style, code formatting, and debug output fixes. Updates golang/go#17431 Reviewed-on: https://go-review.googlesource.com/40983 commit 94699d25f31353bf03419eda56b15993a39f3275 Author: Ian Lance Taylor <iant@golang.org> Date: Tue Apr 18 17:30:09 2017 -0700 compiler: add Ptrmask_symbol_expression Add an expression to evaluate to the ptrmask for a type. This will be used for global variables, which always use a ptrmask no matter how large they are. Reviewed-on: https://go-review.googlesource.com/40981 commit bfff1654eac5b9288fa6c431e66cba8c9da6a660 Author: Ian Lance Taylor <iant@golang.org> Date: Mon Apr 17 10:51:16 2017 -0700 runtime: change g's in systemstack The systemstack function in the gc toolchain changes to a different g. This is often used to get more stack space; the gofrontend uses a different stack growth mechanism that does not require changing g's, so we've been running with a version of systemstack that keeps the same g. However, the garbage collector has various tests to verify that it is running on g0 rather than on a normal g. For simplicity, change the gofrontend version of systemstack to change to a different g just as the gc toolchain does. This permits us to uncomment some sanity checks in notetsleep. Doing that requires us to fix up a couple of places where C code calls {start,stop}TheWorldWithSema while not on g0. Note that this does slow down some code in the runtime package unnecessarily. It may be useful to find some places where the runtime calls systemstack only to get more stack space and change it to use some other function. That other function would act like systemstack in the gc toolchain but simply call the argument in the gofrontend. Reviewed-on: https://go-review.googlesource.com/40973 commit b2ccc7601ce71a7c5732154cf9b2eeea64681469 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 10:36:12 2017 -0700 compiler, runtime: include ptrmask in GC roots Change the list of registered GC roots to include a ptrmask, and change the data structures to be easily used from Go code. The new ptrmask will be used by the Go 1.8 GC to only scan pointers. Tweak the current GC to use the new structures, but ignore the new ptrmask information for now. The new GC root data includes the size of the variable. The size is not currently used, but will be used later by the cgo checking code. Reviewed-on: https://go-review.googlesource.com/41075 commit 9e065149970bc180e4ca83bb99c74d9c4f43b47b Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 12:23:16 2017 -0700 compiler, runtime: don't pass size to __go_new There is no reason to pass the size to __go_new, as the type descriptor includes the size anyhow. This makes the function correspond to the Go 1.8 function runtime.newobject, which is what we will use when we update to the Go 1.8 memory allocator. Reviewed-on: https://go-review.googlesource.com/41080 commit c321de7b738c4a3387c1842919c9305acfa04c57 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 13:13:56 2017 -0700 compiler, runtime, reflect: make type descriptors more like Go 1.8 Change the type descriptor structure to be more like the one in the Go 1.8 runtime. Specifically we add the ptrdata field, rename the gc field to gcdata and change the type to *byte, and rearrange a few of the fields. The structure is still not identical to the Go 1.8 structure--we don't use any of the tricks to reduce overall executable size--but it is more similar. For now we don't use the new ptrdata field, and the gcdata field is still the old format rather than the new Go 1.8 ptrmask/gcprog format. Reviewed-on: https://go-review.googlesource.com/41081 commit 7b70c52cddeebea9ebeac003f8c6aad59497e5f0 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 14:54:29 2017 -0700 reflect: make sure to clear unusable hash/equal function Otherwise we wind up copying the one from the prototype, which is wrong. Also rewrite the hash/equal functions to look like the ones in Go 1.8, mainly a matter of changing names and using arrayAt. Reviewed-on: https://go-review.googlesource.com/41133 commit 84d26f467f7de8bdbb0d230458135fe1b6b2a99d Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 14:59:13 2017 -0700 runtime: remove duplicate declarations of SetFinalizer/KeepAlive These should have been removed in CL 38794. It's a bug that the compiler even permits these duplicate declarations. Reviewed-on: https://go-review.googlesource.com/41134 commit f85ff7e64c24031f6d0bd7c9c426b6176cb95160 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 15:56:32 2017 -0700 runtime: don't crash if panicstring called with no m It's possible for runtime_panicstring to be called with no m if a signal handler, or scheduler innards, do something wrong. If that happens carry on with the panic rather than crashing. Reviewed-on: https://go-review.googlesource.com/41137 commit 5b362b04f642afb8b20715930416fc3b7d91bb12 Author: Than McIntosh <thanm@google.com> Date: Fri Mar 31 14:35:48 2017 -0400 compiler: fix for expr sharing introduced by Order_eval::statement. When processing an expression statement with a top-level call that returns multiple results, Order_eval::statement can wind up creating a tree that has multiple references to the same call, which results in a confusing AST dump. Change the implementation to avoid introducing this unwanted sharing. Reviewed-on: https://go-review.googlesource.com/39210 commit b05b4260a68695bf9c9cc29e14ae86ca2699458a Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 16:00:28 2017 -0700 runtime: restore correct m in gtraceback If gtraceback is used to get a stack trace of a g running in the same m, as can happen if we collect a stack trace from a g0, then restore the old m value, don't clear it. Reviewed-on: https://go-review.googlesource.com/41138 commit ca8bbf4dfac19b3f4f7ce21a688b96a418c75031 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 16:03:24 2017 -0700 runtime: set startpc field when starting a new goroutine This puts the right value in a trace--previously it was always zero. Reviewed-on: https://go-review.googlesource.com/41139 commit ca8bbf4dfac19b3f4f7ce21a688b96a418c75031 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 16:03:24 2017 -0700 runtime: set startpc field when starting a new goroutine This puts the right value in a trace--previously it was always zero. Reviewed-on: https://go-review.googlesource.com/41139 commit 887690dce42d7bf8f711f8ea082e4928fb70f2a5 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 17:06:11 2017 -0700 runtime: add prefetch functions The Go 1.8 GC uses prefetch functions. Add versions for gccgo that call __builtin_prefetch. Uncomment the test for them in testAtomic64. Don't force the check function to return early, as now taking the address of a local variable in the runtime package does not force it onto the heap. Reviewed-on: https://go-review.googlesource.com/41144 commit 4269db69f9184e5a45c54aaee7352425a1f88bff Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 17:55:21 2017 -0700 runtime: split up ticks to get correct alignment On 32-bit x86 a uint64 variable by itself is aligned to an 8-byte boundary. A uint64 field in a struct is aligned to a 4-byte boundary. The runtime.ticks variable has a uint64 field that must be aligned to an 8-byte boundary. Rather than rely on luck, split up the struct into separate vars so that the required alignment happens reliably. It would be much nicer if issue golang/go#19057 were fixed somehow, but that is for another day. Reviewed-on: https://go-review.googlesource.com/41143 commit 66926cabdbdbf3431b4f172f7756e195c1c6c513 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Apr 20 17:15:38 2017 +0200 libgo: fix bad value for O_CLOEXEC on AIX 7.1 On AIX 7.1, O_CLOEXEC is defined as 0x0000001000000000, which creates an integer constant overflow error when building libgo. This affects only 7.1, O_CLOEXEC is not defined on 6.1 (and defaults to O in sysinfo.go) and is defined as 0x00800000 on AIX 7.2. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/41214 commit af288ff10aeafc47651f5def327ed56425d5be19 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Apr 20 17:15:02 2017 -0700 runtime: preserve stack context in tracebackothers The tracebackothers function works by saving the current stack context in the goroutine's context field and then calling gogo to switch to a new goroutine. The new goroutine will collect its own stack trace and then call gogo to switch back to the original goroutine. This works fine, but if the original goroutine was called by mcall then the contents of its context field are needed to return from the mcall. Fix this by saving the stack context across the calls to the other goroutines. Reviewed-on: https://go-review.googlesource.com/41293 commit 43101e5956e793f1b4de05c15d7738c785e927df Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Fri Apr 21 10:58:52 2017 +0200 os/user: use _posix_* libc functions libc getpwnam_r function has a different signature, we must use _posix_getpwnam_r instead (by default, the pwd.h system include file defines getpwnam_r as a static function calling _posix_getpwnam_r, so a C program calling getpwnam_r will indeed reference the _posix_getpwnam_r symbol). Idem for getpwuid_r, getgrnam_r and getgrgid_r. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/41334 commit 71e1fec4d2a536591ea6657a06916a17b5127071 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 21:24:48 2017 -0700 runtime: don't use pointers in g_ucontext_t or stackcontext The g_ucontext_t type holds registers saved for a goroutine. We have to scan it for pointers, but since registers don't necessarily hold pointers we have to scan it conservatively. That means that it should not have a pointer type, since the GC will always scan pointers. Instead it needs special treatment to be scanned conservatively. The current GC doesn't care when a pointer type holds a non-pointer, but the Go 1.8 GC does. For the current GC this means we have to explicitly scan the g_ucontext_t values in a G. While we're at it change stackcontext to be uintptr too. The entries in stackcontext never hold pointers that the Go GC cares about. Reviewed-on: https://go-review.googlesource.com/41270 commit eab2960aee91d3e3a6baa5b1bce01262d24c714f Author: Ian Lance Taylor <iant@golang.org> Date: Thu Apr 20 17:08:19 2017 -0700 runtime/internal/sys: define Goexperiment The gc toolchain defines Goexperiment based on the environment variable GOEXPERIMENT when the toolchain is built. We just always set Goexperiment to the empty string. Reviewed-on: https://go-review.googlesource.com/41292 commit be4a751943265c0637da859d15a4faf162f5c478 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Apr 20 14:04:35 2017 +0200 net: sockopt implementation for AIX This is a copy of the Linux implementation, it allows to run some simple client/server applications on AIX, while the current sockopt stubs don't. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/41213 commit 46a669c4ca5b80fd6f6a0a42095804d9f704611d Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 29 17:55:06 2017 +0200 math: fix sign for atan/expm1/log1p(-0) AIX libc returns +0 for atan(-0), expm1(-0) and log1p(-0), while matching Go functions must return -0. Code courtesy of Tony Reix. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38699 commit 53b0e809130038a46f0a3d2870e3905f44ab888d Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Apr 26 17:29:22 2017 +0200 runtime: fix context clobbering on AIX On AIX 64-bits, r13 is a pointer to thread data. setcontext() overwrites r13 with the value saved by getcontext(). So, when a goroutine is scheduled on a new thread, r13 will point to the old thread data after calling setcontext(). Code courtesy of Damien Bergamini. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/41854 commit f8d5ebd71c71e6e777200530d8204b92619157f8 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Apr 26 18:01:19 2017 +0200 runtime: fix wrong time calculation in semasleep tv_nsec is added twice when calculating the sleep end time. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/41855 commit ef56097f4ea848d48fbf61eba1c757fe7fce99d3 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Fri Apr 28 10:27:32 2017 +0200 libgo: pass $(NM) value when running benchmarks On AIX, we need to use "nm -B" instead of "nm", to have the epxected output format, so the configured $(NM) value from the Makefile should be exported before running gotest, which defaults to "nm" if $NM is not set. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/42051 commit 0fb550083ae474fb964435927b899ec8e4b62771 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Nov 16 21:12:53 2016 -0800 runtime: copy garbage collector from Go 1.8 runtime This giant patch replaces the old Go 1.4 memory allocator and garbage collector with the new Go 1.8 code. The memory allocator is fairly similar, though now written in Go rather than C. The garbage collector is completely different. It now uses ptrmask and gcprog information, which requires changes in the compiler and the reflect package as well as the runtime. And, of course, the garbage collector now runs concurrently with program execution. In the gc toolchain the garbage collector is strict and precise at all levels. In the gofrontend we do not have stack maps, so stacks, and register values, are collected conservatively. That means that an old, no longer used, pointer on a stack or in a register can cause a memory object to live longer than it should. That in turns means that we must disable some checks for invalid pointers in the garbage collection code. Not only can we get an invalid pointer on the stack; the concurrent nature of the collector means that we can in effect resurrect a block that was already unmarked but that the collector had not yet gotten around to freeing, and that block can in turn point to other blocks that the collector had managed to already free. So we must disable pointer checks in general. In effect we are relying on the fact that the strict pointer checks in the gc toolchain ensure that the garbage collector is correct, and we just assume that it is correct for the gofrontend since we are using the same code. Reviewed-on: https://go-review.googlesource.com/41307 commit a95078d501175240d095500a8c5fbfb21bec65cb Author: Ian Lance Taylor <iant@golang.org> Date: Mon Apr 24 16:33:47 2017 -0700 libgo/Makefile: clean more files Fix up the mostlyclean, clean, and distclean targets to better follow https://www.gnu.org/prep/standards/html_node/Standard-Targets.html. Reviewed-on: https://go-review.googlesource.com/41625 commit 5956bf1055451cf4239cdfeca259c23b1ded54d8 Author: Ian Lance Taylor <iant@golang.org> Date: Mon May 8 13:35:11 2017 -0700 libgo: delete goc2c The last .goc file has been removed, so remove goc2c. The goc2c program was my first contribution to the gc repository that was more than 100 lines: golang/go@2b57a11 The program was used in gc for a few years under various guises but was finally removed in https://golang.org/cl/132680043. Now we can remove it from gofrontend as well. Reviewed-on: https://go-review.googlesource.com/42911 commit a222e35d041de0cd42506b61c93b8209e07702b9 Author: Than McIntosh <thanm@google.com> Date: Tue May 9 10:33:10 2017 -0400 compiler: set "need_init_fn" when adding gc root Variables that back slice initializers in certain cases have to be added to the gc roots list, since they can be modified at runtime. The code that was doing this addition did not update the flag that tracks whether the package being compiled needs an initializer function, which resulted in the call in question being left out of the final generated code in certain cases. Fix is to change Gogo::add_gc_root() to update the "needs init" flag. Reviewed-on: https://go-review.googlesource.com/43030 commit 822ab419bf7d1c705cdce1c12133e7a11f56be2e Author: Than McIntosh <thanm@google.com> Date: Tue May 9 11:36:51 2017 -0400 compiler: fix variable context nit in write barrier generation Update the write barrier generation code to insure that the "lvalue context" tag on the space var expression is set only in the case where the expr feeds directly into an assignment. This is somewhat counter-intuitive, but needed in the case where the backend looks at context tags. Reviewed-on: https://go-review.googlesource.com/43031 git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@247848 138bc75d-0d04-0410-961f-82ee72b054a4

ugorji · 2017-12-26T04:18:01Z

My concern with the language changes are that it makes it hard to have source code that compiles in older versions of go. The benefit of a comment (e.g. //go:align cache) is that it can be ignored by go versions which cannot interpret it. I currently have libraries that compile as far back as go 1.4 (github.com/ugorji/go/codec). I would like to leverage a better cache-line alignment model that the hacked (_ [N]byte // padding) i have all around the place.

aclements · 2018-10-09T18:49:39Z

@dvyukov points out in https://go-review.googlesource.com/c/go/+/138076/3/src/runtime/mheap.go#146 that we've in fact broken cache line padding in the runtime for various arrays because only align the size of the array element to a multiple of the cache line size, but have no guarantee that it will start on a cache line. As a result, neighboring elements can alias to the same cache line. Currently we can only solve this by adding a full cache line's worth of padding between each element, which is wasteful and can needlessly split elements across cache lines. Having a way to indicate that these arrays must be cache-line aligned would be a much better solution.

markdryan · 2019-04-29T10:57:11Z

One motivation for taking another look at this issue could be the assembler's support of AVX-512 added in Go 1.11. AVX-512 code operating on 512 bit registers works best when the data on which it operates is 64 byte aligned, as each unaligned access is a cache-line split. As there doesn't currently seem to be a way to specify the alignment of slices and arrays, it's difficult to take full advantage of AVX-512 in Go without relying on a custom allocator or adding potentially unnecessary peeling code to the AVX-512 algorithm to handle leading unaligned data.

To test this out I created a simple AVX-512 function in Go assembler that adds one array of 32 bit integers to another. The function naively assumes that the input and output arrays are the same size and that this size is divisible by 16.

//func sum(a []int32, b []int32)	
TEXT ·sum(SB), NOSPLIT, $0
   	MOVQ a_base+0(FP), SI
	MOVQ a_len+8(FP), DX
	MOVQ b_base+24(FP), DI

loop1:
	VMOVDQU32 -64(SI)(DX * 4), Z25
	VMOVDQU32 -64(DI)(DX * 4), Z26
	VPADDD Z25, Z26, Z25
	VMOVDQU32 Z25, -64(SI)(DX * 4)
	SUBQ $16, DX	
	JNE loop1
	RET

I then created aligned and unaligned versions of a benchmark to test this code. The unaligned version looked something like this (I've edited out some initialisation code)

//go:noescape
func sum(in []int32, out []int32)

type container struct {
	in  [10 * 1024]int32
	out [10 * 1024]int32
}

var c container

func BenchmarkAlign(b *testing.B) {
	for n := 0; n < b.N; n++ {
		sum(c.in[:], c.out[:])
	}
}

In my first test, c.in and c.out happened to be 32 byte and not 64 byte aligned. I forced them to be 64 byte aligned by preceding them with a _ [32]int8, e.g.,

type container struct {
        _   [32]int8
	in  [10 * 1024]int32
	out [10 * 1024]int32
}

and re-ran the benchmark. Doing so improved the speed of the benchmark by 1.33x on my i9-7900X.

Of course, there's no guarantee that the addition of a 32 byte array to the start of a structure will always provide 64 byte alignment for the subsequent element. Therefore, it would be much nicer if I could write

type container struct {
        _ runtime.Aligned64
	in  [10 * 1024]int32
	out [10 * 1024]int32
}

Note I was also able to get the structure to be 64 byte aligned by simply deleting a fmt.Printf debug statement in the initialisation code. Doing so removed the fmt import, changed the alignment of c, and improved the speed of my benchmark by 33%.

seebs · 2019-08-19T15:10:08Z

This is probably an awful idea, but what about struct tags?

type fussy struct {
    dontCare uint64
    alignMe uint64 `align:32`
}

A naive analysis suggests that this struct would necessarily have a size of 64 bytes. If the struct's total size is not a multiple of 32 bytes, then an array of these would meet the alignment requirement only sometimes. If the offset of alignMe in the struct is not a multiple of 32 bytes, then it won't be aligned unless the struct itself is specifically not aligned.

... Now that I've written it down, I vote against it. I think it's more Go-like to require that the padding be explicit, using _ members.

Possible idiom:

type runtime.Aligned struct{}
type fussy struct {
    dontCare uint64
    _ [32]runtime.Aligned
    alignMe uint64
}

(and if you do [17]runtime.Aligned, the compiler comes after you with a lead pipe.)

iand · 2019-08-19T16:29:47Z

What about allowing tags to be applied to structs as well as fields? These could then also be surfaced via reflect which may be useful.

We would then have something like:

type WaitGroup struct `runtime:"nocopy,align64"` {
    state uint64
    sema uint32
}

gopherbot · 2020-06-12T20:57:47Z

Change https://golang.org/cl/237737 mentions this issue: syscall: add Get/Set methods to Stat_t.Size, Flock_t.{Start,Len}

gopherbot · 2021-04-09T22:00:03Z

Change https://golang.org/cl/308971 mentions this issue: cmd/compile: add internal/align package for runtime

rogpeppe · 2021-04-14T11:53:06Z

One other thought that's probably horrible, but I'll leave it here anyway: how about making runtime.Aligned work by using the size of the array that it's part of (Aligned would be zero sized) ?

type WaitGroup struct {
    _ [8]runtime.Aligned
    state uint64
    sema uint32
}

That allows any alignment to be specified without introducing a zillion new types.

Edit: I see that's pretty similar to this.

For the [17]runtime.Aligned case, maybe it's reasonable to give the compiler the freedom to arbitrarily increase the alignment if it's not compatible with hardware constraints. So on a machine that didn't allow unaligned word access, it could round up to the nearest word-size multiple.

mdempsky · 2021-04-16T22:09:31Z

how about making runtime.Aligned work by using the size of the array that it's part of (Aligned would be zero sized) ?

This is how internal/align.elemT works in https://golang.org/cl/308971.

GCC release freeze. * go-backend.c: Include "go-c.h". * go-gcc.cc (Gcc_backend::write_export_data): New method. * go-gcc.cc (Gcc_backend::Gcc_backend): Declare __builtin_prefetch. * Make-lang.in (GO_OBJS): Add go/wb.o. commit 884c9f2cafb3fc1decaca70f1817ae269e4c6889 Author: Than McIntosh <thanm@google.com> Date: Mon Jan 23 15:07:07 2017 -0500 compiler: insert additional conversion for type desc ptr expr Change the method Type::type_descriptor_pointer to apply an additional type conversion to its result Bexpression, to avoid type clashes in the back end. The backend expression for a given type descriptor var is given a type of "_type", however the virtual calls that create the variable use types derived from _type, hence the need to force a conversion. Reviewed-on: https://go-review.googlesource.com/35506 commit 5f0647c71e3b29eddcd0eecc44e7ba44ae7fc8dd Author: Than McIntosh <thanm@google.com> Date: Mon Jan 23 15:22:26 2017 -0500 compiler: insure tree integrity in Call_expression::set_result Depending on the back end, it can be problematic to reuse Bexpressions (passing the same Bexpression to more than one Backend call to create additional Bexpressions or Bstatements). The Call_expression::set_result method was reusing its Bexpression input in more than one tree context; the fix is to pass in an Expression instead and generate multiple Bexpression references to it within the method. Reviewed-on: https://go-review.googlesource.com/35505 commit 7a8e49870885af898c3c790275e513d1764a2828 Author: Ian Lance Taylor <iant@golang.org> Date: Tue Jan 24 21:19:06 2017 -0800 runtime: copy more of the scheduler from the Go 1.8 runtime Copies mstart, newm, m0, g0, and friends. Reviewed-on: https://go-review.googlesource.com/35645 commit 3546e2f002d0277d805ec59c5403bc1d4eda4ed9 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Jan 26 19:47:37 2017 -0800 runtime: remove a few C functions that are no longer used Reviewed-on: https://go-review.googlesource.com/35849 commit a71b835254f6d3164a0e6beaf54f2b175d1a6a92 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Jan 26 16:51:16 2017 -0800 runtime: copy over more of the Go 1.8 scheduler In particular __go_go (aka newproc) and goexit[01]. Reviewed-on: https://go-review.googlesource.com/35847 commit c3ffff725adbe54d8283c373b6aa7dc95d6fc27f Author: Ian Lance Taylor <iant@golang.org> Date: Fri Jan 27 16:58:20 2017 -0800 runtime: copy syscall handling from Go 1.8 runtime Entering a syscall still has to start in C, to save the registers. Fix entersyscallblock to save them more reliably. This copies over the tracing code for syscalls, which we previously weren't doing, and lets us turn on runtime/trace/check. Reviewed-on: https://go-review.googlesource.com/35912 commit d5b921de4a28b04000fc4c8dac7f529a4a624dfc Author: Ian Lance Taylor <iant@golang.org> Date: Fri Jan 27 18:34:11 2017 -0800 runtime: copy SIGPROF handling from Go 1.8 runtime Also copy over Breakpoint. Fix Func.Name and Func.Entry to not crash on a nil Func. Reviewed-on: https://go-review.googlesource.com/35913 commit cc60235e55aef14b15c3d2114030245beb3adfef Author: Than McIntosh <thanm@google.com> Date: Mon Feb 6 11:12:12 2017 -0500 compiler: convert go_write_export_data to Backend method. Convert the helper function 'go_write_export_data' into a Backend class method, to allow for an implementation of this function that needs to access backend state. Reviewed-on: https://go-review.googlesource.com/36357 commit e387439bfd24d5e142874b8e68e7039f74c744d7 Author: Than McIntosh <thanm@google.com> Date: Wed Feb 8 11:13:46 2017 -0500 compiler: insert backend conversion in temporary statement init Insert an additional type conversion in Temporary_statement::do_get_backend when assigning a Bexpression initializer to the temporary variable, to avoid potential clashes in the back end. This can come up when assigning something of concrete pointer-to-function type to a variable of generic pointer-to-function type. Reviewed-on: https://go-review.googlesource.com/36591 commit c5acf0ce09e61ff623847a35a99da465b8571609 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 1 17:57:53 2017 +0100 libgo: build tags for aix Build tags for the libgo source files required to build libgo on AIX. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37633 commit 67ed19616898ea18a101ec9325b82d028cd395d9 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 2 15:41:31 2017 +0100 libgo: handle AIX tag in match.sh and gotest Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37638 commit 83ea2d694c10b2dd83fc8620c43da13d20db754e Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 1 17:48:16 2017 +0100 libgo: add AIX support in configure and Makefile - support for GOOS=aix - CFLAGS/GOCFLAGS/LDFLAGS for AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37632 commit 35d577fe22ffa16a3ccaadf5dae9f6f425c8ec8c Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Mon Mar 6 15:00:15 2017 +0100 runtime: adapt memory management to AIX mmap On AIX: * mmap does not allow to map an already mapped range, * mmap range start at 0x30000000 for 32 bits processes, * mmap range start at 0x70000000_00000000 for 64 bits processes Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37845 commit 4e49e56a5fd4072b4ca7fcefe4158d6885d9ee62 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Mon Mar 6 13:42:26 2017 +0100 runtime: add getproccount implementation for AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37844 commit ff626470294237ac664127894826614edc46a3d0 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Mon Mar 6 17:31:21 2017 +0100 runtime: handle ERESTART errno with AIX's wait4 On AIX, wait4 may return with errno set to ERESTART, which causes unexepected behavior (for instance, go build may exit with the message "wait: restart system call" after running a command, even if it was successfull). Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37846 commit 37daabbfc83d533b826ef9ab10e2dee7406e7198 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Mon Mar 6 11:02:58 2017 +0100 runtime: support for AIX's procfs tree On AIX, the process executable file is available under /proc/<pid>/object/a.out Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37842 commit a0275c039d56acf4bf48151978c1a4ec5758cc2c Author: Ian Lance Taylor <iant@golang.org> Date: Wed Mar 8 07:00:05 2017 -0800 libgo/Makefile.am: don't use nonportable \n or \t in sed expression The resulting zstdpktlist.go is less pretty, but it works. Reviewed-on: https://go-review.googlesource.com/37940 commit 29b190f76105aafa2b50b48249afdafecc97a4be Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 9 16:02:34 2017 +0100 runtime: netpoll and semaphores for AIX semaphore implementation based on Solaris implementation in libgo/go/runtime/os_solaris.go netpoll is just a stub to avoid build failure on AIX. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37966 commit 55ca6d3f3cddf0ff9ccb074b2694da9fc54de7ec Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 9 15:38:30 2017 +0100 libmain: ensure initfn is called when loading a go library AIX does not support .init_array. The alterative is to export the __go_init function and tell the linker it is an init function with the -Wl,-binitfini:__go_init option. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37965 commit 349a30d17d880ac8bc1a35e1a2ffee6d6e870ae9 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Fri Mar 10 11:15:08 2017 +0100 libgo: use an import list for missing symbols libgo depends on symbols provided by Go programs at runtime. On AIX, this requires either to build libgo with -Wl,-berok linker option and the programs with -Wl,-brtl, or to provide a list of imported symbols when building libgo. The second options seems preferable, to avoid requiring an additional option for every Go program. There are also some symbols that are specific to GNU ld and do not exist when linking with AIX ld (__data_start, __edata, __etext and __bss_start). Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37969 commit 91db0ea1ff068ca1d97b9c99612100ea5b96ddb2 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 8 15:34:45 2017 +0100 crypto/x509: add certificate files locations for AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/37952 commit 92e521c854e91709b949548c47e267377850f26a Author: Ian Lance Taylor <iant@golang.org> Date: Fri Mar 10 14:10:11 2017 -0800 compiler: fix check for pointer in Temporary_reference_expression The check for an unrepresentable pointer in Temporary_reference_expression::do_get_backend was incorrectly translated from C to Go in https://golang.org/cl/14346043. Fix the check to use points_to rather than has_pointer and deref. This should not make any difference in practice as either way the condition will only be true for a pointer to void, but points_to is correct and more efficient. Reviewed-on: https://go-review.googlesource.com/38009 commit 9a0b676e59e7171a630c48fdc3d4de6712bad0ca Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 16 16:51:53 2017 +0100 libgo: add missing _arpcom struct to *sysinfo.go This struct is filtered due to having a field of type _in6_addr, but other types exported to *sysinfo.go are depending on it. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38251 commit 61262a757bdd3d9a595ab6a90f68c0c4ebed7bc1 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 16 18:27:46 2017 +0100 syscall: raw_ptrace stub for AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38253 commit 8029632b50880fd9b5e39299c738b38e3386595f Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 15 16:58:37 2017 +0100 libgo: adapt runtime.inc to AIX * Two AIX types are wrongfully exported to runtime.inc as their names make them look like a Go type. * The sigset go type conflicts with a system sigset type. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38192 commit 25f3a90d14bc268479369ecc0eada72791612f86 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 15 16:58:37 2017 +0100 libgo: update Makefile.in, accidentally omitted from last change Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38310 commit d52b4895616b66f93b460366527e74336829aaa5 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 16 18:39:26 2017 +0100 syscall: TIOCSCTTY does not exist on AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38254 commit ff1ec3847a4472008e5d53a98b6694b1e54ca322 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 16 18:07:34 2017 +0100 syscall: syscall does not exist on AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38252 commit c1ee60dabf0b243a0b0286215481a5d326c34596 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Fri Mar 17 17:18:18 2017 +0100 net: EAI_OVERFLOW does not exist on AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38266 commit ad4ad29aed9f70b14b39b488bfeb9ee745382ec4 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Fri Mar 17 17:23:56 2017 +0100 net: sockopt/sockoptip stubs for AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38267 commit 5d7db2d7542fe7082f426d42f8c2ce14aad6df55 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Fri Mar 17 16:35:05 2017 +0100 os/user: add listgroups stub for AIX This is required to build os/user. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38263 commit 4e57a7973e9fa4cb5ab977c6d792e62a8f7c5795 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 22 11:11:30 2017 +0100 os: fix readdirnames for AIX Largefile implementation should be used on AIX. readdir64_r function returns 9 and sets result to NULL when reaching end of directory, so this return code should not always be considered as an error. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38359 commit b34036967d1ec57b25e3debe077439b4210a1d4a Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Fri Mar 17 17:39:31 2017 +0100 libgo: adapt sigtab.go to AIX On AIX, _NSIG is not directly defined to its integer value in gen-sysinfo.go. The real value is _SIGMAX32+1 or _SIGMAX64+1, depending if we are building a 32bit ligbo or a 64bit libgo, so we need to read one of those constants to set nsig value in mksigtab.sh This change also ensures that all signal numbers from 0 to nsig-1 are referenced in sigtable. Reviewed-on: https://go-review.googlesource.com/38268 commit 20991c32671a183ec859b4f285df37fdd4634247 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 23 17:28:09 2017 +0100 syscall: missing import in socket_bsd.go Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38369 commit c34754bd9adf5496c4c26257eaa50793553c11e8 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 22 17:57:01 2017 +0100 sycall: WCOREDUMP macro is not defined on AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38363 commit 4f38813482227b12ea0ac6ac1b981ff9ef9853ef Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 23 17:44:43 2017 +0100 libgo: additional build tags for AIX Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38510 commit d117ede6ff5a7083e9c40eba28a0f94f3535d773 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 23 17:48:46 2017 +0100 go/build: add AIX to "go build" command known OS Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38511 commit 7b0ddaa6a6a71f9eb1c374122d29775b13c2cac5 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Mar 23 09:57:01 2017 -0700 compiler: don't crash if imported package imports this one When building a test it's OK if test code imports a package that imports this one. The go tool is supposed to catch cases where this creates an impossible initialization order. The compiler already has code to permit this in Gogo::add_import_init_fn. This CL avoids a compiler crash on a similar case when writing out the export data. I have no test case for this. Basically it pushes a compiler crash into an error reported elsewhere. Problem was reported by Tony Reix. Reviewed-on: https://go-review.googlesource.com/38462 commit 925636975d075e3e3353823b09db3f933f23cb03 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Mar 29 14:14:18 2017 -0700 runtime: copy finalizer support from Go 1.8 runtime Reviewed-on: https://go-review.googlesource.com/38794 commit 1ccb22b96cb3b1011db0e427877d9ddecb577fa9 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Mar 30 15:21:06 2017 +0200 runtime: initcontext and setcontext stubs for AIX Further investigations are required to understand the clobbering issue and implement a proper fix. Until then, those stubs are required to allow the build to complete. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38930 commit 27db481f369b54256063c72b911d22390c59199c Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 29 18:07:25 2017 +0200 os: fix Readlink failure on AIX AIX readlink routine returns an error if the link is longer than the buffer, instead of truncating the link. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38700 commit c93babbf48eddd0bc34d4179ffb302dc60087299 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 29 17:26:35 2017 +0200 compiler: implement support for reading AIX big archives This is required to read go export from a Go library. Code courtesy of Damien Bergamini from Atos Infogérance. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38698 commit 930dd53482bdee3a9074850d168d0b9d7819c135 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Apr 6 18:50:11 2017 -0700 compiler: fix whether conversions are static initializers The compiler was incorrectly treating type conversions from string to int or vice-versa as static initializers. That doesn't work, as those conversions are implemented via a function call. This case may never actually arise but it seems like the right thing to do. Reviewed-on: https://go-review.googlesource.com/39872 commit f02691e4195728dbf06f4dde0853c6bccc922183 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Apr 6 17:24:08 2017 -0700 compiler, runtime: don't let slices point past end of memory block When Go code uses a slice expression like [:len(str)] or [:cap(slice)], it's natural for the resulting pointer to point just past the end of the memory block. If the next memory block is not live, we now have a live pointer to a dead block, which will unnecessarily keep the block alive. That wastes space, and with the new Go 1.8 GC (not yet committed) will trigger an error when using GODEBUG=gccheckmark=1. This changes the implementation of slice expressions to not move the pointer if the resulting string length or slice capacity is 0. When the length/capacity is zero, the pointer is never used anyhow. Reviewed-on: https://go-review.googlesource.com/39870 commit 17527c35b027e1afcc318faf5563909e1e9d44a6 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Apr 6 15:30:11 2017 -0700 compiler: emit write barriers The Go 1.8 concurrent GC requires optional write barriers for all assignments that may change pointer values in the heap or in a global variable. For details see https://blog.golang.org/go15gc. This changes the gofrontend code to emit write barriers as needed. This is in preparation for future changes. At the moment the write barriers will do nothing. They test runtime.writeBarrier.enabled, which will never be non-zero. They call simple functions which just do a move without doing any of the other operations required by the write barrier. Reviewed-on: https://go-review.googlesource.com/39852 commit c0b00f072bf34b2c288e1271ec8118b88c4f6f6f Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Tue Apr 11 17:47:29 2017 +0200 libgo: allow building gox files from PIC objects libtool builds non-PIC objects in the same directory as .lo files and PIC objects in a .libs subdirectory. BUILDGOX rule uses the non-PIC objects to build the gox files, but on AIX only the PIC objects are built. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/40355 commit ea0f3da174c5503a209043f14ddda34871cfec52 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Apr 6 19:06:14 2017 -0700 compiler: add code to generate a ptrmask for a type The Go 1.8 garbage collector uses a ptrmask for all types below a certain size. A ptrmask is simply a bit vector with a single bit for each pointer-sized word in the value. The bit is 1 if the type has a pointer in that position, 0 if it does not. This change adds code to the compiler to generate a ptrmask. The code is not used by anything yet, it is just compiled. It will be used when we switch over to the Go 1.8 garbage collector. The new Array_type::int_length method, and the new memory_size methods, will also be used by other patches coming later. Reviewed-on: https://go-review.googlesource.com/39873 commit 3029e1df3be3614d196a03c15e50e68ff850aa4c Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 7 10:31:39 2017 -0700 compiler: add code to generate a gcprog for a type The Go 1.8 garbage collector uses a gcprog for all types above a certain size. A gcprog describes where the pointers are in the type, using a simple bytecode machine that supports repeating bits. The effect is to permit using much less space to describe arrays. The format is described in runtime/mbitmap.go in the docs for runGCProg. This is not yet added to the gofrontend, but can be seen in the gc sources. This change adds code to the compiler to generate a gcprog. The code is not used by anything yet, it is just compiled. It will be used when we switch over to the Go 1.8 garbage collector. Reviewed-on: https://go-review.googlesource.com/39923 commit 8b01ef1e9176d20f4c9e667972fe031069a4d057 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Apr 13 07:00:35 2017 -0700 compiler: add ptrdata computations and expressions For the upcoming Go 1.8 GC we need to compute the "ptrdata" of a type: the number of bytes in the type that can contain pointers. For types that do not contain pointers this number is zero. For many types it is a number larger than zero but smaller than the total size of the type. The Go 1.8 GC uses this number to make loops looking for pointers run faster by not scanning the suffix of a value that can not contain a pointer. Unfortunately there are two subtly different definitions of ptrdata, and we need both. The first is the simple one: the prefix that can contain pointers. The second is the number of bytes described by the gcprog for the type. Recall that we describe the exact position of pointers in a type using either a ptrmask or a gcprog. The ptrmask is simpler, the gcprog uses less space. We use the gcprog for large types, currently defined as types that are more than 2048 bytes. When the Go 1.8 runtime expands a gcprog, it verifies that the gcprog describes exactly the same number of bytes as the ptrdata field in the type descriptor. If the last pointer-containing portion of a type is an array, and if the elements of the array have a ptrdata that is less than the size of the element type, then the simple definition of the ptrdata will not include the final non-pointer-containing bytes of the last element of the array. However, the gcprog will define the array using a repeat count, and will therefore include the full size of the last element of the array. So for a type that needs a gcprog, the ptrdata field in the type descriptor must be the size of the data described by the gcprog, and that is not necessarily the same as the simple ptrdata. It might seem that we can always use the gcprog version of the ptrdata calculation, since that is what will appear in a type descriptor, but it turns out that for global variables we always use a ptrmask, not a gcprog, even if the global variable is large. This is because gcprogs are handled by expanding them into a ptrmask at runtime, and for a global variable there is no natural place to put the ptrmask. Simpler to always use the ptrmask. That means that we need to describe the size of the ptrmask, and that means that we need an expression for the simple form of the ptrdata. This CL implements the ptrdata calculation. This code is not actually used yet. It will be used later when the Go 1.8 GC is committed. Reviewed-on: https://go-review.googlesource.com/40573 commit 7a37331303b572412179a08141f1dd35339d40c8 Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 14 06:55:48 2017 -0700 compiler: zero length arrays never contain pointers Reviewed-on: https://go-review.googlesource.com/40696 commit c242f0508a64d3d74a28d498cbaeda785ff76258 Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 14 07:26:54 2017 -0700 bytes: disable allocations test on gccgo It turns out that testing.AllocsPerRun has not been producing correct results with the current gccgo memory allocator. When we update to the Go 1.8 memory allocator, testing.AllocsPerRun will work again, and this test will fail due to lack of escape analysis. Reviewed-on: https://go-review.googlesource.com/40700 commit 0dc369f1d63376a36bfb0999a1b0377fd444bfab Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Tue Apr 11 16:22:38 2017 +0200 os: alternative way to find executable path, using Args[0] AIX does not provide a proper way to find the original executable path from procfs, which contains just an hardlink. Executable path can be found using Args[0], Getcwd and $PATH. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/40353 commit f9bad1342569b338e3b2ea9f12ffc6d3d3fa3028 Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 14 08:01:19 2017 -0700 compiler: don't write struct with multiple sink fields to C header file When writing a struct to the C header file used by the C runtime code, a single sink field is fine: it will be called "_", which is valid C. There are structs with single sink fields that we want to write out, such as finblock. As it happens, though, the Go 1.8 runtime has a struct with two sink fields, gcControllerState, which will produce a C definition with two fields named "_", which will fail. Since we don't need to know that struct in C, rather than fix the general case, just punt if the struct has multiple sink fields. After the conversion to the Go 1.8 GC, we may be able to get rid of the C header file anyhow. I'm not sure yet. Reviewed-on: https://go-review.googlesource.com/40701 commit cfc28901a572aeb15b2f10a38f79eec04c64dfb2 Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 14 10:07:23 2017 -0700 runtime: disable allocations test on gccgo It turns out that testing.AllocsPerRun has not been producing correct results with the current gccgo memory allocator. When we update to the Go 1.8 memory allocator, testing.AllocsPerRun will work again, and these tests will fail due to lack of escape analysis. Reviewed-on: https://go-review.googlesource.com/40703 commit 36fedd76edaa48b9ec09709a70d9e4abaddf0caf Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 14 10:47:06 2017 -0700 runtime: remove unused size argument from hash/equal fns The size argument was removed from hash and equal functions in CL 34983. Somehow I missed removing them from three of the predefined functions. Reviewed-on: https://go-review.googlesource.com/40770 commit 90f6accb48d2e78cad8955b9292933f6ce3fe4c8 Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 14 13:23:05 2017 -0700 runtime: remove unused stack.go We're never going to use stack.go for gccgo. Although a build tag keeps it from being built, even having it around can be confusing. Remove it. Reviewed-on: https://go-review.googlesource.com/40774 commit befa71603fc66a214e01ac219f2bba36e19f136f Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 14 13:18:34 2017 -0700 runtime: build fastlog Take out the build tags which were preventing fastlog2 from being built. It's used by the upcoming Go 1.8 GC. Reviewed-on: https://go-review.googlesource.com/40773 commit b7e19e9be4ab4c3cd8f4c9506d79a8cd56bace40 Author: Ian Lance Taylor <iant@golang.org> Date: Fri Apr 14 10:04:23 2017 -0700 runtime: add tests from Go 1.8 Some runtime package tests never made it into the gofrontend repo for some reason. Add them now. Reviewed-on: https://go-review.googlesource.com/40869 commit 1feef185aebd71bc2a09b9a04287461806096610 Author: Ian Lance Taylor <iant@golang.org> Date: Mon Apr 17 16:26:11 2017 -0700 runtime: change mcall to take a Go function value For future work in bringing in the Go 1.8 GC, change the mcall function to take a Go function value, which means that mcall can take a closure rather than just a straight C function pointer. As part of this change move kickoff from C to Go, which we want to do anyhow so that we run the write barriers that it generates. Reviewed-on: https://go-review.googlesource.com/40935 commit c3db34f4efc2d610f74a01dd2ad7775f48889b29 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Tue Apr 11 16:11:26 2017 +0200 runtime: netpoll implementation for AIX Code courtesy of Damien Bergamini from Atos Infogérance. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/40352 commit f5634dff40e53ad9ce61afd67fd07334e3af9d1f Author: Ian Lance Taylor <iant@golang.org> Date: Tue Apr 18 22:06:07 2017 -0700 runtime: move mstart from Go to C The assignments done in mstart must be done without write barriers, as mstart is running without an m or p. In the gc toolchain the equivalent code to intialize g and g->m is written in assembler; on GNU/Linux, it's in the clone function. Reviewed-on: https://go-review.googlesource.com/40989 commit 671d7c74592f4b6fe3665af279482ba0ea47ca2d Author: Ian Lance Taylor <iant@golang.org> Date: Tue Apr 18 17:47:28 2017 -0700 compiler: varargs slices do not escape in runtime Also, don't try to allocate an empty slice on the stack, as it will confuse the GCC backend. Also add a few trivial style, code formatting, and debug output fixes. Updates golang/go#17431 Reviewed-on: https://go-review.googlesource.com/40983 commit 94699d25f31353bf03419eda56b15993a39f3275 Author: Ian Lance Taylor <iant@golang.org> Date: Tue Apr 18 17:30:09 2017 -0700 compiler: add Ptrmask_symbol_expression Add an expression to evaluate to the ptrmask for a type. This will be used for global variables, which always use a ptrmask no matter how large they are. Reviewed-on: https://go-review.googlesource.com/40981 commit bfff1654eac5b9288fa6c431e66cba8c9da6a660 Author: Ian Lance Taylor <iant@golang.org> Date: Mon Apr 17 10:51:16 2017 -0700 runtime: change g's in systemstack The systemstack function in the gc toolchain changes to a different g. This is often used to get more stack space; the gofrontend uses a different stack growth mechanism that does not require changing g's, so we've been running with a version of systemstack that keeps the same g. However, the garbage collector has various tests to verify that it is running on g0 rather than on a normal g. For simplicity, change the gofrontend version of systemstack to change to a different g just as the gc toolchain does. This permits us to uncomment some sanity checks in notetsleep. Doing that requires us to fix up a couple of places where C code calls {start,stop}TheWorldWithSema while not on g0. Note that this does slow down some code in the runtime package unnecessarily. It may be useful to find some places where the runtime calls systemstack only to get more stack space and change it to use some other function. That other function would act like systemstack in the gc toolchain but simply call the argument in the gofrontend. Reviewed-on: https://go-review.googlesource.com/40973 commit b2ccc7601ce71a7c5732154cf9b2eeea64681469 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 10:36:12 2017 -0700 compiler, runtime: include ptrmask in GC roots Change the list of registered GC roots to include a ptrmask, and change the data structures to be easily used from Go code. The new ptrmask will be used by the Go 1.8 GC to only scan pointers. Tweak the current GC to use the new structures, but ignore the new ptrmask information for now. The new GC root data includes the size of the variable. The size is not currently used, but will be used later by the cgo checking code. Reviewed-on: https://go-review.googlesource.com/41075 commit 9e065149970bc180e4ca83bb99c74d9c4f43b47b Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 12:23:16 2017 -0700 compiler, runtime: don't pass size to __go_new There is no reason to pass the size to __go_new, as the type descriptor includes the size anyhow. This makes the function correspond to the Go 1.8 function runtime.newobject, which is what we will use when we update to the Go 1.8 memory allocator. Reviewed-on: https://go-review.googlesource.com/41080 commit c321de7b738c4a3387c1842919c9305acfa04c57 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 13:13:56 2017 -0700 compiler, runtime, reflect: make type descriptors more like Go 1.8 Change the type descriptor structure to be more like the one in the Go 1.8 runtime. Specifically we add the ptrdata field, rename the gc field to gcdata and change the type to *byte, and rearrange a few of the fields. The structure is still not identical to the Go 1.8 structure--we don't use any of the tricks to reduce overall executable size--but it is more similar. For now we don't use the new ptrdata field, and the gcdata field is still the old format rather than the new Go 1.8 ptrmask/gcprog format. Reviewed-on: https://go-review.googlesource.com/41081 commit 7b70c52cddeebea9ebeac003f8c6aad59497e5f0 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 14:54:29 2017 -0700 reflect: make sure to clear unusable hash/equal function Otherwise we wind up copying the one from the prototype, which is wrong. Also rewrite the hash/equal functions to look like the ones in Go 1.8, mainly a matter of changing names and using arrayAt. Reviewed-on: https://go-review.googlesource.com/41133 commit 84d26f467f7de8bdbb0d230458135fe1b6b2a99d Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 14:59:13 2017 -0700 runtime: remove duplicate declarations of SetFinalizer/KeepAlive These should have been removed in CL 38794. It's a bug that the compiler even permits these duplicate declarations. Reviewed-on: https://go-review.googlesource.com/41134 commit f85ff7e64c24031f6d0bd7c9c426b6176cb95160 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 15:56:32 2017 -0700 runtime: don't crash if panicstring called with no m It's possible for runtime_panicstring to be called with no m if a signal handler, or scheduler innards, do something wrong. If that happens carry on with the panic rather than crashing. Reviewed-on: https://go-review.googlesource.com/41137 commit 5b362b04f642afb8b20715930416fc3b7d91bb12 Author: Than McIntosh <thanm@google.com> Date: Fri Mar 31 14:35:48 2017 -0400 compiler: fix for expr sharing introduced by Order_eval::statement. When processing an expression statement with a top-level call that returns multiple results, Order_eval::statement can wind up creating a tree that has multiple references to the same call, which results in a confusing AST dump. Change the implementation to avoid introducing this unwanted sharing. Reviewed-on: https://go-review.googlesource.com/39210 commit b05b4260a68695bf9c9cc29e14ae86ca2699458a Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 16:00:28 2017 -0700 runtime: restore correct m in gtraceback If gtraceback is used to get a stack trace of a g running in the same m, as can happen if we collect a stack trace from a g0, then restore the old m value, don't clear it. Reviewed-on: https://go-review.googlesource.com/41138 commit ca8bbf4dfac19b3f4f7ce21a688b96a418c75031 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 16:03:24 2017 -0700 runtime: set startpc field when starting a new goroutine This puts the right value in a trace--previously it was always zero. Reviewed-on: https://go-review.googlesource.com/41139 commit ca8bbf4dfac19b3f4f7ce21a688b96a418c75031 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 16:03:24 2017 -0700 runtime: set startpc field when starting a new goroutine This puts the right value in a trace--previously it was always zero. Reviewed-on: https://go-review.googlesource.com/41139 commit 887690dce42d7bf8f711f8ea082e4928fb70f2a5 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 17:06:11 2017 -0700 runtime: add prefetch functions The Go 1.8 GC uses prefetch functions. Add versions for gccgo that call __builtin_prefetch. Uncomment the test for them in testAtomic64. Don't force the check function to return early, as now taking the address of a local variable in the runtime package does not force it onto the heap. Reviewed-on: https://go-review.googlesource.com/41144 commit 4269db69f9184e5a45c54aaee7352425a1f88bff Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 17:55:21 2017 -0700 runtime: split up ticks to get correct alignment On 32-bit x86 a uint64 variable by itself is aligned to an 8-byte boundary. A uint64 field in a struct is aligned to a 4-byte boundary. The runtime.ticks variable has a uint64 field that must be aligned to an 8-byte boundary. Rather than rely on luck, split up the struct into separate vars so that the required alignment happens reliably. It would be much nicer if issue golang/go#19057 were fixed somehow, but that is for another day. Reviewed-on: https://go-review.googlesource.com/41143 commit 66926cabdbdbf3431b4f172f7756e195c1c6c513 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Apr 20 17:15:38 2017 +0200 libgo: fix bad value for O_CLOEXEC on AIX 7.1 On AIX 7.1, O_CLOEXEC is defined as 0x0000001000000000, which creates an integer constant overflow error when building libgo. This affects only 7.1, O_CLOEXEC is not defined on 6.1 (and defaults to O in sysinfo.go) and is defined as 0x00800000 on AIX 7.2. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/41214 commit af288ff10aeafc47651f5def327ed56425d5be19 Author: Ian Lance Taylor <iant@golang.org> Date: Thu Apr 20 17:15:02 2017 -0700 runtime: preserve stack context in tracebackothers The tracebackothers function works by saving the current stack context in the goroutine's context field and then calling gogo to switch to a new goroutine. The new goroutine will collect its own stack trace and then call gogo to switch back to the original goroutine. This works fine, but if the original goroutine was called by mcall then the contents of its context field are needed to return from the mcall. Fix this by saving the stack context across the calls to the other goroutines. Reviewed-on: https://go-review.googlesource.com/41293 commit 43101e5956e793f1b4de05c15d7738c785e927df Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Fri Apr 21 10:58:52 2017 +0200 os/user: use _posix_* libc functions libc getpwnam_r function has a different signature, we must use _posix_getpwnam_r instead (by default, the pwd.h system include file defines getpwnam_r as a static function calling _posix_getpwnam_r, so a C program calling getpwnam_r will indeed reference the _posix_getpwnam_r symbol). Idem for getpwuid_r, getgrnam_r and getgrgid_r. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/41334 commit 71e1fec4d2a536591ea6657a06916a17b5127071 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Apr 19 21:24:48 2017 -0700 runtime: don't use pointers in g_ucontext_t or stackcontext The g_ucontext_t type holds registers saved for a goroutine. We have to scan it for pointers, but since registers don't necessarily hold pointers we have to scan it conservatively. That means that it should not have a pointer type, since the GC will always scan pointers. Instead it needs special treatment to be scanned conservatively. The current GC doesn't care when a pointer type holds a non-pointer, but the Go 1.8 GC does. For the current GC this means we have to explicitly scan the g_ucontext_t values in a G. While we're at it change stackcontext to be uintptr too. The entries in stackcontext never hold pointers that the Go GC cares about. Reviewed-on: https://go-review.googlesource.com/41270 commit eab2960aee91d3e3a6baa5b1bce01262d24c714f Author: Ian Lance Taylor <iant@golang.org> Date: Thu Apr 20 17:08:19 2017 -0700 runtime/internal/sys: define Goexperiment The gc toolchain defines Goexperiment based on the environment variable GOEXPERIMENT when the toolchain is built. We just always set Goexperiment to the empty string. Reviewed-on: https://go-review.googlesource.com/41292 commit be4a751943265c0637da859d15a4faf162f5c478 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Thu Apr 20 14:04:35 2017 +0200 net: sockopt implementation for AIX This is a copy of the Linux implementation, it allows to run some simple client/server applications on AIX, while the current sockopt stubs don't. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/41213 commit 46a669c4ca5b80fd6f6a0a42095804d9f704611d Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Mar 29 17:55:06 2017 +0200 math: fix sign for atan/expm1/log1p(-0) AIX libc returns +0 for atan(-0), expm1(-0) and log1p(-0), while matching Go functions must return -0. Code courtesy of Tony Reix. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/38699 commit 53b0e809130038a46f0a3d2870e3905f44ab888d Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Apr 26 17:29:22 2017 +0200 runtime: fix context clobbering on AIX On AIX 64-bits, r13 is a pointer to thread data. setcontext() overwrites r13 with the value saved by getcontext(). So, when a goroutine is scheduled on a new thread, r13 will point to the old thread data after calling setcontext(). Code courtesy of Damien Bergamini. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/41854 commit f8d5ebd71c71e6e777200530d8204b92619157f8 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Wed Apr 26 18:01:19 2017 +0200 runtime: fix wrong time calculation in semasleep tv_nsec is added twice when calculating the sleep end time. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/41855 commit ef56097f4ea848d48fbf61eba1c757fe7fce99d3 Author: Matthieu Sarter <matthieu.sarter.external@atos.net> Date: Fri Apr 28 10:27:32 2017 +0200 libgo: pass $(NM) value when running benchmarks On AIX, we need to use "nm -B" instead of "nm", to have the epxected output format, so the configured $(NM) value from the Makefile should be exported before running gotest, which defaults to "nm" if $NM is not set. Issue golang/go#19200 Reviewed-on: https://go-review.googlesource.com/42051 commit 0fb550083ae474fb964435927b899ec8e4b62771 Author: Ian Lance Taylor <iant@golang.org> Date: Wed Nov 16 21:12:53 2016 -0800 runtime: copy garbage collector from Go 1.8 runtime This giant patch replaces the old Go 1.4 memory allocator and garbage collector with the new Go 1.8 code. The memory allocator is fairly similar, though now written in Go rather than C. The garbage collector is completely different. It now uses ptrmask and gcprog information, which requires changes in the compiler and the reflect package as well as the runtime. And, of course, the garbage collector now runs concurrently with program execution. In the gc toolchain the garbage collector is strict and precise at all levels. In the gofrontend we do not have stack maps, so stacks, and register values, are collected conservatively. That means that an old, no longer used, pointer on a stack or in a register can cause a memory object to live longer than it should. That in turns means that we must disable some checks for invalid pointers in the garbage collection code. Not only can we get an invalid pointer on the stack; the concurrent nature of the collector means that we can in effect resurrect a block that was already unmarked but that the collector had not yet gotten around to freeing, and that block can in turn point to other blocks that the collector had managed to already free. So we must disable pointer checks in general. In effect we are relying on the fact that the strict pointer checks in the gc toolchain ensure that the garbage collector is correct, and we just assume that it is correct for the gofrontend since we are using the same code. Reviewed-on: https://go-review.googlesource.com/41307 commit a95078d501175240d095500a8c5fbfb21bec65cb Author: Ian Lance Taylor <iant@golang.org> Date: Mon Apr 24 16:33:47 2017 -0700 libgo/Makefile: clean more files Fix up the mostlyclean, clean, and distclean targets to better follow https://www.gnu.org/prep/standards/html_node/Standard-Targets.html. Reviewed-on: https://go-review.googlesource.com/41625 commit 5956bf1055451cf4239cdfeca259c23b1ded54d8 Author: Ian Lance Taylor <iant@golang.org> Date: Mon May 8 13:35:11 2017 -0700 libgo: delete goc2c The last .goc file has been removed, so remove goc2c. The goc2c program was my first contribution to the gc repository that was more than 100 lines: golang/go@2b57a11 The program was used in gc for a few years under various guises but was finally removed in https://golang.org/cl/132680043. Now we can remove it from gofrontend as well. Reviewed-on: https://go-review.googlesource.com/42911 commit a222e35d041de0cd42506b61c93b8209e07702b9 Author: Than McIntosh <thanm@google.com> Date: Tue May 9 10:33:10 2017 -0400 compiler: set "need_init_fn" when adding gc root Variables that back slice initializers in certain cases have to be added to the gc roots list, since they can be modified at runtime. The code that was doing this addition did not update the flag that tracks whether the package being compiled needs an initializer function, which resulted in the call in question being left out of the final generated code in certain cases. Fix is to change Gogo::add_gc_root() to update the "needs init" flag. Reviewed-on: https://go-review.googlesource.com/43030 commit 822ab419bf7d1c705cdce1c12133e7a11f56be2e Author: Than McIntosh <thanm@google.com> Date: Tue May 9 11:36:51 2017 -0400 compiler: fix variable context nit in write barrier generation Update the write barrier generation code to insure that the "lvalue context" tag on the space var expression is set only in the case where the expr feeds directly into an assignment. This is somewhat counter-intuitive, but needed in the case where the backend looks at context tags. Reviewed-on: https://go-review.googlesource.com/43031 From-SVN: r247848

yuseferi · 2023-07-04T14:16:39Z

sounds like an interesting topic and someone gave a talk at GopherConf Eu 2023,
any update on this proposal?

…on' panic on 32-bit architectures The issue has been introduced in bace9a2 The improper fix was in the d4c0615 , since it fixed the issue just by an accident, because Go comiler aligned the rawRowsShards field by 4-byte boundary inside partition struct. The proper fix is to use atomic.Int64 field - this guarantees that the access to this field won't result in unaligned 64-bit atomic operation. See golang/go#50860 and golang/go#19057

ianlancetaylor added the Proposal label Feb 13, 2017

ianlancetaylor added this to the Proposal milestone Feb 13, 2017

bcmills mentioned this issue Sep 6, 2017

proposal: spec: forbid zero-size arrays #21765

Closed

ianlancetaylor mentioned this issue Aug 25, 2019

net/http: possibly unaligned 64-bit word accessed atomically #33821

Closed

danscales mentioned this issue Dec 12, 2019

cmd/compile: make 64-bit fields 64-bit aligned on 32-bit systems #599

Closed

danscales mentioned this issue Jan 16, 2020

proposal: cmd/compile: make 64-bit fields be 64-bit aligned on 32-bit systems, add //go:packed directive on structs #36606

Open

josharian mentioned this issue Dec 25, 2020

Change the representation of IP to a pair of uint64s. inetaf/netaddr#63

Merged

mdempsky mentioned this issue Jun 13, 2021

cmd/compile: replace //go:notinheap with runtime/internal/sys.NotInHeap #46731

Closed

jmkristian mentioned this issue Aug 28, 2021

panic in ServeHTTP: unaligned 64-bit atomic operation drakkan/sftpgo#516

Closed

bcmills mentioned this issue Jan 28, 2022

sync/atomic: add typed atomic values #50860

Closed

josharian mentioned this issue Oct 18, 2022

Replace conditional variable with semaphore jackc/puddle#24

Closed

zephyrtronium mentioned this issue Aug 21, 2023

proposal: support padding for generic structs #62172

Open

qmuntal mentioned this issue Jan 12, 2024

cmd/compile: in specific conditions in go1.22rc1, Windows SetFileInformationByHandle returns "Invalid access to memory location" #65069

Closed

ncruces mentioned this issue Feb 12, 2024

wazevo: aligns globals at 16 byte tetratelabs/wazero#2042

Merged

dr2chase mentioned this issue Mar 19, 2024

proposal: structs: add HostLayout "directive" type #66408

Open

proposal: runtime: add AlignedN types that can be used to increase alignment #19057

proposal: runtime: add AlignedN types that can be used to increase alignment #19057

Comments

ianlancetaylor commented Feb 13, 2017 • edited

josharian commented Feb 13, 2017

ianlancetaylor commented Feb 13, 2017

cespare commented Feb 13, 2017

ianlancetaylor commented Feb 13, 2017

rsc commented Feb 13, 2017

mdempsky commented Feb 13, 2017 • edited

ianlancetaylor commented Feb 14, 2017

minux commented Feb 14, 2017 via email

ianlancetaylor commented Feb 14, 2017

cespare commented Feb 14, 2017

rsc commented Feb 14, 2017

minux commented Feb 14, 2017 via email • edited

josharian commented Feb 14, 2017

rsc commented Feb 14, 2017

josharian commented Feb 14, 2017

ianlancetaylor commented Feb 14, 2017

josharian commented Feb 14, 2017

cespare commented Feb 14, 2017

minux commented Feb 14, 2017 via email

rsc commented Feb 14, 2017

minux commented Feb 14, 2017 via email

josharian commented Feb 14, 2017 • edited

rsc commented Feb 14, 2017

bcmills commented Feb 14, 2017 • edited

rsc commented Feb 15, 2017

minux commented Feb 15, 2017 via email

bcmills commented Feb 15, 2017 • edited

josharian commented Feb 15, 2017

rsc commented Apr 10, 2017

gopherbot commented Apr 20, 2017

ugorji commented Dec 26, 2017

aclements commented Oct 9, 2018

markdryan commented Apr 29, 2019

seebs commented Aug 19, 2019 • edited

iand commented Aug 19, 2019

gopherbot commented Jun 12, 2020

gopherbot commented Apr 9, 2021

rogpeppe commented Apr 14, 2021 • edited

mdempsky commented Apr 16, 2021

yuseferi commented Jul 4, 2023

ianlancetaylor commented Feb 13, 2017 •

edited

mdempsky commented Feb 13, 2017 •

edited

minux commented Feb 14, 2017 via email •

edited

josharian commented Feb 14, 2017 •

edited

bcmills commented Feb 14, 2017 •

edited

bcmills commented Feb 15, 2017 •

edited

seebs commented Aug 19, 2019 •

edited

rogpeppe commented Apr 14, 2021 •

edited