Package runtime

allocfreetrace: setting allocfreetrace=1 causes every allocation to be
profiled and a stack trace printed on each object's allocation and free.

clobberfree: setting clobberfree=1 causes the garbage collector to
clobber the memory content of an object with bad content when it frees
the object.

cgocheck: setting cgocheck=0 disables all checks for packages
using cgo to incorrectly pass Go pointers to non-Go code.
Setting cgocheck=1 (the default) enables relatively cheap
checks that may miss some errors.  Setting cgocheck=2 enables
expensive checks that should not miss any errors, but will
cause your program to run slower.

efence: setting efence=1 causes the allocator to run in a mode
where each object is allocated on a unique page and addresses are
never recycled.

gccheckmark: setting gccheckmark=1 enables verification of the
garbage collector's concurrent mark phase by performing a
second mark pass while the world is stopped.  If the second
pass finds a reachable object that was not found by concurrent
mark, the garbage collector will panic.

gcpacertrace: setting gcpacertrace=1 causes the garbage collector to
print information about the internal state of the concurrent pacer.

gcshrinkstackoff: setting gcshrinkstackoff=1 disables moving goroutines
onto smaller stacks. In this mode, a goroutine's stack can only grow.

gcstoptheworld: setting gcstoptheworld=1 disables concurrent garbage collection,
making every garbage collection a stop-the-world event. Setting gcstoptheworld=2
also disables concurrent sweeping after the garbage collection finishes.

gctrace: setting gctrace=1 causes the garbage collector to emit a single line to standard
error at each collection, summarizing the amount of memory collected and the
length of the pause. The format of this line is subject to change.
Currently, it is:
	gc # @#s #%: #+#+# ms clock, #+#/#/#+# ms cpu, #->#-># MB, # MB goal, # P
where the fields are as follows:
	gc #        the GC number, incremented at each GC
	@#s         time in seconds since program start
	#%          percentage of time spent in GC since program start
	#+...+#     wall-clock/CPU times for the phases of the GC
	#->#-># MB  heap size at GC start, at GC end, and live heap
	# MB goal   goal heap size
	# P         number of processors used
The phases are stop-the-world (STW) sweep termination, concurrent
mark and scan, and STW mark termination. The CPU times
for mark/scan are broken down in to assist time (GC performed in
line with allocation), background GC time, and idle GC time.
If the line ends with "(forced)", this GC was forced by a
runtime.GC() call.

inittrace: setting inittrace=1 causes the runtime to emit a single line to standard
error for each package with init work, summarizing the execution time and memory
allocation. No information is printed for inits executed as part of plugin loading
and for packages without both user defined and compiler generated init work.
The format of this line is subject to change. Currently, it is:
	init # @#ms, # ms clock, # bytes, # allocs
where the fields are as follows:
	init #      the package name
	@# ms       time in milliseconds when the init started since program start
	# clock     wall-clock time for package initialization work
	# bytes     memory allocated on the heap
	# allocs    number of heap allocations

madvdontneed: setting madvdontneed=0 will use MADV_FREE
instead of MADV_DONTNEED on Linux when returning memory to the
kernel. This is more efficient, but means RSS numbers will
drop only when the OS is under memory pressure.

memprofilerate: setting memprofilerate=X will update the value of runtime.MemProfileRate.
When set to 0 memory profiling is disabled.  Refer to the description of
MemProfileRate for the default value.

invalidptr: invalidptr=1 (the default) causes the garbage collector and stack
copier to crash the program if an invalid pointer value (for example, 1)
is found in a pointer-typed location. Setting invalidptr=0 disables this check.
This should only be used as a temporary workaround to diagnose buggy code.
The real fix is to not store integers in pointer-typed locations.

sbrk: setting sbrk=1 replaces the memory allocator and garbage collector
with a trivial allocator that obtains memory from the operating system and
never reclaims any memory.

scavenge: scavenge=1 enables debugging mode of heap scavenger.

scavtrace: setting scavtrace=1 causes the runtime to emit a single line to standard
error, roughly once per GC cycle, summarizing the amount of work done by the
scavenger as well as the total amount of memory returned to the operating system
and an estimate of physical memory utilization. The format of this line is subject
to change, but currently it is:
	scav # # KiB work, # KiB total, #% util
where the fields are as follows:
	scav #       the scavenge cycle number
	# KiB work   the amount of memory returned to the OS since the last line
	# KiB total  the total amount of memory returned to the OS
	#% util      the fraction of all unscavenged memory which is in-use
If the line ends with "(forced)", then scavenging was forced by a
debug.FreeOSMemory() call.

scheddetail: setting schedtrace=X and scheddetail=1 causes the scheduler to emit
detailed multiline info every X milliseconds, describing state of the scheduler,
processors, threads and goroutines.

schedtrace: setting schedtrace=X causes the scheduler to emit a single line to standard
error every X milliseconds, summarizing the scheduler state.

tracebackancestors: setting tracebackancestors=N extends tracebacks with the stacks at
which goroutines were created, where N limits the number of ancestor goroutines to
report. This also extends the information returned by runtime.Stack. Ancestor's goroutine
IDs will refer to the ID of the goroutine at the time of creation; it's possible for this
ID to be reused for another goroutine. Setting N to 0 will report no ancestry information.

asyncpreemptoff: asyncpreemptoff=1 disables signal-based
asynchronous goroutine preemption. This makes some loops
non-preemptible for long periods, which may delay GC and
goroutine scheduling. This is useful for debugging GC issues
because it also disables the conservative stack scanning used
for asynchronously preempted goroutines.

Constants

const (
    c0 = uintptr((8-sys.PtrSize)/4*2860486313 + (sys.PtrSize-4)/4*33054211828000289)
    c1 = uintptr((8-sys.PtrSize)/4*3267000013 + (sys.PtrSize-4)/4*23344194077549503)
)

const (
    maxAlign  = 8
    hchanSize = unsafe.Sizeof(hchan{}) + uintptr(-int(unsafe.Sizeof(hchan{}))&(maxAlign-1))
    debugChan = false
)

Offsets into internal/cpu records for use in assembly.

const (
    offsetX86HasAVX  = unsafe.Offsetof(cpu.X86.HasAVX)
    offsetX86HasAVX2 = unsafe.Offsetof(cpu.X86.HasAVX2)
    offsetX86HasERMS = unsafe.Offsetof(cpu.X86.HasERMS)
    offsetX86HasSSE2 = unsafe.Offsetof(cpu.X86.HasSSE2)

    offsetARMHasIDIVA = unsafe.Offsetof(cpu.ARM.HasIDIVA)

    offsetMIPS64XHasMSA = unsafe.Offsetof(cpu.MIPS64X.HasMSA)
)

const (
    debugCallSystemStack = "executing on Go runtime stack"
    debugCallUnknownFunc = "call from unknown function"
    debugCallRuntime     = "call from within the Go runtime"
    debugCallUnsafePoint = "call not at safe point"
)

const (
    debugLogUnknown = 1 + iota
    debugLogBoolTrue
    debugLogBoolFalse
    debugLogInt
    debugLogUint
    debugLogHex
    debugLogPtr
    debugLogString
    debugLogConstString
    debugLogStringOverflow

    debugLogPC
    debugLogTraceback
)

const (
    // debugLogHeaderSize is the number of bytes in the framing
    // header of every dlog record.
    debugLogHeaderSize = 2

    // debugLogSyncSize is the number of bytes in a sync record.
    debugLogSyncSize = debugLogHeaderSize + 2*8
)

const (
    _EINTR  = 0x4
    _EAGAIN = 0xb
    _ENOMEM = 0xc
    _ENOSYS = 0x26

    _PROT_NONE  = 0x0
    _PROT_READ  = 0x1
    _PROT_WRITE = 0x2
    _PROT_EXEC  = 0x4

    _MAP_ANON    = 0x20
    _MAP_PRIVATE = 0x2
    _MAP_FIXED   = 0x10

    _MADV_DONTNEED   = 0x4
    _MADV_FREE       = 0x8
    _MADV_HUGEPAGE   = 0xe
    _MADV_NOHUGEPAGE = 0xf

    _SA_RESTART  = 0x10000000
    _SA_ONSTACK  = 0x8000000
    _SA_RESTORER = 0x4000000
    _SA_SIGINFO  = 0x4

    _SIGHUP    = 0x1
    _SIGINT    = 0x2
    _SIGQUIT   = 0x3
    _SIGILL    = 0x4
    _SIGTRAP   = 0x5
    _SIGABRT   = 0x6
    _SIGBUS    = 0x7
    _SIGFPE    = 0x8
    _SIGKILL   = 0x9
    _SIGUSR1   = 0xa
    _SIGSEGV   = 0xb
    _SIGUSR2   = 0xc
    _SIGPIPE   = 0xd
    _SIGALRM   = 0xe
    _SIGSTKFLT = 0x10
    _SIGCHLD   = 0x11
    _SIGCONT   = 0x12
    _SIGSTOP   = 0x13
    _SIGTSTP   = 0x14
    _SIGTTIN   = 0x15
    _SIGTTOU   = 0x16
    _SIGURG    = 0x17
    _SIGXCPU   = 0x18
    _SIGXFSZ   = 0x19
    _SIGVTALRM = 0x1a
    _SIGPROF   = 0x1b
    _SIGWINCH  = 0x1c
    _SIGIO     = 0x1d
    _SIGPWR    = 0x1e
    _SIGSYS    = 0x1f

    _FPE_INTDIV = 0x1
    _FPE_INTOVF = 0x2
    _FPE_FLTDIV = 0x3
    _FPE_FLTOVF = 0x4
    _FPE_FLTUND = 0x5
    _FPE_FLTRES = 0x6
    _FPE_FLTINV = 0x7
    _FPE_FLTSUB = 0x8

    _BUS_ADRALN = 0x1
    _BUS_ADRERR = 0x2
    _BUS_OBJERR = 0x3

    _SEGV_MAPERR = 0x1
    _SEGV_ACCERR = 0x2

    _ITIMER_REAL    = 0x0
    _ITIMER_VIRTUAL = 0x1
    _ITIMER_PROF    = 0x2

    _EPOLLIN       = 0x1
    _EPOLLOUT      = 0x4
    _EPOLLERR      = 0x8
    _EPOLLHUP      = 0x10
    _EPOLLRDHUP    = 0x2000
    _EPOLLET       = 0x80000000
    _EPOLL_CLOEXEC = 0x80000
    _EPOLL_CTL_ADD = 0x1
    _EPOLL_CTL_DEL = 0x2
    _EPOLL_CTL_MOD = 0x3

    _AF_UNIX    = 0x1
    _SOCK_DGRAM = 0x2
)

const (
    _O_RDONLY   = 0x0
    _O_NONBLOCK = 0x800
    _O_CLOEXEC  = 0x80000
)

const (
    // Constants for multiplication: four random odd 64-bit numbers.
    m1 = 16877499708836156737
    m2 = 2820277070424839065
    m3 = 9497967016996688599
    m4 = 15839092249703872147
)

const (
    fieldKindEol       = 0
    fieldKindPtr       = 1
    fieldKindIface     = 2
    fieldKindEface     = 3
    tagEOF             = 0
    tagObject          = 1
    tagOtherRoot       = 2
    tagType            = 3
    tagGoroutine       = 4
    tagStackFrame      = 5
    tagParams          = 6
    tagFinalizer       = 7
    tagItab            = 8
    tagOSThread        = 9
    tagMemStats        = 10
    tagQueuedFinalizer = 11
    tagData            = 12
    tagBSS             = 13
    tagDefer           = 14
    tagPanic           = 15
    tagMemProf         = 16
    tagAllocSample     = 17
)

Cache of types that have been serialized already. We use a type's hash field to pick a bucket. Inside a bucket, we keep a list of types that have been serialized so far, most recently used first. Note: when a bucket overflows we may end up serializing a type more than once. That's ok.

const (
    typeCacheBuckets = 256
    typeCacheAssoc   = 4
)

const (
    // For the time histogram type, we use an HDR histogram.
    // Values are placed in super-buckets based solely on the most
    // significant set bit. Thus, super-buckets are power-of-2 sized.
    // Values are then placed into sub-buckets based on the value of
    // the next timeHistSubBucketBits most significant bits. Thus,
    // sub-buckets are linear within a super-bucket.
    //
    // Therefore, the number of sub-buckets (timeHistNumSubBuckets)
    // defines the error. This error may be computed as
    // 1/timeHistNumSubBuckets*100%. For example, for 16 sub-buckets
    // per super-bucket the error is approximately 6%.
    //
    // The number of super-buckets (timeHistNumSuperBuckets), on the
    // other hand, defines the range. To reserve room for sub-buckets,
    // bit timeHistSubBucketBits is the first bit considered for
    // super-buckets, so super-bucket indices are adjusted accordingly.
    //
    // As an example, consider 45 super-buckets with 16 sub-buckets.
    //
    //    00110
    //    ^----
    //    │  ^
    //    │  └---- Lowest 4 bits -> sub-bucket 6
    //    └------- Bit 4 unset -> super-bucket 0
    //
    //    10110
    //    ^----
    //    │  ^
    //    │  └---- Next 4 bits -> sub-bucket 6
    //    └------- Bit 4 set -> super-bucket 1
    //    100010
    //    ^----^
    //    │  ^ └-- Lower bits ignored
    //    │  └---- Next 4 bits -> sub-bucket 1
    //    └------- Bit 5 set -> super-bucket 2
    //
    // Following this pattern, bucket 45 will have the bit 48 set. We don't
    // have any buckets for higher values, so the highest sub-bucket will
    // contain values of 2^48-1 nanoseconds or approx. 3 days. This range is
    // more than enough to handle durations produced by the runtime.
    timeHistSubBucketBits   = 4
    timeHistNumSubBuckets   = 1 << timeHistSubBucketBits
    timeHistNumSuperBuckets = 45
    timeHistTotalBuckets    = timeHistNumSuperBuckets*timeHistNumSubBuckets + 1
)

const (
    fInf    = 0x7FF0000000000000
    fNegInf = 0xFFF0000000000000
)

const (
    // addrBits is the number of bits needed to represent a virtual address.
    //
    // See heapAddrBits for a table of address space sizes on
    // various architectures. 48 bits is enough for all
    // architectures except s390x.
    //
    // On AMD64, virtual addresses are 48-bit (or 57-bit) numbers sign extended to 64.
    // We shift the address left 16 to eliminate the sign extended part and make
    // room in the bottom for the count.
    //
    // On s390x, virtual addresses are 64-bit. There's not much we
    // can do about this, so we just hope that the kernel doesn't
    // get to really high addresses and panic if it does.
    addrBits = 48

    // In addition to the 16 bits taken from the top, we can take 3 from the
    // bottom, because node must be pointer-aligned, giving a total of 19 bits
    // of count.
    cntBits = 64 - addrBits + 3

    // On AIX, 64-bit addresses are split into 36-bit segment number and 28-bit
    // offset in segment.  Segment numbers in the range 0x0A0000000-0x0AFFFFFFF(LSA)
    // are available for mmap.
    // We assume all lfnode addresses are from memory allocated with mmap.
    // We use one bit to distinguish between the two ranges.
    aixAddrBits = 57
    aixCntBits  = 64 - aixAddrBits + 3
)

const (
    mutex_unlocked = 0
    mutex_locked   = 1
    mutex_sleeping = 2

    active_spin     = 4
    active_spin_cnt = 30
    passive_spin    = 1
)

const (
    debugMalloc = false

    maxTinySize   = _TinySize
    tinySizeClass = _TinySizeClass
    maxSmallSize  = _MaxSmallSize

    pageShift = _PageShift
    pageSize  = _PageSize
    pageMask  = _PageMask
    // By construction, single page spans of the smallest object class
    // have the most objects per span.
    maxObjsPerSpan = pageSize / 8

    concurrentSweep = _ConcurrentSweep

    _PageSize = 1 << _PageShift
    _PageMask = _PageSize - 1

    // _64bit = 1 on 64-bit systems, 0 on 32-bit systems
    _64bit = 1 << (^uintptr(0) >> 63) / 2

    // Tiny allocator parameters, see "Tiny allocator" comment in malloc.go.
    _TinySize      = 16
    _TinySizeClass = int8(2)

    _FixAllocChunk = 16 << 10 // Chunk size for FixAlloc

    // Per-P, per order stack segment cache size.
    _StackCacheSize = 32 * 1024

    // Number of orders that get caching. Order 0 is FixedStack
    // and each successive order is twice as large.
    // We want to cache 2KB, 4KB, 8KB, and 16KB stacks. Larger stacks
    // will be allocated directly.
    // Since FixedStack is different on different systems, we
    // must vary NumStackOrders to keep the same maximum cached size.
    //   OS               | FixedStack | NumStackOrders
    //   -----------------+------------+---------------
    //   linux/darwin/bsd | 2KB        | 4
    //   windows/32       | 4KB        | 3
    //   windows/64       | 8KB        | 2
    //   plan9            | 4KB        | 3
    _NumStackOrders = 4 - sys.PtrSize/4*sys.GoosWindows - 1*sys.GoosPlan9

    // heapAddrBits is the number of bits in a heap address. On
    // amd64, addresses are sign-extended beyond heapAddrBits. On
    // other arches, they are zero-extended.
    //
    // On most 64-bit platforms, we limit this to 48 bits based on a
    // combination of hardware and OS limitations.
    //
    // amd64 hardware limits addresses to 48 bits, sign-extended
    // to 64 bits. Addresses where the top 16 bits are not either
    // all 0 or all 1 are "non-canonical" and invalid. Because of
    // these "negative" addresses, we offset addresses by 1<<47
    // (arenaBaseOffset) on amd64 before computing indexes into
    // the heap arenas index. In 2017, amd64 hardware added
    // support for 57 bit addresses; however, currently only Linux
    // supports this extension and the kernel will never choose an
    // address above 1<<47 unless mmap is called with a hint
    // address above 1<<47 (which we never do).
    //
    // arm64 hardware (as of ARMv8) limits user addresses to 48
    // bits, in the range [0, 1<<48).
    //
    // ppc64, mips64, and s390x support arbitrary 64 bit addresses
    // in hardware. On Linux, Go leans on stricter OS limits. Based
    // on Linux's processor.h, the user address space is limited as
    // follows on 64-bit architectures:
    //
    // Architecture  Name              Maximum Value (exclusive)
    // ---------------------------------------------------------------------
    // amd64         TASK_SIZE_MAX     0x007ffffffff000 (47 bit addresses)
    // arm64         TASK_SIZE_64      0x01000000000000 (48 bit addresses)
    // ppc64{,le}    TASK_SIZE_USER64  0x00400000000000 (46 bit addresses)
    // mips64{,le}   TASK_SIZE64       0x00010000000000 (40 bit addresses)
    // s390x         TASK_SIZE         1<<64 (64 bit addresses)
    //
    // These limits may increase over time, but are currently at
    // most 48 bits except on s390x. On all architectures, Linux
    // starts placing mmap'd regions at addresses that are
    // significantly below 48 bits, so even if it's possible to
    // exceed Go's 48 bit limit, it's extremely unlikely in
    // practice.
    //
    // On 32-bit platforms, we accept the full 32-bit address
    // space because doing so is cheap.
    // mips32 only has access to the low 2GB of virtual memory, so
    // we further limit it to 31 bits.
    //
    // On ios/arm64, although 64-bit pointers are presumably
    // available, pointers are truncated to 33 bits. Furthermore,
    // only the top 4 GiB of the address space are actually available
    // to the application, but we allow the whole 33 bits anyway for
    // simplicity.
    // TODO(mknyszek): Consider limiting it to 32 bits and using
    // arenaBaseOffset to offset into the top 4 GiB.
    //
    // WebAssembly currently has a limit of 4GB linear memory.
    heapAddrBits = (_64bit*(1-sys.GoarchWasm)*(1-sys.GoosIos*sys.GoarchArm64))*48 + (1-_64bit+sys.GoarchWasm)*(32-(sys.GoarchMips+sys.GoarchMipsle)) + 33*sys.GoosIos*sys.GoarchArm64

    // maxAlloc is the maximum size of an allocation. On 64-bit,
    // it's theoretically possible to allocate 1<<heapAddrBits bytes. On
    // 32-bit, however, this is one less than 1<<32 because the
    // number of bytes in the address space doesn't actually fit
    // in a uintptr.
    maxAlloc = (1 << heapAddrBits) - (1-_64bit)*1

    // heapArenaBytes is the size of a heap arena. The heap
    // consists of mappings of size heapArenaBytes, aligned to
    // heapArenaBytes. The initial heap mapping is one arena.
    //
    // This is currently 64MB on 64-bit non-Windows and 4MB on
    // 32-bit and on Windows. We use smaller arenas on Windows
    // because all committed memory is charged to the process,
    // even if it's not touched. Hence, for processes with small
    // heaps, the mapped arena space needs to be commensurate.
    // This is particularly important with the race detector,
    // since it significantly amplifies the cost of committed
    // memory.
    heapArenaBytes = 1 << logHeapArenaBytes

    // logHeapArenaBytes is log_2 of heapArenaBytes. For clarity,
    // prefer using heapArenaBytes where possible (we need the
    // constant to compute some other constants).
    logHeapArenaBytes = (6+20)*(_64bit*(1-sys.GoosWindows)*(1-sys.GoarchWasm)) + (2+20)*(_64bit*sys.GoosWindows) + (2+20)*(1-_64bit) + (2+20)*sys.GoarchWasm

    // heapArenaBitmapBytes is the size of each heap arena's bitmap.
    heapArenaBitmapBytes = heapArenaBytes / (sys.PtrSize * 8 / 2)

    pagesPerArena = heapArenaBytes / pageSize

    // arenaL1Bits is the number of bits of the arena number
    // covered by the first level arena map.
    //
    // This number should be small, since the first level arena
    // map requires PtrSize*(1<<arenaL1Bits) of space in the
    // binary's BSS. It can be zero, in which case the first level
    // index is effectively unused. There is a performance benefit
    // to this, since the generated code can be more efficient,
    // but comes at the cost of having a large L2 mapping.
    //
    // We use the L1 map on 64-bit Windows because the arena size
    // is small, but the address space is still 48 bits, and
    // there's a high cost to having a large L2.
    arenaL1Bits = 6 * (_64bit * sys.GoosWindows)

    // arenaL2Bits is the number of bits of the arena number
    // covered by the second level arena index.
    //
    // The size of each arena map allocation is proportional to
    // 1<<arenaL2Bits, so it's important that this not be too
    // large. 48 bits leads to 32MB arena index allocations, which
    // is about the practical threshold.
    arenaL2Bits = heapAddrBits - logHeapArenaBytes - arenaL1Bits

    // arenaL1Shift is the number of bits to shift an arena frame
    // number by to compute an index into the first level arena map.
    arenaL1Shift = arenaL2Bits

    // arenaBits is the total bits in a combined arena map index.
    // This is split between the index into the L1 arena map and
    // the L2 arena map.
    arenaBits = arenaL1Bits + arenaL2Bits

    // arenaBaseOffset is the pointer value that corresponds to
    // index 0 in the heap arena map.
    //
    // On amd64, the address space is 48 bits, sign extended to 64
    // bits. This offset lets us handle "negative" addresses (or
    // high addresses if viewed as unsigned).
    //
    // On aix/ppc64, this offset allows to keep the heapAddrBits to
    // 48. Otherwize, it would be 60 in order to handle mmap addresses
    // (in range 0x0a00000000000000 - 0x0afffffffffffff). But in this
    // case, the memory reserved in (s *pageAlloc).init for chunks
    // is causing important slowdowns.
    //
    // On other platforms, the user address space is contiguous
    // and starts at 0, so no offset is necessary.
    arenaBaseOffset = 0xffff800000000000*sys.GoarchAmd64 + 0x0a00000000000000*sys.GoosAix
    // A typed version of this constant that will make it into DWARF (for viewcore).
    arenaBaseOffsetUintptr = uintptr(arenaBaseOffset)

    // Max number of threads to run garbage collection.
    // 2, 3, and 4 are all plausible maximums depending
    // on the hardware details of the machine. The garbage
    // collector scales well to 32 cpus.
    _MaxGcproc = 32

    // minLegalPointer is the smallest possible legal pointer.
    // This is the smallest possible architectural page size,
    // since we assume that the first page is never mapped.
    //
    // This should agree with minZeroPage in the compiler.
    minLegalPointer uintptr = 4096
)

const (
    // Maximum number of key/elem pairs a bucket can hold.
    bucketCntBits = 3
    bucketCnt     = 1 << bucketCntBits

    // Maximum average load of a bucket that triggers growth is 6.5.
    // Represent as loadFactorNum/loadFactorDen, to allow integer math.
    loadFactorNum = 13
    loadFactorDen = 2

    // Maximum key or elem size to keep inline (instead of mallocing per element).
    // Must fit in a uint8.
    // Fast versions cannot handle big elems - the cutoff size for
    // fast versions in cmd/compile/internal/gc/walk.go must be at most this elem.
    maxKeySize  = 128
    maxElemSize = 128

    // data offset should be the size of the bmap struct, but needs to be
    // aligned correctly. For amd64p32 this means 64-bit alignment
    // even though pointers are 32 bit.
    dataOffset = unsafe.Offsetof(struct {
        b bmap
        v int64
    }{}.v)

    // Possible tophash values. We reserve a few possibilities for special marks.
    // Each bucket (including its overflow buckets, if any) will have either all or none of its
    // entries in the evacuated* states (except during the evacuate() method, which only happens
    // during map writes and thus no one else can observe the map during that time).
    emptyRest      = 0 // this cell is empty, and there are no more non-empty cells at higher indexes or overflows.
    emptyOne       = 1 // this cell is empty
    evacuatedX     = 2 // key/elem is valid.  Entry has been evacuated to first half of larger table.
    evacuatedY     = 3 // same as above, but evacuated to second half of larger table.
    evacuatedEmpty = 4 // cell is empty, bucket is evacuated.
    minTopHash     = 5 // minimum tophash for a normal filled cell.

    // flags
    iterator     = 1 // there may be an iterator using buckets
    oldIterator  = 2 // there may be an iterator using oldbuckets
    hashWriting  = 4 // a goroutine is writing to the map
    sameSizeGrow = 8 // the current map growth is to a new map of the same size

    // sentinel bucket ID for iterator checks
    noCheck = 1<<(8*sys.PtrSize) - 1
)

const (
    bitPointer = 1 << 0
    bitScan    = 1 << 4

    heapBitsShift      = 1     // shift offset between successive bitPointer or bitScan entries
    wordsPerBitmapByte = 8 / 2 // heap words described by one bitmap byte

    // all scan/pointer bits in a byte
    bitScanAll    = bitScan | bitScan<<heapBitsShift | bitScan<<(2*heapBitsShift) | bitScan<<(3*heapBitsShift)
    bitPointerAll = bitPointer | bitPointer<<heapBitsShift | bitPointer<<(2*heapBitsShift) | bitPointer<<(3*heapBitsShift)
)

const (
    _EACCES = 13
    _EINVAL = 22
)

const (
    _DebugGC         = 0
    _ConcurrentSweep = true
    _FinBlockSize    = 4 * 1024

    // debugScanConservative enables debug logging for stack
    // frames that are scanned conservatively.
    debugScanConservative = false

    // sweepMinHeapDistance is a lower bound on the heap distance
    // (in bytes) reserved for concurrent sweeping between GC
    // cycles.
    sweepMinHeapDistance = 1024 * 1024
)

const (
    _GCoff             = iota // GC not running; sweeping in background, write barrier disabled
    _GCmark                   // GC marking roots and workbufs: allocate black, write barrier ENABLED
    _GCmarktermination        // GC mark termination: allocate black, P's help GC, write barrier ENABLED
)

const (
    fixedRootFinalizers = iota
    fixedRootFreeGStacks
    fixedRootCount

    // rootBlockBytes is the number of bytes to scan per data or
    // BSS root.
    rootBlockBytes = 256 << 10

    // maxObletBytes is the maximum bytes of an object to scan at
    // once. Larger objects will be split up into "oblets" of at
    // most this size. Since we can scan 1–2 MB/ms, 128 KB bounds
    // scan preemption at ~100 µs.
    //
    // This must be > _MaxSmallSize so that the object base is the
    // span base.
    maxObletBytes = 128 << 10

    // drainCheckThreshold specifies how many units of work to do
    // between self-preemption checks in gcDrain. Assuming a scan
    // rate of 1 MB/ms, this is ~100 µs. Lower values have higher
    // overhead in the scan loop (the scheduler check may perform
    // a syscall, so its overhead is nontrivial). Higher values
    // make the system less responsive to incoming work.
    drainCheckThreshold = 100000

    // pagesPerSpanRoot indicates how many pages to scan from a span root
    // at a time. Used by special root marking.
    //
    // Higher values improve throughput by increasing locality, but
    // increase the minimum latency of a marking operation.
    //
    // Must be a multiple of the pageInUse bitmap element size and
    // must also evenly divide pagesPerArena.
    pagesPerSpanRoot = 512
)

const (
    // The background scavenger is paced according to these parameters.
    //
    // scavengePercent represents the portion of mutator time we're willing
    // to spend on scavenging in percent.
    scavengePercent = 1 // 1%

    // retainExtraPercent represents the amount of memory over the heap goal
    // that the scavenger should keep as a buffer space for the allocator.
    //
    // The purpose of maintaining this overhead is to have a greater pool of
    // unscavenged memory available for allocation (since using scavenged memory
    // incurs an additional cost), to account for heap fragmentation and
    // the ever-changing layout of the heap.
    retainExtraPercent = 10

    // maxPagesPerPhysPage is the maximum number of supported runtime pages per
    // physical page, based on maxPhysPageSize.
    maxPagesPerPhysPage = maxPhysPageSize / pageSize

    // scavengeCostRatio is the approximate ratio between the costs of using previously
    // scavenged memory and scavenging memory.
    //
    // For most systems the cost of scavenging greatly outweighs the costs
    // associated with using scavenged memory, making this constant 0. On other systems
    // (especially ones where "sysUsed" is not just a no-op) this cost is non-trivial.
    //
    // This ratio is used as part of multiplicative factor to help the scavenger account
    // for the additional costs of using scavenged memory in its pacing.
    scavengeCostRatio = 0.7 * (sys.GoosDarwin + sys.GoosIos)

    // scavengeReservationShards determines the amount of memory the scavenger
    // should reserve for scavenging at a time. Specifically, the amount of
    // memory reserved is (heap size in bytes) / scavengeReservationShards.
    scavengeReservationShards = 64
)

const (
    _WorkbufSize = 2048 // in bytes; larger values result in less contention

    // workbufAlloc is the number of bytes to allocate at a time
    // for new workbufs. This must be a multiple of pageSize and
    // should be a multiple of _WorkbufSize.
    //
    // Larger values reduce workbuf allocation overhead. Smaller
    // values reduce heap fragmentation.
    workbufAlloc = 32 << 10
)

const (
    // minPhysPageSize is a lower-bound on the physical page size. The
    // true physical page size may be larger than this. In contrast,
    // sys.PhysPageSize is an upper-bound on the physical page size.
    minPhysPageSize = 4096

    // maxPhysPageSize is the maximum page size the runtime supports.
    maxPhysPageSize = 512 << 10

    // maxPhysHugePageSize sets an upper-bound on the maximum huge page size
    // that the runtime supports.
    maxPhysHugePageSize = pallocChunkBytes

    // pagesPerReclaimerChunk indicates how many pages to scan from the
    // pageInUse bitmap at a time. Used by the page reclaimer.
    //
    // Higher values reduce contention on scanning indexes (such as
    // h.reclaimIndex), but increase the minimum latency of the
    // operation.
    //
    // The time required to scan this many pages can vary a lot depending
    // on how many spans are actually freed. Experimentally, it can
    // scan for pages at ~300 GB/ms on a 2.6GHz Core i7, but can only
    // free spans at ~32 MB/ms. Using 512 pages bounds this at
    // roughly 100µs.
    //
    // Must be a multiple of the pageInUse bitmap element size and
    // must also evenly divide pagesPerArena.
    pagesPerReclaimerChunk = 512

    // physPageAlignedStacks indicates whether stack allocations must be
    // physical page aligned. This is a requirement for MAP_STACK on
    // OpenBSD.
    physPageAlignedStacks = GOOS == "openbsd"
)

const (
    numSpanClasses = _NumSizeClasses << 1
    tinySpanClass  = spanClass(tinySizeClass<<1 | 1)
)

const (
    _KindSpecialFinalizer = 1
    _KindSpecialProfile   = 2
)

const (
    // The size of a bitmap chunk, i.e. the amount of bits (that is, pages) to consider
    // in the bitmap at once.
    pallocChunkPages    = 1 << logPallocChunkPages
    pallocChunkBytes    = pallocChunkPages * pageSize
    logPallocChunkPages = 9
    logPallocChunkBytes = logPallocChunkPages + pageShift

    // The number of radix bits for each level.
    //
    // The value of 3 is chosen such that the block of summaries we need to scan at
    // each level fits in 64 bytes (2^3 summaries * 8 bytes per summary), which is
    // close to the L1 cache line width on many systems. Also, a value of 3 fits 4 tree
    // levels perfectly into the 21-bit pallocBits summary field at the root level.
    //
    // The following equation explains how each of the constants relate:
    // summaryL0Bits + (summaryLevels-1)*summaryLevelBits + logPallocChunkBytes = heapAddrBits
    //
    // summaryLevels is an architecture-dependent value defined in mpagealloc_*.go.
    summaryLevelBits = 3
    summaryL0Bits    = heapAddrBits - logPallocChunkBytes - (summaryLevels-1)*summaryLevelBits

    // pallocChunksL2Bits is the number of bits of the chunk index number
    // covered by the second level of the chunks map.
    //
    // See (*pageAlloc).chunks for more details. Update the documentation
    // there should this change.
    pallocChunksL2Bits  = heapAddrBits - logPallocChunkBytes - pallocChunksL1Bits
    pallocChunksL1Shift = pallocChunksL2Bits
)

const (
    pallocSumBytes = unsafe.Sizeof(pallocSum(0))

    // maxPackedValue is the maximum value that any of the three fields in
    // the pallocSum may take on.
    maxPackedValue    = 1 << logMaxPackedValue
    logMaxPackedValue = logPallocChunkPages + (summaryLevels-1)*summaryLevelBits

    freeChunkSum = pallocSum(uint64(pallocChunkPages) |
        uint64(pallocChunkPages<<logMaxPackedValue) |
        uint64(pallocChunkPages<<(2*logMaxPackedValue)))
)

const (
    // The number of levels in the radix tree.
    summaryLevels = 5

    // Constants for testing.
    pageAlloc32Bit = 0
    pageAlloc64Bit = 1

    // Number of bits needed to represent all indices into the L1 of the
    // chunks map.
    //
    // See (*pageAlloc).chunks for more details. Update the documentation
    // there should this number change.
    pallocChunksL1Bits = 13
)

const (
    spanSetBlockEntries = 512 // 4KB on 64-bit
    spanSetInitSpineCap = 256 // Enough for 1GB heap on 64-bit
)

const (
    // wbBufEntries is the number of write barriers between
    // flushes of the write barrier buffer.
    //
    // This trades latency for throughput amortization. Higher
    // values amortize flushing overhead more, but increase the
    // latency of flushing. Higher values also increase the cache
    // footprint of the buffer.
    //
    // TODO: What is the latency cost of this? Tune this value.
    wbBufEntries = 256

    // wbBufEntryPointers is the number of pointers added to the
    // buffer by each write barrier.
    wbBufEntryPointers = 2
)

Error codes returned by runtime_pollReset and runtime_pollWait. These must match the values in internal/poll/fd_poll_runtime.go.

const (
    pollNoError        = 0 // no error
    pollErrClosing     = 1 // descriptor is closed
    pollErrTimeout     = 2 // I/O timeout
    pollErrNotPollable = 3 // general error polling descriptor
)

pollDesc contains 2 binary semaphores, rg and wg, to park reader and writer goroutines respectively. The semaphore can be in the following states: pdReady - io readiness notification is pending;

a goroutine consumes the notification by changing the state to nil.

pdWait - a goroutine prepares to park on the semaphore, but not yet parked;

the goroutine commits to park by changing the state to G pointer,
or, alternatively, concurrent io notification changes the state to pdReady,
or, alternatively, concurrent timeout/close changes the state to nil.

G pointer - the goroutine is blocked on the semaphore;

io notification or timeout/close changes the state to pdReady or nil respectively
and unparks the goroutine.

nil - none of the above.

const (
    pdReady uintptr = 1
    pdWait  uintptr = 2
)

const (
    _FUTEX_PRIVATE_FLAG = 128
    _FUTEX_WAIT_PRIVATE = 0 | _FUTEX_PRIVATE_FLAG
    _FUTEX_WAKE_PRIVATE = 1 | _FUTEX_PRIVATE_FLAG
)

Clone, the Linux rfork.

const (
    _CLONE_VM             = 0x100
    _CLONE_FS             = 0x200
    _CLONE_FILES          = 0x400
    _CLONE_SIGHAND        = 0x800
    _CLONE_PTRACE         = 0x2000
    _CLONE_VFORK          = 0x4000
    _CLONE_PARENT         = 0x8000
    _CLONE_THREAD         = 0x10000
    _CLONE_NEWNS          = 0x20000
    _CLONE_SYSVSEM        = 0x40000
    _CLONE_SETTLS         = 0x80000
    _CLONE_PARENT_SETTID  = 0x100000
    _CLONE_CHILD_CLEARTID = 0x200000
    _CLONE_UNTRACED       = 0x800000
    _CLONE_CHILD_SETTID   = 0x1000000
    _CLONE_STOPPED        = 0x2000000
    _CLONE_NEWUTS         = 0x4000000
    _CLONE_NEWIPC         = 0x8000000

    cloneFlags = _CLONE_VM |
        _CLONE_FS |
        _CLONE_FILES |
        _CLONE_SIGHAND |
        _CLONE_SYSVSEM |
        _CLONE_THREAD /* revisit - okay for now */
)

const (
    _AT_NULL   = 0  // End of vector
    _AT_PAGESZ = 6  // System physical page size
    _AT_HWCAP  = 16 // hardware capability bit vector
    _AT_RANDOM = 25 // introduced in 2.6.29
    _AT_HWCAP2 = 26 // hardware capability bit vector 2
)

const (
    _SS_DISABLE  = 2
    _NSIG        = 65
    _SI_USER     = 0
    _SIG_BLOCK   = 0
    _SIG_UNBLOCK = 1
    _SIG_SETMASK = 2
)

const (
    deferHeaderSize = unsafe.Sizeof(_defer{})
    minDeferAlloc   = (deferHeaderSize + 15) &^ 15
    minDeferArgs    = minDeferAlloc - deferHeaderSize
)

Keep a cached value to make gotraceback fast, since we call it on every call to gentraceback. The cached value is a uint32 in which the low bits are the "crash" and "all" settings and the remaining bits are the traceback value (0 off, 1 on, 2 include system).

const (
    tracebackCrash = 1 << iota
    tracebackAll
    tracebackShift = iota
)

defined constants

const (

    // _Gidle means this goroutine was just allocated and has not
    // yet been initialized.
    _Gidle = iota // 0

    // _Grunnable means this goroutine is on a run queue. It is
    // not currently executing user code. The stack is not owned.
    _Grunnable // 1

    // _Grunning means this goroutine may execute user code. The
    // stack is owned by this goroutine. It is not on a run queue.
    // It is assigned an M and a P (g.m and g.m.p are valid).
    _Grunning // 2

    // _Gsyscall means this goroutine is executing a system call.
    // It is not executing user code. The stack is owned by this
    // goroutine. It is not on a run queue. It is assigned an M.
    _Gsyscall // 3

    // _Gwaiting means this goroutine is blocked in the runtime.
    // It is not executing user code. It is not on a run queue,
    // but should be recorded somewhere (e.g., a channel wait
    // queue) so it can be ready()d when necessary. The stack is
    // not owned *except* that a channel operation may read or
    // write parts of the stack under the appropriate channel
    // lock. Otherwise, it is not safe to access the stack after a
    // goroutine enters _Gwaiting (e.g., it may get moved).
    _Gwaiting // 4

    // _Gmoribund_unused is currently unused, but hardcoded in gdb
    // scripts.
    _Gmoribund_unused // 5

    // _Gdead means this goroutine is currently unused. It may be
    // just exited, on a free list, or just being initialized. It
    // is not executing user code. It may or may not have a stack
    // allocated. The G and its stack (if any) are owned by the M
    // that is exiting the G or that obtained the G from the free
    // list.
    _Gdead // 6

    // _Genqueue_unused is currently unused.
    _Genqueue_unused // 7

    // _Gcopystack means this goroutine's stack is being moved. It
    // is not executing user code and is not on a run queue. The
    // stack is owned by the goroutine that put it in _Gcopystack.
    _Gcopystack // 8

    // _Gpreempted means this goroutine stopped itself for a
    // suspendG preemption. It is like _Gwaiting, but nothing is
    // yet responsible for ready()ing it. Some suspendG must CAS
    // the status to _Gwaiting to take responsibility for
    // ready()ing this G.
    _Gpreempted // 9

    // _Gscan combined with one of the above states other than
    // _Grunning indicates that GC is scanning the stack. The
    // goroutine is not executing user code and the stack is owned
    // by the goroutine that set the _Gscan bit.
    //
    // _Gscanrunning is different: it is used to briefly block
    // state transitions while GC signals the G to scan its own
    // stack. This is otherwise like _Grunning.
    //
    // atomicstatus&~Gscan gives the state the goroutine will
    // return to when the scan completes.
    _Gscan          = 0x1000
    _Gscanrunnable  = _Gscan + _Grunnable  // 0x1001
    _Gscanrunning   = _Gscan + _Grunning   // 0x1002
    _Gscansyscall   = _Gscan + _Gsyscall   // 0x1003
    _Gscanwaiting   = _Gscan + _Gwaiting   // 0x1004
    _Gscanpreempted = _Gscan + _Gpreempted // 0x1009
)

const (

    // _Pidle means a P is not being used to run user code or the
    // scheduler. Typically, it's on the idle P list and available
    // to the scheduler, but it may just be transitioning between
    // other states.
    //
    // The P is owned by the idle list or by whatever is
    // transitioning its state. Its run queue is empty.
    _Pidle = iota

    // _Prunning means a P is owned by an M and is being used to
    // run user code or the scheduler. Only the M that owns this P
    // is allowed to change the P's status from _Prunning. The M
    // may transition the P to _Pidle (if it has no more work to
    // do), _Psyscall (when entering a syscall), or _Pgcstop (to
    // halt for the GC). The M may also hand ownership of the P
    // off directly to another M (e.g., to schedule a locked G).
    _Prunning

    // _Psyscall means a P is not running user code. It has
    // affinity to an M in a syscall but is not owned by it and
    // may be stolen by another M. This is similar to _Pidle but
    // uses lightweight transitions and maintains M affinity.
    //
    // Leaving _Psyscall must be done with a CAS, either to steal
    // or retake the P. Note that there's an ABA hazard: even if
    // an M successfully CASes its original P back to _Prunning
    // after a syscall, it must understand the P may have been
    // used by another M in the interim.
    _Psyscall

    // _Pgcstop means a P is halted for STW and owned by the M
    // that stopped the world. The M that stopped the world
    // continues to use its P, even in _Pgcstop. Transitioning
    // from _Prunning to _Pgcstop causes an M to release its P and
    // park.
    //
    // The P retains its run queue and startTheWorld will restart
    // the scheduler on Ps with non-empty run queues.
    _Pgcstop

    // _Pdead means a P is no longer used (GOMAXPROCS shrank). We
    // reuse Ps if GOMAXPROCS increases. A dead P is mostly
    // stripped of its resources, though a few things remain
    // (e.g., trace buffers).
    _Pdead
)

Values for the flags field of a sigTabT.

const (
    _SigNotify   = 1 << iota // let signal.Notify have signal, even if from kernel
    _SigKill                 // if signal.Notify doesn't take it, exit quietly
    _SigThrow                // if signal.Notify doesn't take it, exit loudly
    _SigPanic                // if the signal is from the kernel, panic
    _SigDefault              // if the signal isn't explicitly requested, don't monitor it
    _SigGoExit               // cause all runtime procs to exit (only used on Plan 9).
    _SigSetStack             // add SA_ONSTACK to libc handler
    _SigUnblock              // always unblock; see blockableSig
    _SigIgn                  // _SIG_DFL action is to ignore the signal
)

const (
    _TraceRuntimeFrames = 1 << iota // include frames for internal runtime functions.
    _TraceTrap                      // the initial PC, SP are from a trap, not a return PC from a call
    _TraceJumpStack                 // if traceback is on a systemstack, resume trace at g that called into it
)

const (
    _SIG_DFL uintptr = 0
    _SIG_IGN uintptr = 1
)

const (
    sigIdle = iota
    sigReceiving
    sigSending
    sigFixup
)

const (
    _MaxSmallSize   = 32768
    smallSizeDiv    = 8
    smallSizeMax    = 1024
    largeSizeDiv    = 128
    _NumSizeClasses = 68
    _PageShift      = 13
)

const (
    mantbits64 uint = 52
    expbits64  uint = 11
    bias64          = -1<<(expbits64-1) + 1

    nan64 uint64 = (1<<expbits64-1)<<mantbits64 + 1<<(mantbits64-1) // quiet NaN, 0 payload
    inf64 uint64 = (1<<expbits64 - 1) << mantbits64
    neg64 uint64 = 1 << (expbits64 + mantbits64)

    mantbits32 uint = 23
    expbits32  uint = 8
    bias32          = -1<<(expbits32-1) + 1

    nan32 uint32 = (1<<expbits32-1)<<mantbits32 + 1<<(mantbits32-1) // quiet NaN, 0 payload
    inf32 uint32 = (1<<expbits32 - 1) << mantbits32
    neg32 uint32 = 1 << (expbits32 + mantbits32)
)

const (
    // StackSystem is a number of additional bytes to add
    // to each stack below the usual guard area for OS-specific
    // purposes like signal handling. Used on Windows, Plan 9,
    // and iOS because they do not use a separate stack.
    _StackSystem = sys.GoosWindows*512*sys.PtrSize + sys.GoosPlan9*512 + sys.GoosIos*sys.GoarchArm64*1024

    // The minimum size of stack used by Go code
    _StackMin = 2048

    // The minimum stack size to allocate.
    // The hackery here rounds FixedStack0 up to a power of 2.
    _FixedStack0 = _StackMin + _StackSystem
    _FixedStack1 = _FixedStack0 - 1
    _FixedStack2 = _FixedStack1 | (_FixedStack1 >> 1)
    _FixedStack3 = _FixedStack2 | (_FixedStack2 >> 2)
    _FixedStack4 = _FixedStack3 | (_FixedStack3 >> 4)
    _FixedStack5 = _FixedStack4 | (_FixedStack4 >> 8)
    _FixedStack6 = _FixedStack5 | (_FixedStack5 >> 16)
    _FixedStack  = _FixedStack6 + 1

    // Functions that need frames bigger than this use an extra
    // instruction to do the stack split check, to avoid overflow
    // in case SP - framesize wraps below zero.
    // This value can be no bigger than the size of the unmapped
    // space at zero.
    _StackBig = 4096

    // The stack guard is a pointer this many bytes above the
    // bottom of the stack.
    _StackGuard = 928*sys.StackGuardMultiplier + _StackSystem

    // After a stack split check the SP is allowed to be this
    // many bytes below the stack guard. This saves an instruction
    // in the checking sequence for tiny frames.
    _StackSmall = 128

    // The maximum number of bytes that a chain of NOSPLIT
    // functions can use.
    _StackLimit = _StackGuard - _StackSystem - _StackSmall
)

const (
    // stackDebug == 0: no logging
    //            == 1: logging of per-stack operations
    //            == 2: logging of per-frame operations
    //            == 3: logging of per-word updates
    //            == 4: logging of per-word reads
    stackDebug       = 0
    stackFromSystem  = 0 // allocate stacks from system memory instead of the heap
    stackFaultOnFree = 0 // old stacks are mapped noaccess to detect use after free
    stackPoisonCopy  = 0 // fill stack that should not be accessed with garbage, to detect bad dereferences during copy
    stackNoCache     = 0 // disable per-P small stack caches

    // check the BP links during traceback.
    debugCheckBP = false
)

const (
    uintptrMask = 1<<(8*sys.PtrSize) - 1

    // Goroutine preemption request.
    // Stored into g->stackguard0 to cause split stack check failure.
    // Must be greater than any real sp.
    // 0xfffffade in hex.
    stackPreempt = uintptrMask & -1314

    // Thread is forking.
    // Stored into g->stackguard0 to cause split stack check failure.
    // Must be greater than any real sp.
    stackFork = uintptrMask & -1234
)

const (
    maxUint = ^uint(0)
    maxInt  = int(maxUint >> 1)
)

PCDATA and FUNCDATA table indexes.

See funcdata.h and ../cmd/internal/objabi/funcdata.go.

const (
    _PCDATA_UnsafePoint   = 0
    _PCDATA_StackMapIndex = 1
    _PCDATA_InlTreeIndex  = 2

    _FUNCDATA_ArgsPointerMaps    = 0
    _FUNCDATA_LocalsPointerMaps  = 1
    _FUNCDATA_StackObjects       = 2
    _FUNCDATA_InlTree            = 3
    _FUNCDATA_OpenCodedDeferInfo = 4

    _ArgsSizeUnknown = -0x80000000
)

const (
    // PCDATA_UnsafePoint values.
    _PCDATA_UnsafePointSafe   = -1 // Safe for async preemption
    _PCDATA_UnsafePointUnsafe = -2 // Unsafe for async preemption

    // _PCDATA_Restart1(2) apply on a sequence of instructions, within
    // which if an async preemption happens, we should back off the PC
    // to the start of the sequence when resume.
    // We need two so we can distinguish the start/end of the sequence
    // in case that two sequences are next to each other.
    _PCDATA_Restart1 = -3
    _PCDATA_Restart2 = -4

    // Like _PCDATA_RestartAtEntry, but back to function entry if async
    // preempted.
    _PCDATA_RestartAtEntry = -5
)

Values for the timer status field.

const (
    // Timer has no status set yet.
    timerNoStatus = iota

    // Waiting for timer to fire.
    // The timer is in some P's heap.
    timerWaiting

    // Running the timer function.
    // A timer will only have this status briefly.
    timerRunning

    // The timer is deleted and should be removed.
    // It should not be run, but it is still in some P's heap.
    timerDeleted

    // The timer is being removed.
    // The timer will only have this status briefly.
    timerRemoving

    // The timer has been stopped.
    // It is not in any P's heap.
    timerRemoved

    // The timer is being modified.
    // The timer will only have this status briefly.
    timerModifying

    // The timer has been modified to an earlier time.
    // The new when value is in the nextwhen field.
    // The timer is in some P's heap, possibly in the wrong place.
    timerModifiedEarlier

    // The timer has been modified to the same or a later time.
    // The new when value is in the nextwhen field.
    // The timer is in some P's heap, possibly in the wrong place.
    timerModifiedLater

    // The timer has been modified and is being moved.
    // The timer will only have this status briefly.
    timerMoving
)

Event types in the trace, args are given in square brackets.

const (
    traceEvNone              = 0  // unused
    traceEvBatch             = 1  // start of per-P batch of events [pid, timestamp]
    traceEvFrequency         = 2  // contains tracer timer frequency [frequency (ticks per second)]
    traceEvStack             = 3  // stack [stack id, number of PCs, array of {PC, func string ID, file string ID, line}]
    traceEvGomaxprocs        = 4  // current value of GOMAXPROCS [timestamp, GOMAXPROCS, stack id]
    traceEvProcStart         = 5  // start of P [timestamp, thread id]
    traceEvProcStop          = 6  // stop of P [timestamp]
    traceEvGCStart           = 7  // GC start [timestamp, seq, stack id]
    traceEvGCDone            = 8  // GC done [timestamp]
    traceEvGCSTWStart        = 9  // GC STW start [timestamp, kind]
    traceEvGCSTWDone         = 10 // GC STW done [timestamp]
    traceEvGCSweepStart      = 11 // GC sweep start [timestamp, stack id]
    traceEvGCSweepDone       = 12 // GC sweep done [timestamp, swept, reclaimed]
    traceEvGoCreate          = 13 // goroutine creation [timestamp, new goroutine id, new stack id, stack id]
    traceEvGoStart           = 14 // goroutine starts running [timestamp, goroutine id, seq]
    traceEvGoEnd             = 15 // goroutine ends [timestamp]
    traceEvGoStop            = 16 // goroutine stops (like in select{}) [timestamp, stack]
    traceEvGoSched           = 17 // goroutine calls Gosched [timestamp, stack]
    traceEvGoPreempt         = 18 // goroutine is preempted [timestamp, stack]
    traceEvGoSleep           = 19 // goroutine calls Sleep [timestamp, stack]
    traceEvGoBlock           = 20 // goroutine blocks [timestamp, stack]
    traceEvGoUnblock         = 21 // goroutine is unblocked [timestamp, goroutine id, seq, stack]
    traceEvGoBlockSend       = 22 // goroutine blocks on chan send [timestamp, stack]
    traceEvGoBlockRecv       = 23 // goroutine blocks on chan recv [timestamp, stack]
    traceEvGoBlockSelect     = 24 // goroutine blocks on select [timestamp, stack]
    traceEvGoBlockSync       = 25 // goroutine blocks on Mutex/RWMutex [timestamp, stack]
    traceEvGoBlockCond       = 26 // goroutine blocks on Cond [timestamp, stack]
    traceEvGoBlockNet        = 27 // goroutine blocks on network [timestamp, stack]
    traceEvGoSysCall         = 28 // syscall enter [timestamp, stack]
    traceEvGoSysExit         = 29 // syscall exit [timestamp, goroutine id, seq, real timestamp]
    traceEvGoSysBlock        = 30 // syscall blocks [timestamp]
    traceEvGoWaiting         = 31 // denotes that goroutine is blocked when tracing starts [timestamp, goroutine id]
    traceEvGoInSyscall       = 32 // denotes that goroutine is in syscall when tracing starts [timestamp, goroutine id]
    traceEvHeapAlloc         = 33 // memstats.heap_live change [timestamp, heap_alloc]
    traceEvNextGC            = 34 // memstats.next_gc change [timestamp, next_gc]
    traceEvTimerGoroutine    = 35 // not currently used; previously denoted timer goroutine [timer goroutine id]
    traceEvFutileWakeup      = 36 // denotes that the previous wakeup of this goroutine was futile [timestamp]
    traceEvString            = 37 // string dictionary entry [ID, length, string]
    traceEvGoStartLocal      = 38 // goroutine starts running on the same P as the last event [timestamp, goroutine id]
    traceEvGoUnblockLocal    = 39 // goroutine is unblocked on the same P as the last event [timestamp, goroutine id, stack]
    traceEvGoSysExitLocal    = 40 // syscall exit on the same P as the last event [timestamp, goroutine id, real timestamp]
    traceEvGoStartLabel      = 41 // goroutine starts running with label [timestamp, goroutine id, seq, label string id]
    traceEvGoBlockGC         = 42 // goroutine blocks on GC assist [timestamp, stack]
    traceEvGCMarkAssistStart = 43 // GC mark assist start [timestamp, stack]
    traceEvGCMarkAssistDone  = 44 // GC mark assist done [timestamp]
    traceEvUserTaskCreate    = 45 // trace.NewContext [timestamp, internal task id, internal parent task id, stack, name string]
    traceEvUserTaskEnd       = 46 // end of a task [timestamp, internal task id, stack]
    traceEvUserRegion        = 47 // trace.WithRegion [timestamp, internal task id, mode(0:start, 1:end), stack, name string]
    traceEvUserLog           = 48 // trace.Log [timestamp, internal task id, key string id, stack, value string]
    traceEvCount             = 49
)

const (
    // Timestamps in trace are cputicks/traceTickDiv.
    // This makes absolute values of timestamp diffs smaller,
    // and so they are encoded in less number of bytes.
    // 64 on x86 is somewhat arbitrary (one tick is ~20ns on a 3GHz machine).
    // The suggested increment frequency for PowerPC's time base register is
    // 512 MHz according to Power ISA v2.07 section 6.2, so we use 16 on ppc64
    // and ppc64le.
    // Tracing won't work reliably for architectures where cputicks is emulated
    // by nanotime, so the value doesn't matter for those architectures.
    traceTickDiv = 16 + 48*(sys.Goarch386|sys.GoarchAmd64)
    // Maximum number of PCs in a single stack trace.
    // Since events contain only stack id rather than whole stack trace,
    // we can allow quite large values here.
    traceStackSize = 128
    // Identifier of a fake P that is used when we trace without a real P.
    traceGlobProc = -1
    // Maximum number of bytes to encode uint64 in base-128.
    traceBytesPerNumber = 10
    // Shift of the number of arguments in the first event byte.
    traceArgCountShift = 6
    // Flag passed to traceGoPark to denote that the previous wakeup of this
    // goroutine was futile. For example, a goroutine was unblocked on a mutex,
    // but another goroutine got ahead and acquired the mutex before the first
    // goroutine is scheduled, so the first goroutine has to block again.
    // Such wakeups happen on buffered channels and sync.Mutex,
    // but are generally not interesting for end user.
    traceFutileWakeup byte = 128
)

const (
    kindBool = 1 + iota
    kindInt
    kindInt8
    kindInt16
    kindInt32
    kindInt64
    kindUint
    kindUint8
    kindUint16
    kindUint32
    kindUint64
    kindUintptr
    kindFloat32
    kindFloat64
    kindComplex64
    kindComplex128
    kindArray
    kindChan
    kindFunc
    kindInterface
    kindMap
    kindPtr
    kindSlice
    kindString
    kindStruct
    kindUnsafePointer

    kindDirectIface = 1 << 5
    kindGCProg      = 1 << 6
    kindMask        = (1 << 5) - 1
)

Numbers fundamental to the encoding.

const (
    runeError = '\uFFFD'     // the "error" Rune or "Unicode replacement character"
    runeSelf  = 0x80         // characters below runeSelf are represented as themselves in a single byte.
    maxRune   = '\U0010FFFF' // Maximum valid Unicode code point.
)

Code points in the surrogate range are not valid for UTF-8.

const (
    surrogateMin = 0xD800
    surrogateMax = 0xDFFF
)

const (
    t1 = 0x00 // 0000 0000
    tx = 0x80 // 1000 0000
    t2 = 0xC0 // 1100 0000
    t3 = 0xE0 // 1110 0000
    t4 = 0xF0 // 1111 0000
    t5 = 0xF8 // 1111 1000

    maskx = 0x3F // 0011 1111
    mask2 = 0x1F // 0001 1111
    mask3 = 0x0F // 0000 1111
    mask4 = 0x07 // 0000 0111

    rune1Max = 1<<7 - 1
    rune2Max = 1<<11 - 1
    rune3Max = 1<<16 - 1

    // The default lowest and highest continuation byte.
    locb = 0x80 // 1000 0000
    hicb = 0xBF // 1011 1111
)

const (
    _AT_SYSINFO_EHDR = 33

    _PT_LOAD    = 1 /* Loadable program segment */
    _PT_DYNAMIC = 2 /* Dynamic linking information */

    _DT_NULL     = 0          /* Marks end of dynamic section */
    _DT_HASH     = 4          /* Dynamic symbol hash table */
    _DT_STRTAB   = 5          /* Address of string table */
    _DT_SYMTAB   = 6          /* Address of symbol table */
    _DT_GNU_HASH = 0x6ffffef5 /* GNU-style dynamic symbol hash table */
    _DT_VERSYM   = 0x6ffffff0
    _DT_VERDEF   = 0x6ffffffc

    _VER_FLG_BASE = 0x1 /* Version definition of file itself */

    _SHN_UNDEF = 0 /* Undefined section */

    _SHT_DYNSYM = 11 /* Dynamic linker symbol table */

    _STT_FUNC = 2 /* Symbol is a code object */

    _STT_NOTYPE = 0 /* Symbol type is not specified */

    _STB_GLOBAL = 1 /* Global symbol */
    _STB_WEAK   = 2 /* Weak symbol */

    _EI_NIDENT = 16

    // Maximum indices for the array types used when traversing the vDSO ELF structures.
    // Computed from architecture-specific max provided by vdso_linux_*.go
    vdsoSymTabSize     = vdsoArrayMax / unsafe.Sizeof(elfSym{})
    vdsoDynSize        = vdsoArrayMax / unsafe.Sizeof(elfDyn{})
    vdsoSymStringsSize = vdsoArrayMax     // byte
    vdsoVerSymSize     = vdsoArrayMax / 2 // uint16
    vdsoHashSize       = vdsoArrayMax / 4 // uint32

    // vdsoBloomSizeScale is a scaling factor for gnuhash tables which are uint32 indexed,
    // but contain uintptrs
    vdsoBloomSizeScale = unsafe.Sizeof(uintptr(0)) / 4 // uint32
)

Compiler is the name of the compiler toolchain that built the running binary. Known toolchains are:

gc      Also known as cmd/compile.
gccgo   The gccgo front end, part of the GCC compiler suite.

const Compiler = "gc"

GOARCH is the running program's architecture target: one of 386, amd64, arm, s390x, and so on.

const GOARCH string = sys.GOARCH

GOOS is the running program's operating system target: one of darwin, freebsd, linux, and so on. To view possible combinations of GOOS and GOARCH, run "go tool dist list".

const GOOS string = sys.GOOS

const (
    // Number of goroutine ids to grab from sched.goidgen to local per-P cache at once.
    // 16 seems to provide enough amortization, but other than that it's mostly arbitrary number.
    _GoidCacheBatch = 16
)

The maximum number of frames we print for a traceback

const _TracebackMaxFrames = 100

buffer of pending write data

const (
    bufSize = 4096
)

const cgoCheckPointerFail = "cgo argument has Go pointer to Go pointer"

const cgoResultFail = "cgo result has Go pointer"

const cgoWriteBarrierFail = "Go pointer stored into non-Go memory"

debugLogBytes is the size of each per-M ring buffer. This is allocated off-heap to avoid blowing up the M and hence the GC'd heap size.

const debugLogBytes = 16 << 10

debugLogStringLimit is the maximum number of bytes in a string. Above this, the string will be truncated with "..(n more bytes).."

const debugLogStringLimit = debugLogBytes / 8

const debugPcln = false

const debugSelect = false

defaultHeapMinimum is the value of heapminimum for GOGC==100.

const defaultHeapMinimum = 4 << 20

const dlogEnabled = false

const fastlogNumBits = 5

forcePreemptNS is the time slice given to a G before it is preempted.

const forcePreemptNS = 10 * 1000 * 1000 // 10ms

Must agree with cmd/internal/objabi.Framepointer_enabled.

const framepointer_enabled = GOARCH == "amd64" || GOARCH == "arm64" && (GOOS == "linux" || GOOS == "darwin" || GOOS == "ios")

freezeStopWait is a large value that freezetheworld sets sched.stopwait to in order to request that all Gs permanently stop.

const freezeStopWait = 0x7fffffff

gcAssistTimeSlack is the nanoseconds of mutator assist time that can accumulate on a P before updating gcController.assistTime.

const gcAssistTimeSlack = 5000

gcBackgroundUtilization is the fixed CPU utilization for background marking. It must be <= gcGoalUtilization. The difference between gcGoalUtilization and gcBackgroundUtilization will be made up by mark assists. The scheduler will aim to use within 50% of this goal.

Setting this to < gcGoalUtilization avoids saturating the trigger feedback controller when there are no assists, which allows it to better control CPU and heap growth. However, the larger the gap, the more mutator assists are expected to happen, which impact mutator latency.

const gcBackgroundUtilization = 0.25

const gcBitsChunkBytes = uintptr(64 << 10)

const gcBitsHeaderBytes = unsafe.Sizeof(gcBitsHeader{})

gcCreditSlack is the amount of scan work credit that can accumulate locally before updating gcController.scanWork and, optionally, gcController.bgScanCredit. Lower values give a more accurate assist ratio and make it more likely that assists will successfully steal background credit. Higher values reduce memory contention.

const gcCreditSlack = 2000

gcGoalUtilization is the goal CPU utilization for marking as a fraction of GOMAXPROCS.

const gcGoalUtilization = 0.30

gcOverAssistWork determines how many extra units of scan work a GC assist does when an assist happens. This amortizes the cost of an assist by pre-paying for this many bytes of future allocations.

const gcOverAssistWork = 64 << 10

const hashRandomBytes = sys.PtrSize / 4 * 64

const itabInitSize = 512

const mProfCycleWrap = uint32(len(memRecord{}.future)) * (2 << 24)

const maxCPUProfStack = 64

maxWhen is the maximum value for timer's when field.

const maxWhen = 1<<63 - 1

const maxZero = 1024 // must match value in reflect/value.go:maxZero cmd/compile/internal/gc/walk.go:zeroValSize

const minfunc = 16 // minimum function size

const msanenabled = false

osRelaxMinNS is the number of nanoseconds of idleness to tolerate without performing an osRelax. Since osRelax may reduce the precision of timers, this should be enough larger than the relaxed timer precision to keep the timer error acceptable.

const osRelaxMinNS = 0

const pageCachePages = 8 * unsafe.Sizeof(pageCache{}.cache)

const pcbucketsize = 256 * minfunc // size of bucket in the pc->func lookup table

persistentChunkSize is the number of bytes we allocate when we grow a persistentAlloc.

const persistentChunkSize = 256 << 10

const pollBlockSize = 4 * 1024

const preemptMSupported = true

const raceenabled = false

To shake out latent assumptions about scheduling order, we introduce some randomness into scheduling decisions when running with the race detector. The need for this was made obvious by changing the (deterministic) scheduling order in Go 1.5 and breaking many poorly-written tests. With the randomness here, as long as the tests pass consistently with -race, they shouldn't have latent scheduling assumptions.

const randomizeScheduler = raceenabled

const rwmutexMaxReaders = 1 << 30

Prime to not correlate with any user patterns.

const semTabSize = 251

sigPreempt is the signal used for non-cooperative preemption.

There's no good way to choose this signal, but there are some heuristics:

1. It should be a signal that's passed-through by debuggers by default. On Linux, this is SIGALRM, SIGURG, SIGCHLD, SIGIO, SIGVTALRM, SIGPROF, and SIGWINCH, plus some glibc-internal signals.

2. It shouldn't be used internally by libc in mixed Go/C binaries because libc may assume it's the only thing that can handle these signals. For example SIGCANCEL or SIGSETXID.

3. It should be a signal that can happen spuriously without consequences. For example, SIGALRM is a bad choice because the signal handler can't tell if it was caused by the real process alarm or not (arguably this means the signal is broken, but I digress). SIGUSR1 and SIGUSR2 are also bad because those are often used in meaningful ways by applications.

4. We need to deal with platforms without real-time signals (like macOS), so those are out.

We use SIGURG because it meets all of these criteria, is extremely unlikely to be used by an application for its "real" meaning (both because out-of-band data is basically unused and because SIGURG doesn't report which socket has the condition, making it pretty useless), and even if it is, the application has to be ready for spurious SIGURG. SIGIO wouldn't be a bad choice either, but is more likely to be used for real.

const sigPreempt = _SIGURG

const sizeofSkipFunction = 256

const stackTraceDebug = false

testSmallBuf forces a small write barrier buffer to stress write barrier flushing.

const testSmallBuf = false

The constant is known to the compiler. There is no fundamental theory behind this number.

const tmpStringBufSize = 32

const usesLR = sys.MinFrameSize > 0

const (
    // vdsoArrayMax is the byte-size of a maximally sized array on this architecture.
    // See cmd/compile/internal/amd64/galign.go arch.MAXWIDTH initialization.
    vdsoArrayMax = 1<<50 - 1
)

verifyTimers can be set to true to add debugging checks that the timer heaps are valid.

const verifyTimers = false

Variables

var (
    _cgo_init                     unsafe.Pointer
    _cgo_thread_start             unsafe.Pointer
    _cgo_sys_thread_create        unsafe.Pointer
    _cgo_notify_runtime_init_done unsafe.Pointer
    _cgo_callers                  unsafe.Pointer
    _cgo_set_context_function     unsafe.Pointer
    _cgo_yield                    unsafe.Pointer
)

var (
    // Set in runtime.cpuinit.
    // TODO: deprecate these; use internal/cpu directly.
    x86HasPOPCNT bool
    x86HasSSE41  bool
    x86HasFMA    bool

    armHasVFPv4 bool

    arm64HasATOMICS bool
)

var (
    itabLock      mutex                               // lock for accessing itab table
    itabTable     = &itabTableInit                    // pointer to current table
    itabTableInit = itabTableType{size: itabInitSize} // starter table
)

var (
    uint16Eface interface{} = uint16InterfacePtr(0)
    uint32Eface interface{} = uint32InterfacePtr(0)
    uint64Eface interface{} = uint64InterfacePtr(0)
    stringEface interface{} = stringInterfacePtr("")
    sliceEface  interface{} = sliceInterfacePtr(nil)

    uint16Type *_type = efaceOf(&uint16Eface)._type
    uint32Type *_type = efaceOf(&uint32Eface)._type
    uint64Type *_type = efaceOf(&uint64Eface)._type
    stringType *_type = efaceOf(&stringEface)._type
    sliceType  *_type = efaceOf(&sliceEface)._type
)

physHugePageSize is the size in bytes of the OS's default physical huge page size whose allocation is opaque to the application. It is assumed and verified to be a power of two.

If set, this must be set by the OS init code (typically in osinit) before mallocinit. However, setting it at all is optional, and leaving the default value is always safe (though potentially less efficient).

Since physHugePageSize is always assumed to be a power of two, physHugePageShift is defined as physHugePageSize == 1 << physHugePageShift. The purpose of physHugePageShift is to avoid doing divisions in performance critical functions.

var (
    physHugePageSize  uintptr
    physHugePageShift uint
)

var (
    // metrics is a map of runtime/metrics keys to
    // data used by the runtime to sample each metric's
    // value.
    metricsSema uint32 = 1
    metricsInit bool
    metrics     map[string]metricData

    sizeClassBuckets []float64
    timeHistBuckets  []float64
)

var (
    fingCreate  uint32
    fingRunning bool
)

var (
    mbuckets  *bucket // memory profile buckets
    bbuckets  *bucket // blocking profile buckets
    xbuckets  *bucket // mutex profile buckets
    buckhash  *[179999]*bucket
    bucketmem uintptr

    mProf struct {

        // cycle is the global heap profile cycle. This wraps
        // at mProfCycleWrap.
        cycle uint32
        // flushed indicates that future[cycle] in all buckets
        // has been flushed to the active profile.
        flushed bool
    }
)

var (
    // minOffAddr is the minimum address in the offset space, and
    // it corresponds to the virtual address arenaBaseOffset.
    minOffAddr = offAddr{arenaBaseOffset}

    // maxOffAddr is the maximum address in the offset address
    // space. It corresponds to the highest virtual address representable
    // by the page alloc chunk and heap arena maps.
    maxOffAddr = offAddr{(((1 << heapAddrBits) - 1) + arenaBaseOffset) & uintptrMask}
)

var (
    netpollInitLock mutex
    netpollInited   uint32

    pollcache      pollCache
    netpollWaiters uint32
)

var (
    epfd int32 = -1 // epoll descriptor

    netpollBreakRd, netpollBreakWr uintptr // for netpollBreak

    netpollWakeSig uint32 // used to avoid duplicate calls of netpollBreak
)

var (
    // printBacklog is a circular buffer of messages written with the builtin
    // print* functions, for use in postmortem analysis of core dumps.
    printBacklog      [512]byte
    printBacklogIndex int
)

var (
    m0           m
    g0           g
    mcache0      *mcache
    raceprocctx0 uintptr
)

var (
    // allgs contains all Gs ever created (including dead Gs), and thus
    // never shrinks.
    //
    // Access via the slice is protected by allglock or stop-the-world.
    // Readers that cannot take the lock may (carefully!) use the atomic
    // variables below.
    allglock mutex
    allgs    []*g

    // allglen and allgptr are atomic variables that contain len(allg) and
    // &allg[0] respectively. Proper ordering depends on totally-ordered
    // loads and stores. Writes are protected by allglock.
    //
    // allgptr is updated before allglen. Readers should read allglen
    // before allgptr to ensure that allglen is always <= len(allgptr). New
    // Gs appended during the race can be missed. For a consistent view of
    // all Gs, allglock must be held.
    //
    // allgptr copies should always be stored as a concrete type or
    // unsafe.Pointer, not uintptr, to ensure that GC can still reach it
    // even if it points to a stale array.
    allglen uintptr
    allgptr **g
)

var (
    argc int32
    argv **byte
)

var (
    allm       *m
    gomaxprocs int32
    ncpu       int32
    forcegc    forcegcstate
    sched      schedt
    newprocs   int32

    // allpLock protects P-less reads and size changes of allp, idlepMask,
    // and timerpMask, and all writes to allp.
    allpLock mutex
    // len(allp) == gomaxprocs; may change at safe points, otherwise
    // immutable.
    allp []*p
    // Bitmask of Ps in _Pidle list, one bit per P. Reads and writes must
    // be atomic. Length may change at safe points.
    //
    // Each P must update only its own bit. In order to maintain
    // consistency, a P going idle must the idle mask simultaneously with
    // updates to the idle P list under the sched.lock, otherwise a racing
    // pidleget may clear the mask before pidleput sets the mask,
    // corrupting the bitmap.
    //
    // N.B., procresize takes ownership of all Ps in stopTheWorldWithSema.
    idlepMask pMask
    // Bitmask of Ps that may have a timer, one bit per P. Reads and writes
    // must be atomic. Length may change at safe points.
    timerpMask pMask

    // Pool of GC parked background workers. Entries are type
    // *gcBgMarkWorkerNode.
    gcBgMarkWorkerPool lfstack

    // Total number of gcBgMarkWorker goroutines. Protected by worldsema.
    gcBgMarkWorkerCount int32

    // Information about what cpu features are available.
    // Packages outside the runtime should not use these
    // as they are not an external api.
    // Set on startup in asm_{386,amd64}.s
    processorVersionInfo uint32
    isIntel              bool
    lfenceBeforeRdtsc    bool

    goarm uint8 // set by cmd/link on arm systems
)

Set by the linker so the runtime can determine the buildmode.

var (
    islibrary bool // -buildmode=c-shared
    isarchive bool // -buildmode=c-archive
)

var (
    chansendpc = funcPC(chansend)
    chanrecvpc = funcPC(chanrecv)
)

channels for synchronizing signal mask updates with the signal mask thread

var (
    disableSigChan  chan uint32
    enableSigChan   chan uint32
    maskUpdatedChan chan struct{}
)

var (
    vdsoGettimeofdaySym uintptr
    vdsoClockgettimeSym uintptr
)

MemProfileRate controls the fraction of memory allocations that are recorded and reported in the memory profile. The profiler aims to sample an average of one allocation per MemProfileRate bytes allocated.

To include every allocated block in the profile, set MemProfileRate to 1. To turn off profiling entirely, set MemProfileRate to 0.

The tools that process the memory profiles assume that the profile rate is constant across the lifetime of the program and equal to the current value. Programs that change the memory profiling rate should do so just once, as early as possible in the execution of the program (for example, at the beginning of main).

var MemProfileRate int = 512 * 1024

Make the compiler check that heapBits.arena is large enough to hold the maximum arena frame number.

var _ = heapBits{arena: (1<<heapAddrBits)/heapArenaBytes - 1}

_cgo_mmap is filled in by runtime/cgo when it is linked into the program, so it is only non-nil when using cgo.

var _cgo_mmap unsafe.Pointer

_cgo_munmap is filled in by runtime/cgo when it is linked into the program, so it is only non-nil when using cgo.

var _cgo_munmap unsafe.Pointer

var _cgo_setenv unsafe.Pointer // pointer to C function

_cgo_sigaction is filled in by runtime/cgo when it is linked into the program, so it is only non-nil when using cgo.

var _cgo_sigaction unsafe.Pointer

var _cgo_unsetenv unsafe.Pointer // pointer to C function

var addrspace_vec [1]byte

var adviseUnused = uint32(_MADV_FREE)

used in asm_{386,amd64,arm64}.s to seed the hash function

var aeskeysched [hashRandomBytes]byte

var argslice []string

asyncPreemptStack is the bytes of stack space required to inject an asyncPreempt call.

var asyncPreemptStack = ^uintptr(0)

var badginsignalMsg = "fatal: bad g in signal handler\n"

var badmorestackg0Msg = "fatal: morestack on g0\n"

var badmorestackgsignalMsg = "fatal: morestack on gsignal\n"

var badsystemstackMsg = "fatal: systemstack called from unexpected goroutine"

var blockprofilerate uint64 // in CPU ticks

boundsErrorFmts provide error text for various out-of-bounds panics. Note: if you change these strings, you should adjust the size of the buffer in boundsError.Error below as well.

var boundsErrorFmts = [...]string{
    boundsIndex:      "index out of range [%x] with length %y",
    boundsSliceAlen:  "slice bounds out of range [:%x] with length %y",
    boundsSliceAcap:  "slice bounds out of range [:%x] with capacity %y",
    boundsSliceB:     "slice bounds out of range [%x:%y]",
    boundsSlice3Alen: "slice bounds out of range [::%x] with length %y",
    boundsSlice3Acap: "slice bounds out of range [::%x] with capacity %y",
    boundsSlice3B:    "slice bounds out of range [:%x:%y]",
    boundsSlice3C:    "slice bounds out of range [%x:%y:]",
}

boundsNegErrorFmts are overriding formats if x is negative. In this case there's no need to report y.

var boundsNegErrorFmts = [...]string{
    boundsIndex:      "index out of range [%x]",
    boundsSliceAlen:  "slice bounds out of range [:%x]",
    boundsSliceAcap:  "slice bounds out of range [:%x]",
    boundsSliceB:     "slice bounds out of range [%x:]",
    boundsSlice3Alen: "slice bounds out of range [::%x]",
    boundsSlice3Acap: "slice bounds out of range [::%x]",
    boundsSlice3B:    "slice bounds out of range [:%x:]",
    boundsSlice3C:    "slice bounds out of range [%x::]",
}

var buf [bufSize]byte

var buildVersion = sys.TheVersion

cgoAlwaysFalse is a boolean value that is always false. The cgo-generated code says if cgoAlwaysFalse { cgoUse(p) }. The compiler cannot see that cgoAlwaysFalse is always false, so it emits the test and keeps the call, giving the desired escape analysis result. The test is cheaper than the call.

var cgoAlwaysFalse bool

var cgoContext unsafe.Pointer

cgoHasExtraM is set on startup when an extra M is created for cgo. The extra M must be created before any C/C++ code calls cgocallback.

var cgoHasExtraM bool

var cgoSymbolizer unsafe.Pointer

When running with cgo, we call _cgo_thread_start to start threads for us so that we can play nicely with foreign code.

var cgoThreadStart unsafe.Pointer

var cgoTraceback unsafe.Pointer

var cgo_yield = &_cgo_yield

var class_to_allocnpages = [_NumSizeClasses]uint8{0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2, 1, 2, 1, 3, 2, 3, 1, 3, 2, 3, 4, 5, 6, 1, 7, 6, 5, 4, 3, 5, 7, 2, 9, 7, 5, 8, 3, 10, 7, 4}

var class_to_divmagic = [_NumSizeClasses]divMagic{{0, 0, 0, 0}, {3, 0, 1, 65528}, {4, 0, 1, 65520}, {3, 11, 683, 0}, {5, 0, 1, 65504}, {4, 11, 683, 0}, {6, 0, 1, 65472}, {4, 10, 205, 0}, {5, 9, 171, 0}, {4, 11, 293, 0}, {7, 0, 1, 65408}, {4, 13, 911, 0}, {5, 10, 205, 0}, {4, 12, 373, 0}, {6, 9, 171, 0}, {4, 13, 631, 0}, {5, 11, 293, 0}, {4, 13, 547, 0}, {8, 0, 1, 65280}, {5, 9, 57, 0}, {6, 9, 103, 0}, {5, 12, 373, 0}, {7, 7, 43, 0}, {5, 10, 79, 0}, {6, 10, 147, 0}, {5, 11, 137, 0}, {9, 0, 1, 65024}, {6, 9, 57, 0}, {7, 9, 103, 0}, {6, 11, 187, 0}, {8, 7, 43, 0}, {7, 8, 37, 0}, {10, 0, 1, 64512}, {7, 9, 57, 0}, {8, 6, 13, 0}, {7, 11, 187, 0}, {9, 5, 11, 0}, {8, 8, 37, 0}, {11, 0, 1, 63488}, {8, 9, 57, 0}, {7, 10, 49, 0}, {10, 5, 11, 0}, {7, 10, 41, 0}, {7, 9, 19, 0}, {12, 0, 1, 61440}, {8, 9, 27, 0}, {8, 10, 49, 0}, {11, 5, 11, 0}, {7, 13, 161, 0}, {7, 13, 155, 0}, {8, 9, 19, 0}, {13, 0, 1, 57344}, {8, 12, 111, 0}, {9, 9, 27, 0}, {11, 6, 13, 0}, {7, 14, 193, 0}, {12, 3, 3, 0}, {8, 13, 155, 0}, {11, 8, 37, 0}, {14, 0, 1, 49152}, {11, 8, 29, 0}, {7, 13, 55, 0}, {12, 5, 7, 0}, {8, 14, 193, 0}, {13, 3, 3, 0}, {7, 14, 77, 0}, {12, 7, 19, 0}, {15, 0, 1, 32768}}

var class_to_size = [_NumSizeClasses]uint16{0, 8, 16, 24, 32, 48, 64, 80, 96, 112, 128, 144, 160, 176, 192, 208, 224, 240, 256, 288, 320, 352, 384, 416, 448, 480, 512, 576, 640, 704, 768, 896, 1024, 1152, 1280, 1408, 1536, 1792, 2048, 2304, 2688, 3072, 3200, 3456, 4096, 4864, 5376, 6144, 6528, 6784, 6912, 8192, 9472, 9728, 10240, 10880, 12288, 13568, 14336, 16384, 18432, 19072, 20480, 21760, 24576, 27264, 28672, 32768}

crashing is the number of m's we have waited for when implementing GOTRACEBACK=crash when a signal is received.

var crashing int32

var dbgvars = []dbgVar{
    {"allocfreetrace", &debug.allocfreetrace},
    {"clobberfree", &debug.clobberfree},
    {"cgocheck", &debug.cgocheck},
    {"efence", &debug.efence},
    {"gccheckmark", &debug.gccheckmark},
    {"gcpacertrace", &debug.gcpacertrace},
    {"gcshrinkstackoff", &debug.gcshrinkstackoff},
    {"gcstoptheworld", &debug.gcstoptheworld},
    {"gctrace", &debug.gctrace},
    {"invalidptr", &debug.invalidptr},
    {"madvdontneed", &debug.madvdontneed},
    {"sbrk", &debug.sbrk},
    {"scavenge", &debug.scavenge},
    {"scavtrace", &debug.scavtrace},
    {"scheddetail", &debug.scheddetail},
    {"schedtrace", &debug.schedtrace},
    {"tracebackancestors", &debug.tracebackancestors},
    {"asyncpreemptoff", &debug.asyncpreemptoff},
    {"inittrace", &debug.inittrace},
}

Holds variables parsed from GODEBUG env var, except for "memprofilerate" since there is an existing int var for that value, which may already have an initial value.

var debug struct {
    cgocheck           int32
    clobberfree        int32
    efence             int32
    gccheckmark        int32
    gcpacertrace       int32
    gcshrinkstackoff   int32
    gcstoptheworld     int32
    gctrace            int32
    invalidptr         int32
    madvdontneed       int32 // for Linux; issue 28466
    scavenge           int32
    scavtrace          int32
    scheddetail        int32
    schedtrace         int32
    tracebackancestors int32
    asyncpreemptoff    int32

    // debug.malloc is used as a combined debug check
    // in the malloc function and should be set
    // if any of the below debug options is != 0.
    malloc         bool
    allocfreetrace int32
    inittrace      int32
    sbrk           int32
}

var debugPtrmask struct {
    lock mutex
    data *byte
}

var didothers bool

var divideError = error(errorString("integer divide by zero"))

var dumpfd uintptr // fd to write the dump to.

var dumphdr = []byte("go1.7 heap dump\n")

var earlycgocallback = []byte("fatal error: cgo callback before cgo call\n")

var envs []string

var extraMCount uint32 // Protected by lockextra

var extraMWaiters uint32

var extram uintptr

var failallocatestack = []byte("runtime: failed to allocate stack for the new OS thread\n")

var failthreadcreate = []byte("runtime: failed to create new OS thread\n")

faketime is the simulated time in nanoseconds since 1970 for the playground.

Zero means not to use faketime.

var faketime int64

var fastlog2Table = [1<<fastlogNumBits + 1]float64{
    0,
    0.0443941193584535,
    0.08746284125033943,
    0.12928301694496647,
    0.16992500144231248,
    0.2094533656289499,
    0.24792751344358555,
    0.28540221886224837,
    0.3219280948873623,
    0.3575520046180837,
    0.39231742277876036,
    0.4262647547020979,
    0.4594316186372973,
    0.4918530963296748,
    0.5235619560570128,
    0.5545888516776374,
    0.5849625007211563,
    0.6147098441152082,
    0.6438561897747247,
    0.6724253419714956,
    0.7004397181410922,
    0.7279204545631992,
    0.7548875021634686,
    0.7813597135246596,
    0.8073549220576042,
    0.8328900141647417,
    0.8579809951275721,
    0.8826430493618412,
    0.9068905956085185,
    0.9307373375628862,
    0.9541963103868752,
    0.9772799234999164,
    1,
}

var fastrandseed uintptr

var finalizer1 = [...]byte{

    1<<0 | 1<<1 | 0<<2 | 1<<3 | 1<<4 | 1<<5 | 1<<6 | 0<<7,
    1<<0 | 1<<1 | 1<<2 | 1<<3 | 0<<4 | 1<<5 | 1<<6 | 1<<7,
    1<<0 | 0<<1 | 1<<2 | 1<<3 | 1<<4 | 1<<5 | 0<<6 | 1<<7,
    1<<0 | 1<<1 | 1<<2 | 0<<3 | 1<<4 | 1<<5 | 1<<6 | 1<<7,
    0<<0 | 1<<1 | 1<<2 | 1<<3 | 1<<4 | 0<<5 | 1<<6 | 1<<7,
}

var fingwait bool

var fingwake bool

var finptrmask [_FinBlockSize / sys.PtrSize / 8]byte

var floatError = error(errorString("floating point error"))

forcegcperiod is the maximum time in nanoseconds between garbage collections. If we go this long without a garbage collection, one is forced to run.

This is a variable for testing purposes. It normally doesn't change.

var forcegcperiod int64 = 2 * 60 * 1e9

Bit vector of free marks. Needs to be as big as the largest number of objects per span.

var freemark [_PageSize / 8]bool

freezing is set to non-zero if the runtime is trying to freeze the world.

var freezing uint32

Stores the signal handlers registered before Go installed its own. These signal handlers will be invoked in cases where Go doesn't want to handle a particular signal (e.g., signal occurred on a non-Go thread). See sigfwdgo for more information on when the signals are forwarded.

This is read by the signal handler; accesses should use atomic.Loaduintptr and atomic.Storeuintptr.

var fwdSig [_NSIG]uintptr

var gStatusStrings = [...]string{
    _Gidle:      "idle",
    _Grunnable:  "runnable",
    _Grunning:   "running",
    _Gsyscall:   "syscall",
    _Gwaiting:   "waiting",
    _Gdead:      "dead",
    _Gcopystack: "copystack",
    _Gpreempted: "preempted",
}

var gcBitsArenas struct {
    lock     mutex
    free     *gcBitsArena
    next     *gcBitsArena // Read atomically. Write atomically under lock.
    current  *gcBitsArena
    previous *gcBitsArena
}

gcBlackenEnabled is 1 if mutator assists and background mark workers are allowed to blacken objects. This must only be set when gcphase == _GCmark.

var gcBlackenEnabled uint32

gcMarkDoneFlushed counts the number of P's with flushed work.

Ideally this would be a captured local in gcMarkDone, but forEachP escapes its callback closure, so it can't capture anything.

This is protected by markDoneSema.

var gcMarkDoneFlushed uint32

gcMarkWorkerModeStrings are the strings labels of gcMarkWorkerModes to use in execution traces.

var gcMarkWorkerModeStrings = [...]string{
    "Not worker",
    "GC (dedicated)",
    "GC (fractional)",
    "GC (idle)",
}

Initialized from $GOGC. GOGC=off means no GC.

var gcpercent int32

Garbage collector phase. Indicates to write barrier and synchronization task to perform.

var gcphase uint32

Holding gcsema grants the M the right to block a GC, and blocks until the current GC is done. In particular, it prevents gomaxprocs from changing concurrently.

TODO(mknyszek): Once gomaxprocs and the execution tracer can handle being changed/enabled during a GC, remove this.

var gcsema uint32 = 1

var globalAlloc struct {
    mutex
    persistentAlloc
}

handlingSig is indexed by signal number and is non-zero if we are currently handling the signal. Or, to put it another way, whether the signal handler is currently set to the Go signal handler or not. This is uint32 rather than bool so that we can use atomic instructions.

var handlingSig [_NSIG]uint32

exported value for testing

var hashLoad = float32(loadFactorNum) / float32(loadFactorDen)

used in hash{32,64}.go to seed the hash function

var hashkey [4]uintptr

heapminimum is the minimum heap size at which to trigger GC. For small heaps, this overrides the usual GOGC*live set rule.

When there is a very small live set but a lot of allocation, simply collecting when the heap reaches GOGC*live results in many GC cycles and high total per-GC overhead. This minimum amortizes this per-GC overhead while keeping the heap reasonably small.

During initialization this is set to 4MB*GOGC/100. In the case of GOGC==0, this will set heapminimum to 0, resulting in constant collection even when the heap size is small, which is useful for debugging.

var heapminimum uint64 = defaultHeapMinimum

inForkedChild is true while manipulating signals in the child process. This is used to avoid calling libc functions in case we are using vfork.

var inForkedChild bool

var inf = float64frombits(0x7FF0000000000000)

iscgo is set to true by the runtime/cgo package

var iscgo bool

var labelSync uintptr

levelBits is the number of bits in the radix for a given level in the super summary structure.

The sum of all the entries of levelBits should equal heapAddrBits.

var levelBits = [summaryLevels]uint{
    summaryL0Bits,
    summaryLevelBits,
    summaryLevelBits,
    summaryLevelBits,
    summaryLevelBits,
}

levelLogPages is log2 the maximum number of runtime pages in the address space a summary in the given level represents.

The leaf level always represents exactly log2 of 1 chunk's worth of pages.

var levelLogPages = [summaryLevels]uint{
    logPallocChunkPages + 4*summaryLevelBits,
    logPallocChunkPages + 3*summaryLevelBits,
    logPallocChunkPages + 2*summaryLevelBits,
    logPallocChunkPages + 1*summaryLevelBits,
    logPallocChunkPages,
}

levelShift is the number of bits to shift to acquire the radix for a given level in the super summary structure.

With levelShift, one can compute the index of the summary at level l related to a pointer p by doing:

p >> levelShift[l]

var levelShift = [summaryLevels]uint{
    heapAddrBits - summaryL0Bits,
    heapAddrBits - summaryL0Bits - 1*summaryLevelBits,
    heapAddrBits - summaryL0Bits - 2*summaryLevelBits,
    heapAddrBits - summaryL0Bits - 3*summaryLevelBits,
    heapAddrBits - summaryL0Bits - 4*summaryLevelBits,
}

lockNames gives the names associated with each of the above ranks

var lockNames = []string{
    lockRankDummy: "",

    lockRankSysmon:       "sysmon",
    lockRankScavenge:     "scavenge",
    lockRankForcegc:      "forcegc",
    lockRankSweepWaiters: "sweepWaiters",
    lockRankAssistQueue:  "assistQueue",
    lockRankCpuprof:      "cpuprof",
    lockRankSweep:        "sweep",

    lockRankPollDesc: "pollDesc",
    lockRankSched:    "sched",
    lockRankDeadlock: "deadlock",
    lockRankPanic:    "panic",
    lockRankAllg:     "allg",
    lockRankAllp:     "allp",

    lockRankTimers:      "timers",
    lockRankItab:        "itab",
    lockRankReflectOffs: "reflectOffs",

    lockRankHchan:         "hchan",
    lockRankFin:           "fin",
    lockRankNotifyList:    "notifyList",
    lockRankTraceBuf:      "traceBuf",
    lockRankTraceStrings:  "traceStrings",
    lockRankMspanSpecial:  "mspanSpecial",
    lockRankProf:          "prof",
    lockRankGcBitsArenas:  "gcBitsArenas",
    lockRankRoot:          "root",
    lockRankTrace:         "trace",
    lockRankTraceStackTab: "traceStackTab",
    lockRankNetpollInit:   "netpollInit",

    lockRankRwmutexW: "rwmutexW",
    lockRankRwmutexR: "rwmutexR",

    lockRankSpanSetSpine: "spanSetSpine",
    lockRankGscan:        "gscan",
    lockRankStackpool:    "stackpool",
    lockRankStackLarge:   "stackLarge",
    lockRankDefer:        "defer",
    lockRankSudog:        "sudog",

    lockRankWbufSpans:    "wbufSpans",
    lockRankMheap:        "mheap",
    lockRankMheapSpecial: "mheapSpecial",

    lockRankGlobalAlloc: "globalAlloc.mutex",

    lockRankGFree:     "gFree",
    lockRankHchanLeaf: "hchanLeaf",

    lockRankNewmHandoff:   "newmHandoff.lock",
    lockRankDebugPtrmask:  "debugPtrmask.lock",
    lockRankFaketimeState: "faketimeState.lock",
    lockRankTicks:         "ticks.lock",
    lockRankRaceFini:      "raceFiniLock",
    lockRankPollCache:     "pollCache.lock",
    lockRankDebug:         "debugLock",
}

lockPartialOrder is a partial order among the various lock types, listing the immediate ordering that has actually been observed in the runtime. Each entry (which corresponds to a particular lock rank) specifies the list of locks that can already be held immediately "above" it.

So, for example, the lockRankSched entry shows that all the locks preceding it in rank can actually be held. The allp lock shows that only the sysmon or sched lock can be held immediately above it when it is acquired.

var lockPartialOrder [][]lockRank = [][]lockRank{
    lockRankDummy:         {},
    lockRankSysmon:        {},
    lockRankScavenge:      {lockRankSysmon},
    lockRankForcegc:       {lockRankSysmon},
    lockRankSweepWaiters:  {},
    lockRankAssistQueue:   {},
    lockRankCpuprof:       {},
    lockRankSweep:         {},
    lockRankPollDesc:      {},
    lockRankSched:         {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankCpuprof, lockRankSweep, lockRankPollDesc},
    lockRankDeadlock:      {lockRankDeadlock},
    lockRankPanic:         {lockRankDeadlock},
    lockRankAllg:          {lockRankSysmon, lockRankSched, lockRankPanic},
    lockRankAllp:          {lockRankSysmon, lockRankSched},
    lockRankTimers:        {lockRankSysmon, lockRankScavenge, lockRankSched, lockRankAllp, lockRankPollDesc, lockRankTimers},
    lockRankItab:          {},
    lockRankReflectOffs:   {lockRankItab},
    lockRankHchan:         {lockRankScavenge, lockRankSweep, lockRankHchan},
    lockRankFin:           {lockRankSysmon, lockRankScavenge, lockRankSched, lockRankAllg, lockRankTimers, lockRankHchan},
    lockRankNotifyList:    {},
    lockRankTraceBuf:      {lockRankSysmon, lockRankScavenge},
    lockRankTraceStrings:  {lockRankTraceBuf},
    lockRankMspanSpecial:  {lockRankSysmon, lockRankScavenge, lockRankAssistQueue, lockRankCpuprof, lockRankSweep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankItab, lockRankReflectOffs, lockRankHchan, lockRankNotifyList, lockRankTraceBuf, lockRankTraceStrings},
    lockRankProf:          {lockRankSysmon, lockRankScavenge, lockRankAssistQueue, lockRankCpuprof, lockRankSweep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankItab, lockRankReflectOffs, lockRankNotifyList, lockRankTraceBuf, lockRankTraceStrings, lockRankHchan},
    lockRankGcBitsArenas:  {lockRankSysmon, lockRankScavenge, lockRankAssistQueue, lockRankCpuprof, lockRankSched, lockRankAllg, lockRankTimers, lockRankItab, lockRankReflectOffs, lockRankNotifyList, lockRankTraceBuf, lockRankTraceStrings, lockRankHchan},
    lockRankRoot:          {},
    lockRankTrace:         {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankAssistQueue, lockRankSched, lockRankHchan, lockRankTraceBuf, lockRankTraceStrings, lockRankRoot, lockRankSweep},
    lockRankTraceStackTab: {lockRankScavenge, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankSched, lockRankAllg, lockRankTimers, lockRankHchan, lockRankFin, lockRankNotifyList, lockRankTraceBuf, lockRankTraceStrings, lockRankRoot, lockRankTrace},
    lockRankNetpollInit:   {lockRankTimers},

    lockRankRwmutexW: {},
    lockRankRwmutexR: {lockRankSysmon, lockRankRwmutexW},

    lockRankSpanSetSpine: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankAssistQueue, lockRankCpuprof, lockRankSweep, lockRankPollDesc, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankItab, lockRankReflectOffs, lockRankNotifyList, lockRankTraceBuf, lockRankTraceStrings, lockRankHchan},
    lockRankGscan:        {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankCpuprof, lockRankSweep, lockRankSched, lockRankTimers, lockRankItab, lockRankReflectOffs, lockRankHchan, lockRankFin, lockRankTraceBuf, lockRankTraceStrings, lockRankRoot, lockRankNotifyList, lockRankProf, lockRankGcBitsArenas, lockRankTrace, lockRankTraceStackTab, lockRankNetpollInit, lockRankSpanSetSpine},
    lockRankStackpool:    {lockRankSysmon, lockRankScavenge, lockRankSweepWaiters, lockRankAssistQueue, lockRankCpuprof, lockRankSweep, lockRankSched, lockRankPollDesc, lockRankTimers, lockRankItab, lockRankReflectOffs, lockRankHchan, lockRankFin, lockRankNotifyList, lockRankTraceBuf, lockRankTraceStrings, lockRankProf, lockRankGcBitsArenas, lockRankRoot, lockRankTrace, lockRankTraceStackTab, lockRankNetpollInit, lockRankRwmutexR, lockRankSpanSetSpine, lockRankGscan},
    lockRankStackLarge:   {lockRankSysmon, lockRankAssistQueue, lockRankSched, lockRankItab, lockRankHchan, lockRankProf, lockRankGcBitsArenas, lockRankRoot, lockRankSpanSetSpine, lockRankGscan},
    lockRankDefer:        {},
    lockRankSudog:        {lockRankNotifyList, lockRankHchan},
    lockRankWbufSpans:    {lockRankSysmon, lockRankScavenge, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankSched, lockRankAllg, lockRankPollDesc, lockRankTimers, lockRankItab, lockRankReflectOffs, lockRankHchan, lockRankFin, lockRankNotifyList, lockRankTraceStrings, lockRankMspanSpecial, lockRankProf, lockRankRoot, lockRankGscan, lockRankDefer, lockRankSudog},
    lockRankMheap:        {lockRankSysmon, lockRankScavenge, lockRankSweepWaiters, lockRankAssistQueue, lockRankCpuprof, lockRankSweep, lockRankSched, lockRankAllg, lockRankAllp, lockRankFin, lockRankPollDesc, lockRankTimers, lockRankItab, lockRankReflectOffs, lockRankNotifyList, lockRankTraceBuf, lockRankTraceStrings, lockRankHchan, lockRankMspanSpecial, lockRankProf, lockRankGcBitsArenas, lockRankRoot, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankDefer, lockRankSudog, lockRankWbufSpans, lockRankSpanSetSpine},
    lockRankMheapSpecial: {lockRankSysmon, lockRankScavenge, lockRankAssistQueue, lockRankCpuprof, lockRankSweep, lockRankSched, lockRankAllg, lockRankAllp, lockRankTimers, lockRankItab, lockRankReflectOffs, lockRankNotifyList, lockRankTraceBuf, lockRankTraceStrings, lockRankHchan},
    lockRankGlobalAlloc:  {lockRankProf, lockRankSpanSetSpine, lockRankMheap, lockRankMheapSpecial},

    lockRankGFree:     {lockRankSched},
    lockRankHchanLeaf: {lockRankGscan, lockRankHchanLeaf},

    lockRankNewmHandoff:   {},
    lockRankDebugPtrmask:  {},
    lockRankFaketimeState: {},
    lockRankTicks:         {},
    lockRankRaceFini:      {},
    lockRankPollCache:     {},
    lockRankDebug:         {},
}

mFixupRace is used to temporarily borrow the race context from the coordinating m during a syscall_runtime_doAllThreadsSyscall and loan it out to each of the m's of the runtime so they can execute a mFixup.fn in that context.

var mFixupRace struct {
    lock mutex
    ctx  uintptr
}

mSpanStateNames are the names of the span states, indexed by mSpanState.

var mSpanStateNames = []string{
    "mSpanDead",
    "mSpanInUse",
    "mSpanManual",
    "mSpanFree",
}

mainStarted indicates that the main M has started.

var mainStarted bool

main_init_done is a signal used by cgocallbackg that initialization has been completed. It is made before _cgo_notify_runtime_init_done, so all cgo calls can rely on it existing. When main_init is complete, it is closed, meaning cgocallbackg can reliably receive from it.

var main_init_done chan bool

Maximum searchAddr value, which indicates that the heap has no free space.

We alias maxOffAddr just to make it clear that this is the maximum address for the page allocator's search space. See maxOffAddr for details.

var maxSearchAddr = maxOffAddr

var maxstackceiling = maxstacksize

var maxstacksize uintptr = 1 << 20 // enough until runtime.main sets it for real

var memoryError = error(errorString("invalid memory address or nil pointer dereference"))

set using cmd/go/internal/modload.ModInfoProg

var modinfo string

var modulesSlice *[]*moduledata // see activeModules

var mutexprofilerate uint64 // fraction sampled

var nbuf uintptr

newmHandoff contains a list of m structures that need new OS threads. This is used by newm in situations where newm itself can't safely start an OS thread.

var newmHandoff struct {
    lock mutex

    // newm points to a list of M structures that need new OS
    // threads. The list is linked through m.schedlink.
    newm muintptr

    // waiting indicates that wake needs to be notified when an m
    // is put on the list.
    waiting bool
    wake    note

    // haveTemplateThread indicates that the templateThread has
    // been started. This is not protected by lock. Use cas to set
    // to 1.
    haveTemplateThread uint32
}

var no_pointers_stackmap uint64 // defined in assembly, for NO_LOCAL_POINTERS macro

ptrmask for an allocation containing a single pointer.

var oneptrmask = [...]uint8{1}

var overflowError = error(errorString("integer overflow"))

var overflowTag [1]unsafe.Pointer // always nil

panicking is non-zero when crashing the program for an unrecovered panic. panicking is incremented and decremented atomically.

var panicking uint32

pendingPreemptSignals is the number of preemption signals that have been sent but not received. This is only used on Darwin. For #41702.

var pendingPreemptSignals uint32

physPageSize is the size in bytes of the OS's physical pages. Mapping and unmapping operations must be done at multiples of physPageSize.

This must be set by the OS init code (typically in osinit) before mallocinit.

var physPageSize uintptr

pinnedTypemaps are the map[typeOff]*_type from the moduledata objects.

These typemap objects are allocated at run time on the heap, but the only direct reference to them is in the moduledata, created by the linker and marked SNOPTRDATA so it is ignored by the GC.

To make sure the map isn't collected, we keep a second reference here.

var pinnedTypemaps []map[typeOff]*_type

var poolcleanup func()

var procAuxv = []byte("/proc/self/auxv\x00")

var prof struct {
    signalLock uint32
    hz         int32
}

var ptrnames = []string{
    0: "scalar",
    1: "ptr",
}

var racecgosync uint64 // represents possible synchronization in C code

reflectOffs holds type offsets defined at run time by the reflect package.

When a type is defined at run time, its *rtype data lives on the heap. There are a wide range of possible addresses the heap may use, that may not be representable as a 32-bit offset. Moreover the GC may one day start moving heap memory, in which case there is no stable offset that can be defined.

To provide stable offsets, we add pin *rtype objects in a global map and treat the offset as an identifier. We use negative offsets that do not overlap with any compile-time module offsets.

Entries are created by reflect.addReflectOff.

var reflectOffs struct {
    lock mutex
    next int32
    m    map[int32]unsafe.Pointer
    minv map[unsafe.Pointer]int32
}

runningPanicDefers is non-zero while running deferred functions for panic. runningPanicDefers is incremented and decremented atomically. This is used to try hard to get a panic stack trace out when exiting.

var runningPanicDefers uint32

runtimeInitTime is the nanotime() at which the runtime started.

var runtimeInitTime int64

Sleep/wait state of the background scavenger.

var scavenge struct {
    lock       mutex
    g          *g
    parked     bool
    timer      *timer
    sysmonWake uint32 // Set atomically.
}

var semtable [semTabSize]struct {
    root semaRoot
    pad  [cpu.CacheLinePadSize - unsafe.Sizeof(semaRoot{})]byte
}

var shiftError = error(errorString("negative shift amount"))

sig handles communication between the signal handler and os/signal. Other than the inuse and recv fields, the fields are accessed atomically.

The wanted and ignored fields are only written by one goroutine at a time; access is controlled by the handlers Mutex in os/signal. The fields are only read by that one goroutine and by the signal handler. We access them atomically to minimize the race between setting them in the goroutine calling os/signal and the signal handler, which may be running in a different thread. That race is unavoidable, as there is no connection between handling a signal and receiving one, but atomic instructions should minimize it.

var sig struct {
    note       note
    mask       [(_NSIG + 31) / 32]uint32
    wanted     [(_NSIG + 31) / 32]uint32
    ignored    [(_NSIG + 31) / 32]uint32
    recv       [(_NSIG + 31) / 32]uint32
    state      uint32
    delivering uint32
    inuse      bool
}

var signalsOK bool

var sigprofCallersUse uint32

sigsetAllExiting is used by sigblock(true) when a thread is exiting. sigset_all is defined in OS specific code, and per GOOS behavior may override this default for sigsetAllExiting: see osinit().

var sigsetAllExiting = sigset_all

var sigset_all = sigset{^uint32(0), ^uint32(0)}

var sigtable = [...]sigTabT{
    {0, "SIGNONE: no trap"},
    {_SigNotify + _SigKill, "SIGHUP: terminal line hangup"},
    {_SigNotify + _SigKill, "SIGINT: interrupt"},
    {_SigNotify + _SigThrow, "SIGQUIT: quit"},
    {_SigThrow + _SigUnblock, "SIGILL: illegal instruction"},
    {_SigThrow + _SigUnblock, "SIGTRAP: trace trap"},
    {_SigNotify + _SigThrow, "SIGABRT: abort"},
    {_SigPanic + _SigUnblock, "SIGBUS: bus error"},
    {_SigPanic + _SigUnblock, "SIGFPE: floating-point exception"},
    {0, "SIGKILL: kill"},
    {_SigNotify, "SIGUSR1: user-defined signal 1"},
    {_SigPanic + _SigUnblock, "SIGSEGV: segmentation violation"},
    {_SigNotify, "SIGUSR2: user-defined signal 2"},
    {_SigNotify, "SIGPIPE: write to broken pipe"},
    {_SigNotify, "SIGALRM: alarm clock"},
    {_SigNotify + _SigKill, "SIGTERM: termination"},
    {_SigThrow + _SigUnblock, "SIGSTKFLT: stack fault"},
    {_SigNotify + _SigUnblock + _SigIgn, "SIGCHLD: child status has changed"},
    {_SigNotify + _SigDefault + _SigIgn, "SIGCONT: continue"},
    {0, "SIGSTOP: stop, unblockable"},
    {_SigNotify + _SigDefault + _SigIgn, "SIGTSTP: keyboard stop"},
    {_SigNotify + _SigDefault + _SigIgn, "SIGTTIN: background read from tty"},
    {_SigNotify + _SigDefault + _SigIgn, "SIGTTOU: background write to tty"},
    {_SigNotify + _SigIgn, "SIGURG: urgent condition on socket"},
    {_SigNotify, "SIGXCPU: cpu limit exceeded"},
    {_SigNotify, "SIGXFSZ: file size limit exceeded"},
    {_SigNotify, "SIGVTALRM: virtual alarm clock"},
    {_SigNotify + _SigUnblock, "SIGPROF: profiling alarm clock"},
    {_SigNotify + _SigIgn, "SIGWINCH: window size change"},
    {_SigNotify, "SIGIO: i/o now possible"},
    {_SigNotify, "SIGPWR: power failure restart"},
    {_SigThrow, "SIGSYS: bad system call"},
    {_SigSetStack + _SigUnblock, "signal 32"},
    {_SigSetStack + _SigUnblock, "signal 33"},
    {_SigSetStack + _SigUnblock, "signal 34"},
    {_SigNotify, "signal 35"},
    {_SigNotify, "signal 36"},
    {_SigNotify, "signal 37"},
    {_SigNotify, "signal 38"},
    {_SigNotify, "signal 39"},
    {_SigNotify, "signal 40"},
    {_SigNotify, "signal 41"},
    {_SigNotify, "signal 42"},
    {_SigNotify, "signal 43"},
    {_SigNotify, "signal 44"},
    {_SigNotify, "signal 45"},
    {_SigNotify, "signal 46"},
    {_SigNotify, "signal 47"},
    {_SigNotify, "signal 48"},
    {_SigNotify, "signal 49"},
    {_SigNotify, "signal 50"},
    {_SigNotify, "signal 51"},
    {_SigNotify, "signal 52"},
    {_SigNotify, "signal 53"},
    {_SigNotify, "signal 54"},
    {_SigNotify, "signal 55"},
    {_SigNotify, "signal 56"},
    {_SigNotify, "signal 57"},
    {_SigNotify, "signal 58"},
    {_SigNotify, "signal 59"},
    {_SigNotify, "signal 60"},
    {_SigNotify, "signal 61"},
    {_SigNotify, "signal 62"},
    {_SigNotify, "signal 63"},
    {_SigNotify, "signal 64"},
}

var size_to_class128 = [(_MaxSmallSize-smallSizeMax)/largeSizeDiv + 1]uint8{32, 33, 34, 35, 36, 37, 37, 38, 38, 39, 39, 40, 40, 40, 41, 41, 41, 42, 43, 43, 44, 44, 44, 44, 44, 45, 45, 45, 45, 45, 45, 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 48, 48, 48, 49, 49, 50, 51, 51, 51, 51, 51, 51, 51, 51, 51, 51, 52, 52, 52, 52, 52, 52, 52, 52, 52, 52, 53, 53, 54, 54, 54, 54, 55, 55, 55, 55, 55, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 57, 57, 57, 57, 57, 57, 57, 57, 57, 57, 58, 58, 58, 58, 58, 58, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 61, 61, 61, 61, 61, 62, 62, 62, 62, 62, 62, 62, 62, 62, 62, 62, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67}

var size_to_class8 = [smallSizeMax/smallSizeDiv + 1]uint8{0, 1, 2, 3, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13, 14, 14, 15, 15, 16, 16, 17, 17, 18, 18, 19, 19, 19, 19, 20, 20, 20, 20, 21, 21, 21, 21, 22, 22, 22, 22, 23, 23, 23, 23, 24, 24, 24, 24, 25, 25, 25, 25, 26, 26, 26, 26, 27, 27, 27, 27, 27, 27, 27, 27, 28, 28, 28, 28, 28, 28, 28, 28, 29, 29, 29, 29, 29, 29, 29, 29, 30, 30, 30, 30, 30, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32}

Global pool of large stack spans.

var stackLarge struct {
    lock mutex
    free [heapAddrBits - pageShift]mSpanList // free lists by log_2(s.npages)
}

Global pool of spans that have free stacks. Stacks are assigned an order according to size.

order = log_2(size/FixedStack)

There is a free list for each order.

var stackpool [_NumStackOrders]struct {
    item stackpoolItem
    _    [cpu.CacheLinePadSize - unsafe.Sizeof(stackpoolItem{})%cpu.CacheLinePadSize]byte
}

var starttime int64

startupRandomData holds random bytes initialized at startup. These come from the ELF AT_RANDOM auxiliary vector.

var startupRandomData []byte

staticuint64s is used to avoid allocating in convTx for small integer values.

var staticuint64s = [...]uint64{
    0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
    0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
    0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17,
    0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f,
    0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27,
    0x28, 0x29, 0x2a, 0x2b, 0x2c, 0x2d, 0x2e, 0x2f,
    0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37,
    0x38, 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f,
    0x40, 0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47,
    0x48, 0x49, 0x4a, 0x4b, 0x4c, 0x4d, 0x4e, 0x4f,
    0x50, 0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57,
    0x58, 0x59, 0x5a, 0x5b, 0x5c, 0x5d, 0x5e, 0x5f,
    0x60, 0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67,
    0x68, 0x69, 0x6a, 0x6b, 0x6c, 0x6d, 0x6e, 0x6f,
    0x70, 0x71, 0x72, 0x73, 0x74, 0x75, 0x76, 0x77,
    0x78, 0x79, 0x7a, 0x7b, 0x7c, 0x7d, 0x7e, 0x7f,
    0x80, 0x81, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87,
    0x88, 0x89, 0x8a, 0x8b, 0x8c, 0x8d, 0x8e, 0x8f,
    0x90, 0x91, 0x92, 0x93, 0x94, 0x95, 0x96, 0x97,
    0x98, 0x99, 0x9a, 0x9b, 0x9c, 0x9d, 0x9e, 0x9f,
    0xa0, 0xa1, 0xa2, 0xa3, 0xa4, 0xa5, 0xa6, 0xa7,
    0xa8, 0xa9, 0xaa, 0xab, 0xac, 0xad, 0xae, 0xaf,
    0xb0, 0xb1, 0xb2, 0xb3, 0xb4, 0xb5, 0xb6, 0xb7,
    0xb8, 0xb9, 0xba, 0xbb, 0xbc, 0xbd, 0xbe, 0xbf,
    0xc0, 0xc1, 0xc2, 0xc3, 0xc4, 0xc5, 0xc6, 0xc7,
    0xc8, 0xc9, 0xca, 0xcb, 0xcc, 0xcd, 0xce, 0xcf,
    0xd0, 0xd1, 0xd2, 0xd3, 0xd4, 0xd5, 0xd6, 0xd7,
    0xd8, 0xd9, 0xda, 0xdb, 0xdc, 0xdd, 0xde, 0xdf,
    0xe0, 0xe1, 0xe2, 0xe3, 0xe4, 0xe5, 0xe6, 0xe7,
    0xe8, 0xe9, 0xea, 0xeb, 0xec, 0xed, 0xee, 0xef,
    0xf0, 0xf1, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7,
    0xf8, 0xf9, 0xfa, 0xfb, 0xfc, 0xfd, 0xfe, 0xff,
}

var sysTHPSizePath = []byte("/sys/kernel/mm/transparent_hugepage/hpage_pmd_size\x00")

testSigtrap and testSigusr1 are used by the runtime tests. If non-nil, it is called on SIGTRAP/SIGUSR1. If it returns true, the normal behavior on this signal is suppressed.

var testSigtrap func(info *siginfo, ctxt *sigctxt, gp *g) bool

var testSigusr1 func(gp *g) bool

TODO: These should be locals in testAtomic64, but we don't 8-byte align stack variables on 386.

var test_z64, test_x64 uint64

var ticks struct {
    lock mutex
    pad  uint32 // ensure 8-byte alignment of val on 386
    val  uint64
}

var tmpbuf []byte

trace is global tracing context.

var trace struct {
    lock          mutex       // protects the following members
    lockOwner     *g          // to avoid deadlocks during recursive lock locks
    enabled       bool        // when set runtime traces events
    shutdown      bool        // set when we are waiting for trace reader to finish after setting enabled to false
    headerWritten bool        // whether ReadTrace has emitted trace header
    footerWritten bool        // whether ReadTrace has emitted trace footer
    shutdownSema  uint32      // used to wait for ReadTrace completion
    seqStart      uint64      // sequence number when tracing was started
    ticksStart    int64       // cputicks when tracing was started
    ticksEnd      int64       // cputicks when tracing was stopped
    timeStart     int64       // nanotime when tracing was started
    timeEnd       int64       // nanotime when tracing was stopped
    seqGC         uint64      // GC start/done sequencer
    reading       traceBufPtr // buffer currently handed off to user
    empty         traceBufPtr // stack of empty buffers
    fullHead      traceBufPtr // queue of full buffers
    fullTail      traceBufPtr
    reader        guintptr        // goroutine that called ReadTrace, or nil
    stackTab      traceStackTable // maps stack traces to unique ids

    // Dictionary for traceEvString.
    //
    // TODO: central lock to access the map is not ideal.
    //   option: pre-assign ids to all user annotation region names and tags
    //   option: per-P cache
    //   option: sync.Map like data structure
    stringsLock mutex
    strings     map[string]uint64
    stringSeq   uint64

    // markWorkerLabels maps gcMarkWorkerMode to string ID.
    markWorkerLabels [len(gcMarkWorkerModeStrings)]uint64

    bufLock mutex       // protects buf
    buf     traceBufPtr // global trace buffer, used when running without a p
}

var traceback_cache uint32 = 2 << tracebackShift

var traceback_env uint32

var typecache [typeCacheBuckets]typeCacheBucket

var urandom_dev = []byte("/dev/urandom\x00")

var useAVXmemmove bool

runtime variable to check if the processor we're running on actually supports the instructions used by the AES-based hash implementation.

var useAeshash bool

If useCheckmark is true, marking of an object uses the checkmark bits instead of the standard mark bits.

var useCheckmark = false

var vdsoLinuxVersion = vdsoVersionKey{"LINUX_2.6", 0x3ae75f6}

var vdsoSymbolKeys = []vdsoSymbolKey{
    {"__vdso_gettimeofday", 0x315ca59, 0xb01bca00, &vdsoGettimeofdaySym},
    {"__vdso_clock_gettime", 0xd35ec75, 0x6e43a318, &vdsoClockgettimeSym},
}

var waitReasonStrings = [...]string{
    waitReasonZero:                  "",
    waitReasonGCAssistMarking:       "GC assist marking",
    waitReasonIOWait:                "IO wait",
    waitReasonChanReceiveNilChan:    "chan receive (nil chan)",
    waitReasonChanSendNilChan:       "chan send (nil chan)",
    waitReasonDumpingHeap:           "dumping heap",
    waitReasonGarbageCollection:     "garbage collection",
    waitReasonGarbageCollectionScan: "garbage collection scan",
    waitReasonPanicWait:             "panicwait",
    waitReasonSelect:                "select",
    waitReasonSelectNoCases:         "select (no cases)",
    waitReasonGCAssistWait:          "GC assist wait",
    waitReasonGCSweepWait:           "GC sweep wait",
    waitReasonGCScavengeWait:        "GC scavenge wait",
    waitReasonChanReceive:           "chan receive",
    waitReasonChanSend:              "chan send",
    waitReasonFinalizerWait:         "finalizer wait",
    waitReasonForceGCIdle:           "force gc (idle)",
    waitReasonSemacquire:            "semacquire",
    waitReasonSleep:                 "sleep",
    waitReasonSyncCondWait:          "sync.Cond.Wait",
    waitReasonTimerGoroutineIdle:    "timer goroutine (idle)",
    waitReasonTraceReaderBlocked:    "trace reader (blocked)",
    waitReasonWaitForGCCycle:        "wait for GC cycle",
    waitReasonGCWorkerIdle:          "GC worker (idle)",
    waitReasonPreempted:             "preempted",
    waitReasonDebugCall:             "debug call",
}

var work struct {
    full  lfstack          // lock-free list of full blocks workbuf
    empty lfstack          // lock-free list of empty blocks workbuf
    pad0  cpu.CacheLinePad // prevents false-sharing between full/empty and nproc/nwait

    wbufSpans struct {
        lock mutex
        // free is a list of spans dedicated to workbufs, but
        // that don't currently contain any workbufs.
        free mSpanList
        // busy is a list of all spans containing workbufs on
        // one of the workbuf lists.
        busy mSpanList
    }

    // Restore 64-bit alignment on 32-bit.
    _ uint32

    // bytesMarked is the number of bytes marked this cycle. This
    // includes bytes blackened in scanned objects, noscan objects
    // that go straight to black, and permagrey objects scanned by
    // markroot during the concurrent scan phase. This is updated
    // atomically during the cycle. Updates may be batched
    // arbitrarily, since the value is only read at the end of the
    // cycle.
    //
    // Because of benign races during marking, this number may not
    // be the exact number of marked bytes, but it should be very
    // close.
    //
    // Put this field here because it needs 64-bit atomic access
    // (and thus 8-byte alignment even on 32-bit architectures).
    bytesMarked uint64

    markrootNext uint32 // next markroot job
    markrootJobs uint32 // number of markroot jobs

    nproc  uint32
    tstart int64
    nwait  uint32

    // Number of roots of various root types. Set by gcMarkRootPrepare.
    nFlushCacheRoots                               int
    nDataRoots, nBSSRoots, nSpanRoots, nStackRoots int

    // Each type of GC state transition is protected by a lock.
    // Since multiple threads can simultaneously detect the state
    // transition condition, any thread that detects a transition
    // condition must acquire the appropriate transition lock,
    // re-check the transition condition and return if it no
    // longer holds or perform the transition if it does.
    // Likewise, any transition must invalidate the transition
    // condition before releasing the lock. This ensures that each
    // transition is performed by exactly one thread and threads
    // that need the transition to happen block until it has
    // happened.
    //
    // startSema protects the transition from "off" to mark or
    // mark termination.
    startSema uint32
    // markDoneSema protects transitions from mark to mark termination.
    markDoneSema uint32

    bgMarkReady note   // signal background mark worker has started
    bgMarkDone  uint32 // cas to 1 when at a background mark completion point

    // mode is the concurrency mode of the current GC cycle.
    mode gcMode

    // userForced indicates the current GC cycle was forced by an
    // explicit user call.
    userForced bool

    // totaltime is the CPU nanoseconds spent in GC since the
    // program started if debug.gctrace > 0.
    totaltime int64

    // initialHeapLive is the value of memstats.heap_live at the
    // beginning of this GC cycle.
    initialHeapLive uint64

    // assistQueue is a queue of assists that are blocked because
    // there was neither enough credit to steal or enough work to
    // do.
    assistQueue struct {
        lock mutex
        q    gQueue
    }

    // sweepWaiters is a list of blocked goroutines to wake when
    // we transition from mark termination to sweep.
    sweepWaiters struct {
        lock mutex
        list gList
    }

    // cycles is the number of completed GC cycles, where a GC
    // cycle is sweep termination, mark, mark termination, and
    // sweep. This differs from memstats.numgc, which is
    // incremented at mark termination.
    cycles uint32

    // Timing/utilization stats for this cycle.
    stwprocs, maxprocs                 int32
    tSweepTerm, tMark, tMarkTerm, tEnd int64 // nanotime() of phase start

    pauseNS    int64 // total STW time this cycle
    pauseStart int64 // nanotime() of last STW

    // debug.gctrace heap sizes for this cycle.
    heap0, heap1, heap2, heapGoal uint64
}

Holding worldsema grants an M the right to try to stop the world.

var worldsema uint32 = 1

The compiler knows about this variable. If you change it, you must change builtin/runtime.go, too. If you change the first four bytes, you must also change the write barrier insertion code.

var writeBarrier struct {
    enabled bool    // compiler emits a check of this before calling write barrier
    pad     [3]byte // compiler uses 32-bit load for "enabled" field
    needed  bool    // whether we need a write barrier for current GC phase
    cgo     bool    // whether we need a write barrier for a cgo check
    alignme uint64  // guarantee alignment so that compiler can use a 32 or 64-bit load
}

var zeroVal [maxZero]byte

base address for all 0-byte allocations

var zerobase uintptr

func BlockProfile ¶ 1.1

func BlockProfile(p []BlockProfileRecord) (n int, ok bool)

BlockProfile returns n, the number of records in the current blocking profile. If len(p) >= n, BlockProfile copies the profile into p and returns n, true. If len(p) < n, BlockProfile does not change p and returns n, false.

Most clients should use the runtime/pprof package or the testing package's -test.blockprofile flag instead of calling BlockProfile directly.

func Breakpoint ¶

func Breakpoint()

Breakpoint executes a breakpoint trap.

func CPUProfile ¶

func CPUProfile() []byte

CPUProfile panics. It formerly provided raw access to chunks of a pprof-format profile generated by the runtime. The details of generating that format have changed, so this functionality has been removed.

Deprecated: Use the runtime/pprof package, or the handlers in the net/http/pprof package, or the testing package's -test.cpuprofile flag instead.

func Caller ¶

func Caller(skip int) (pc uintptr, file string, line int, ok bool)

Caller reports file and line number information about function invocations on the calling goroutine's stack. The argument skip is the number of stack frames to ascend, with 0 identifying the caller of Caller. (For historical reasons the meaning of skip differs between Caller and Callers.) The return values report the program counter, file name, and line number within the file of the corresponding call. The boolean ok is false if it was not possible to recover the information.

func Callers ¶

func Callers(skip int, pc []uintptr) int

Callers fills the slice pc with the return program counters of function invocations on the calling goroutine's stack. The argument skip is the number of stack frames to skip before recording in pc, with 0 identifying the frame for Callers itself and 1 identifying the caller of Callers. It returns the number of entries written to pc.

To translate these PCs into symbolic information such as function names and line numbers, use CallersFrames. CallersFrames accounts for inlined functions and adjusts the return program counters into call program counters. Iterating over the returned slice of PCs directly is discouraged, as is using FuncForPC on any of the returned PCs, since these cannot account for inlining or return program counter adjustment.

func GC ¶

func GC()

GC runs a garbage collection and blocks the caller until the garbage collection is complete. It may also block the entire program.

func GOMAXPROCS ¶

func GOMAXPROCS(n int) int

GOMAXPROCS sets the maximum number of CPUs that can be executing simultaneously and returns the previous setting. It defaults to the value of runtime.NumCPU. If n < 1, it does not change the current setting. This call will go away when the scheduler improves.

func GOROOT ¶

func GOROOT() string

GOROOT returns the root of the Go tree. It uses the GOROOT environment variable, if set at process start, or else the root used during the Go build.

func Goexit ¶

func Goexit()

Goexit terminates the goroutine that calls it. No other goroutine is affected. Goexit runs all deferred calls before terminating the goroutine. Because Goexit is not a panic, any recover calls in those deferred functions will return nil.

Calling Goexit from the main goroutine terminates that goroutine without func main returning. Since func main has not returned, the program continues execution of other goroutines. If all other goroutines exit, the program crashes.

func GoroutineProfile ¶

func GoroutineProfile(p []StackRecord) (n int, ok bool)

GoroutineProfile returns n, the number of records in the active goroutine stack profile. If len(p) >= n, GoroutineProfile copies the profile into p and returns n, true. If len(p) < n, GoroutineProfile does not change p and returns n, false.

Most clients should use the runtime/pprof package instead of calling GoroutineProfile directly.

func Gosched ¶

func Gosched()

Gosched yields the processor, allowing other goroutines to run. It does not suspend the current goroutine, so execution resumes automatically.

func KeepAlive ¶ 1.7

func KeepAlive(x interface{})

KeepAlive marks its argument as currently reachable. This ensures that the object is not freed, and its finalizer is not run, before the point in the program where KeepAlive is called.

A very simplified example showing where KeepAlive is required:

type File struct { d int }
d, err := syscall.Open("/file/path", syscall.O_RDONLY, 0)
// ... do something if err != nil ...
p := &File{d}
runtime.SetFinalizer(p, func(p *File) { syscall.Close(p.d) })
var buf [10]byte
n, err := syscall.Read(p.d, buf[:])
// Ensure p is not finalized until Read returns.
runtime.KeepAlive(p)
// No more uses of p after this point.

Without the KeepAlive call, the finalizer could run at the start of syscall.Read, closing the file descriptor before syscall.Read makes the actual system call.

func LockOSThread ¶

func LockOSThread()

LockOSThread wires the calling goroutine to its current operating system thread. The calling goroutine will always execute in that thread, and no other goroutine will execute in it, until the calling goroutine has made as many calls to UnlockOSThread as to LockOSThread. If the calling goroutine exits without unlocking the thread, the thread will be terminated.

All init functions are run on the startup thread. Calling LockOSThread from an init function will cause the main function to be invoked on that thread.

A goroutine should call LockOSThread before calling OS services or non-Go library functions that depend on per-thread state.

func MemProfile ¶

func MemProfile(p []MemProfileRecord, inuseZero bool) (n int, ok bool)

MemProfile returns a profile of memory allocated and freed per allocation site.

MemProfile returns n, the number of records in the current memory profile. If len(p) >= n, MemProfile copies the profile into p and returns n, true. If len(p) < n, MemProfile does not change p and returns n, false.

If inuseZero is true, the profile includes allocation records where r.AllocBytes > 0 but r.AllocBytes == r.FreeBytes. These are sites where memory was allocated, but it has all been released back to the runtime.

The returned profile may be up to two garbage collection cycles old. This is to avoid skewing the profile toward allocations; because allocations happen in real time but frees are delayed until the garbage collector performs sweeping, the profile only accounts for allocations that have had a chance to be freed by the garbage collector.

Most clients should use the runtime/pprof package or the testing package's -test.memprofile flag instead of calling MemProfile directly.

func MutexProfile ¶ 1.8

func MutexProfile(p []BlockProfileRecord) (n int, ok bool)

MutexProfile returns n, the number of records in the current mutex profile. If len(p) >= n, MutexProfile copies the profile into p and returns n, true. Otherwise, MutexProfile does not change p, and returns n, false.

Most clients should use the runtime/pprof package instead of calling MutexProfile directly.

func NumCPU ¶

func NumCPU() int

NumCPU returns the number of logical CPUs usable by the current process.

The set of available CPUs is checked by querying the operating system at process startup. Changes to operating system CPU allocation after process startup are not reflected.

func NumCgoCall ¶

func NumCgoCall() int64

NumCgoCall returns the number of cgo calls made by the current process.

func NumGoroutine ¶

func NumGoroutine() int

NumGoroutine returns the number of goroutines that currently exist.

func ReadMemStats ¶

func ReadMemStats(m *MemStats)

ReadMemStats populates m with memory allocator statistics.

The returned memory allocator statistics are up to date as of the call to ReadMemStats. This is in contrast with a heap profile, which is a snapshot as of the most recently completed garbage collection cycle.

func ReadTrace ¶ 1.5

func ReadTrace() []byte

ReadTrace returns the next chunk of binary tracing data, blocking until data is available. If tracing is turned off and all the data accumulated while it was on has been returned, ReadTrace returns nil. The caller must copy the returned data before calling ReadTrace again. ReadTrace must be called from one goroutine at a time.

func SetBlockProfileRate ¶ 1.1

func SetBlockProfileRate(rate int)

SetBlockProfileRate controls the fraction of goroutine blocking events that are reported in the blocking profile. The profiler aims to sample an average of one blocking event per rate nanoseconds spent blocked.

To include every blocking event in the profile, pass rate = 1. To turn off profiling entirely, pass rate <= 0.

func SetCPUProfileRate ¶

func SetCPUProfileRate(hz int)

SetCPUProfileRate sets the CPU profiling rate to hz samples per second. If hz <= 0, SetCPUProfileRate turns off profiling. If the profiler is on, the rate cannot be changed without first turning it off.

Most clients should use the runtime/pprof package or the testing package's -test.cpuprofile flag instead of calling SetCPUProfileRate directly.

func SetCgoTraceback ¶ 1.7

func SetCgoTraceback(version int, traceback, context, symbolizer unsafe.Pointer)

SetCgoTraceback records three C functions to use to gather traceback information from C code and to convert that traceback information into symbolic information. These are used when printing stack traces for a program that uses cgo.

The traceback and context functions may be called from a signal handler, and must therefore use only async-signal safe functions. The symbolizer function may be called while the program is crashing, and so must be cautious about using memory. None of the functions may call back into Go.

The context function will be called with a single argument, a pointer to a struct:

struct {
	Context uintptr
}

In C syntax, this struct will be

struct {
	uintptr_t Context;
};

If the Context field is 0, the context function is being called to record the current traceback context. It should record in the Context field whatever information is needed about the current point of execution to later produce a stack trace, probably the stack pointer and PC. In this case the context function will be called from C code.

If the Context field is not 0, then it is a value returned by a previous call to the context function. This case is called when the context is no longer needed; that is, when the Go code is returning to its C code caller. This permits the context function to release any associated resources.

While it would be correct for the context function to record a complete a stack trace whenever it is called, and simply copy that out in the traceback function, in a typical program the context function will be called many times without ever recording a traceback for that context. Recording a complete stack trace in a call to the context function is likely to be inefficient.

The traceback function will be called with a single argument, a pointer to a struct:

struct {
	Context    uintptr
	SigContext uintptr
	Buf        *uintptr
	Max        uintptr
}

In C syntax, this struct will be

struct {
	uintptr_t  Context;
	uintptr_t  SigContext;
	uintptr_t* Buf;
	uintptr_t  Max;
};

The Context field will be zero to gather a traceback from the current program execution point. In this case, the traceback function will be called from C code.

Otherwise Context will be a value previously returned by a call to the context function. The traceback function should gather a stack trace from that saved point in the program execution. The traceback function may be called from an execution thread other than the one that recorded the context, but only when the context is known to be valid and unchanging. The traceback function may also be called deeper in the call stack on the same thread that recorded the context. The traceback function may be called multiple times with the same Context value; it will usually be appropriate to cache the result, if possible, the first time this is called for a specific context value.

If the traceback function is called from a signal handler on a Unix system, SigContext will be the signal context argument passed to the signal handler (a C ucontext_t* cast to uintptr_t). This may be used to start tracing at the point where the signal occurred. If the traceback function is not called from a signal handler, SigContext will be zero.

Buf is where the traceback information should be stored. It should be PC values, such that Buf[0] is the PC of the caller, Buf[1] is the PC of that function's caller, and so on. Max is the maximum number of entries to store. The function should store a zero to indicate the top of the stack, or that the caller is on a different stack, presumably a Go stack.

Unlike runtime.Callers, the PC values returned should, when passed to the symbolizer function, return the file/line of the call instruction. No additional subtraction is required or appropriate.

On all platforms, the traceback function is invoked when a call from Go to C to Go requests a stack trace. On linux/amd64, linux/ppc64le, and freebsd/amd64, the traceback function is also invoked when a signal is received by a thread that is executing a cgo call. The traceback function should not make assumptions about when it is called, as future versions of Go may make additional calls.

The symbolizer function will be called with a single argument, a pointer to a struct:

struct {
	PC      uintptr // program counter to fetch information for
	File    *byte   // file name (NUL terminated)
	Lineno  uintptr // line number
	Func    *byte   // function name (NUL terminated)
	Entry   uintptr // function entry point
	More    uintptr // set non-zero if more info for this PC
	Data    uintptr // unused by runtime, available for function
}

In C syntax, this struct will be

struct {
	uintptr_t PC;
	char*     File;
	uintptr_t Lineno;
	char*     Func;
	uintptr_t Entry;
	uintptr_t More;
	uintptr_t Data;
};

The PC field will be a value returned by a call to the traceback function.

The first time the function is called for a particular traceback, all the fields except PC will be 0. The function should fill in the other fields if possible, setting them to 0/nil if the information is not available. The Data field may be used to store any useful information across calls. The More field should be set to non-zero if there is more information for this PC, zero otherwise. If More is set non-zero, the function will be called again with the same PC, and may return different information (this is intended for use with inlined functions). If More is zero, the function will be called with the next PC value in the traceback. When the traceback is complete, the function will be called once more with PC set to zero; this may be used to free any information. Each call will leave the fields of the struct set to the same values they had upon return, except for the PC field when the More field is zero. The function must not keep a copy of the struct pointer between calls.

When calling SetCgoTraceback, the version argument is the version number of the structs that the functions expect to receive. Currently this must be zero.

The symbolizer function may be nil, in which case the results of the traceback function will be displayed as numbers. If the traceback function is nil, the symbolizer function will never be called. The context function may be nil, in which case the traceback function will only be called with the context field set to zero. If the context function is nil, then calls from Go to C to Go will not show a traceback for the C portion of the call stack.

SetCgoTraceback should be called only once, ideally from an init function.

func SetFinalizer ¶

func SetFinalizer(obj interface{}, finalizer interface{})

SetFinalizer sets the finalizer associated with obj to the provided finalizer function. When the garbage collector finds an unreachable block with an associated finalizer, it clears the association and runs finalizer(obj) in a separate goroutine. This makes obj reachable again, but now without an associated finalizer. Assuming that SetFinalizer is not called again, the next time the garbage collector sees that obj is unreachable, it will free obj.

SetFinalizer(obj, nil) clears any finalizer associated with obj.

The argument obj must be a pointer to an object allocated by calling new, by taking the address of a composite literal, or by taking the address of a local variable. The argument finalizer must be a function that takes a single argument to which obj's type can be assigned, and can have arbitrary ignored return values. If either of these is not true, SetFinalizer may abort the program.

Finalizers are run in dependency order: if A points at B, both have finalizers, and they are otherwise unreachable, only the finalizer for A runs; once A is freed, the finalizer for B can run. If a cyclic structure includes a block with a finalizer, that cycle is not guaranteed to be garbage collected and the finalizer is not guaranteed to run, because there is no ordering that respects the dependencies.

The finalizer is scheduled to run at some arbitrary time after the program can no longer reach the object to which obj points. There is no guarantee that finalizers will run before a program exits, so typically they are useful only for releasing non-memory resources associated with an object during a long-running program. For example, an os.File object could use a finalizer to close the associated operating system file descriptor when a program discards an os.File without calling Close, but it would be a mistake to depend on a finalizer to flush an in-memory I/O buffer such as a bufio.Writer, because the buffer would not be flushed at program exit.

It is not guaranteed that a finalizer will run if the size of *obj is zero bytes.

It is not guaranteed that a finalizer will run for objects allocated in initializers for package-level variables. Such objects may be linker-allocated, not heap-allocated.

A finalizer may run as soon as an object becomes unreachable. In order to use finalizers correctly, the program must ensure that the object is reachable until it is no longer required. Objects stored in global variables, or that can be found by tracing pointers from a global variable, are reachable. For other objects, pass the object to a call of the KeepAlive function to mark the last point in the function where the object must be reachable.

For example, if p points to a struct, such as os.File, that contains a file descriptor d, and p has a finalizer that closes that file descriptor, and if the last use of p in a function is a call to syscall.Write(p.d, buf, size), then p may be unreachable as soon as the program enters syscall.Write. The finalizer may run at that moment, closing p.d, causing syscall.Write to fail because it is writing to a closed file descriptor (or, worse, to an entirely different file descriptor opened by a different goroutine). To avoid this problem, call runtime.KeepAlive(p) after the call to syscall.Write.

A single goroutine runs all finalizers for a program, sequentially. If a finalizer must run for a long time, it should do so by starting a new goroutine.

func SetMutexProfileFraction ¶ 1.8

func SetMutexProfileFraction(rate int) int

SetMutexProfileFraction controls the fraction of mutex contention events that are reported in the mutex profile. On average 1/rate events are reported. The previous rate is returned.

To turn off profiling entirely, pass rate 0. To just read the current rate, pass rate < 0. (For n>1 the details of sampling may change.)

func Stack ¶

func Stack(buf []byte, all bool) int

Stack formats a stack trace of the calling goroutine into buf and returns the number of bytes written to buf. If all is true, Stack formats stack traces of all other goroutines into buf after the trace for the current goroutine.

func StartTrace ¶ 1.5

func StartTrace() error

StartTrace enables tracing for the current process. While tracing, the data will be buffered and available via ReadTrace. StartTrace returns an error if tracing is already enabled. Most clients should use the runtime/trace package or the testing package's -test.trace flag instead of calling StartTrace directly.

func StopTrace ¶ 1.5

func StopTrace()

StopTrace stops tracing, if it was previously enabled. StopTrace only returns after all the reads for the trace have completed.

func ThreadCreateProfile ¶

func ThreadCreateProfile(p []StackRecord) (n int, ok bool)

ThreadCreateProfile returns n, the number of records in the thread creation profile. If len(p) >= n, ThreadCreateProfile copies the profile into p and returns n, true. If len(p) < n, ThreadCreateProfile does not change p and returns n, false.

Most clients should use the runtime/pprof package instead of calling ThreadCreateProfile directly.

func UnlockOSThread ¶

func UnlockOSThread()

UnlockOSThread undoes an earlier call to LockOSThread. If this drops the number of active LockOSThread calls on the calling goroutine to zero, it unwires the calling goroutine from its fixed operating system thread. If there are no active LockOSThread calls, this is a no-op.

Before calling UnlockOSThread, the caller must ensure that the OS thread is suitable for running other goroutines. If the caller made any permanent changes to the state of the thread that would affect other goroutines, it should not call this function and thus leave the goroutine locked to the OS thread until the goroutine (and hence the thread) exits.

func Version ¶

func Version() string

Version returns the Go tree's version string. It is either the commit hash and date at the time of the build or, when possible, a release tag like "go1.3".

func _ELF_ST_BIND ¶

func _ELF_ST_BIND(val byte) byte

How to extract and insert information held in the st_info field.

func _ELF_ST_TYPE ¶

func _ELF_ST_TYPE(val byte) byte

func _ExternalCode ¶

func _ExternalCode()

func _GC ¶

func _GC()

func _LostExternalCode ¶

func _LostExternalCode()

func _LostSIGPROFDuringAtomic64 ¶

func _LostSIGPROFDuringAtomic64()

func _System ¶

func _System()

func _VDSO ¶

func _VDSO()

func _cgo_panic_internal ¶

func _cgo_panic_internal(p *byte)

func abort ¶

func abort()

abort crashes the runtime in situations where even throw might not work. In general it should do something a debugger will recognize (e.g., an INT3 on x86). A crash in abort is recognized by the signal handler, which will attempt to tear down the runtime immediately.

func abs ¶

func abs(x float64) float64

Abs returns the absolute value of x.

Special cases are:

Abs(±Inf) = +Inf
Abs(NaN) = NaN

func access ¶

func access(name *byte, mode int32) int32

Called from write_err_android.go only, but defined in sys_linux_*.s; declared here (instead of in write_err_android.go) for go vet on non-android builds. The return value is the raw syscall result, which may encode an error number.

func acquireLockRank ¶

func acquireLockRank(rank lockRank)

This function may be called in nosplit context and thus must be nosplit.

func acquirep ¶

func acquirep(_p_ *p)

Associate p and the current m.

This function is allowed to have write barriers even if the caller isn't because it immediately acquires _p_.

func add ¶

func add(p unsafe.Pointer, x uintptr) unsafe.Pointer

Should be a built-in for unsafe.Pointer?

func add1 ¶

func add1(p *byte) *byte

add1 returns the byte pointer p+1.

func addAdjustedTimers ¶

func addAdjustedTimers(pp *p, moved []*timer)

addAdjustedTimers adds any timers we adjusted in adjusttimers back to the timer heap.

func addOneOpenDeferFrame ¶

func addOneOpenDeferFrame(gp *g, pc uintptr, sp unsafe.Pointer)

addOneOpenDeferFrame scans the stack for the first frame (if any) with open-coded defers and if it finds one, adds a single record to the defer chain for that frame. If sp is non-nil, it starts the stack scan from the frame specified by sp. If sp is nil, it uses the sp from the current defer record (which has just been finished). Hence, it continues the stack scan from the frame of the defer that just finished. It skips any frame that already has an open-coded _defer record, which would have been been created from a previous (unrecovered) panic.

Note: All entries of the defer chain (including this new open-coded entry) have their pointers (including sp) adjusted properly if the stack moves while running deferred functions. Also, it is safe to pass in the sp arg (which is the direct result of calling getcallersp()), because all pointer variables (including arguments) are adjusted as needed during stack copies.

func addb ¶

func addb(p *byte, n uintptr) *byte

addb returns the byte pointer p+n.

func addfinalizer ¶

func addfinalizer(p unsafe.Pointer, f *funcval, nret uintptr, fint *_type, ot *ptrtype) bool

Adds a finalizer to the object p. Returns true if it succeeded.

func addmoduledata ¶

func addmoduledata()

Called from linker-generated .initarray; declared for go vet; do NOT call from Go.

func addrsToSummaryRange ¶

func addrsToSummaryRange(level int, base, limit uintptr) (lo int, hi int)

addrsToSummaryRange converts base and limit pointers into a range of entries for the given summary level.

The returned range is inclusive on the lower bound and exclusive on the upper bound.

func addspecial ¶

func addspecial(p unsafe.Pointer, s *special) bool

Adds the special record s to the list of special records for the object p. All fields of s should be filled in except for offset & next, which this routine will fill in. Returns true if the special was successfully added, false otherwise. (The add will fail only if a record with the same p and s->kind

already exists.)

func addtimer ¶

func addtimer(t *timer)

addtimer adds a timer to the current P. This should only be called with a newly created timer. That avoids the risk of changing the when field of a timer in some P's heap, which could cause the heap to become unsorted.

func adjustSignalStack ¶

func adjustSignalStack(sig uint32, mp *m, gsigStack *gsignalStack) bool

adjustSignalStack adjusts the current stack guard based on the stack pointer that is actually in use while handling a signal. We do this in case some non-Go code called sigaltstack. This reports whether the stack was adjusted, and if so stores the old signal stack in *gsigstack.

func adjustctxt ¶

func adjustctxt(gp *g, adjinfo *adjustinfo)

func adjustdefers ¶

func adjustdefers(gp *g, adjinfo *adjustinfo)

func adjustframe ¶

func adjustframe(frame *stkframe, arg unsafe.Pointer) bool

Note: the argument/return area is adjusted by the callee.

func adjustpanics ¶

func adjustpanics(gp *g, adjinfo *adjustinfo)

func adjustpointer ¶

func adjustpointer(adjinfo *adjustinfo, vpp unsafe.Pointer)

Adjustpointer checks whether *vpp is in the old stack described by adjinfo. If so, it rewrites *vpp to point into the new stack.

func adjustpointers ¶

func adjustpointers(scanp unsafe.Pointer, bv *bitvector, adjinfo *adjustinfo, f funcInfo)

bv describes the memory starting at address scanp. Adjust any pointers contained therein.

func adjustsudogs ¶

func adjustsudogs(gp *g, adjinfo *adjustinfo)

func adjusttimers ¶

func adjusttimers(pp *p, now int64)

adjusttimers looks through the timers in the current P's heap for any timers that have been modified to run earlier, and puts them in the correct place in the heap. While looking for those timers, it also moves timers that have been modified to run later, and removes deleted timers. The caller must have locked the timers for pp.

func advanceEvacuationMark ¶

func advanceEvacuationMark(h *hmap, t *maptype, newbit uintptr)

func afterfork ¶

func afterfork()

func alginit ¶

func alginit()

func alignDown ¶

func alignDown(n, a uintptr) uintptr

alignDown rounds n down to a multiple of a. a must be a power of 2.

func alignUp ¶

func alignUp(n, a uintptr) uintptr

alignUp rounds n up to a multiple of a. a must be a power of 2.

func allgadd ¶

func allgadd(gp *g)

func appendIntStr ¶

func appendIntStr(b []byte, v int64, signed bool) []byte

func archauxv ¶

func archauxv(tag, val uintptr)

func arenaBase ¶

func arenaBase(i arenaIdx) uintptr

arenaBase returns the low address of the region covered by heap arena i.

func args ¶

func args(c int32, v **byte)

func argv_index ¶

func argv_index(argv **byte, i int32) *byte

nosplit for use in linux startup sysargs

func asmcgocall ¶

func asmcgocall(fn, arg unsafe.Pointer) int32

func asminit ¶

func asminit()

func assertE2I2 ¶

func assertE2I2(inter *interfacetype, e eface) (r iface, b bool)

func assertI2I2 ¶

func assertI2I2(inter *interfacetype, i iface) (r iface, b bool)

func assertLockHeld ¶

func assertLockHeld(l *mutex)

func assertRankHeld ¶

func assertRankHeld(r lockRank)

func assertWorldStopped ¶

func assertWorldStopped()

func assertWorldStoppedOrLockHeld ¶

func assertWorldStoppedOrLockHeld(l *mutex)

func asyncPreempt ¶

func asyncPreempt()

asyncPreempt saves all user registers and calls asyncPreempt2.

When stack scanning encounters an asyncPreempt frame, it scans that frame and its parent frame conservatively.

asyncPreempt is implemented in assembly.

func asyncPreempt2 ¶

func asyncPreempt2()

func atoi ¶

func atoi(s string) (int, bool)

atoi parses an int from a string s. The bool result reports whether s is a number representable by a value of type int.

func atoi32 ¶

func atoi32(s string) (int32, bool)

atoi32 is like atoi but for integers that fit into an int32.

func atomicAllG ¶

func atomicAllG() (**g, uintptr)

atomicAllG returns &allgs[0] and len(allgs) for use with atomicAllGIndex.

func atomicstorep ¶

func atomicstorep(ptr unsafe.Pointer, new unsafe.Pointer)

atomicstorep performs *ptr = new atomically and invokes a write barrier.

func atomicwb ¶

func atomicwb(ptr *unsafe.Pointer, new unsafe.Pointer)

atomicwb performs a write barrier before an atomic pointer write. The caller should guard the call with "if writeBarrier.enabled".

func badPointer ¶

func badPointer(s *mspan, p, refBase, refOff uintptr)

badPointer throws bad pointer in heap panic.

func badTimer ¶

func badTimer()

badTimer is called if the timer data structures have been corrupted, presumably due to racy use by the program. We panic here rather than panicing due to invalid slice access while holding locks. See issue #25686.

func badcgocallback ¶

func badcgocallback()

called from assembly

func badctxt ¶

func badctxt()

func badmcall ¶

func badmcall(fn func(*g))

called from assembly

func badmcall2 ¶

func badmcall2(fn func(*g))

func badmorestackg0 ¶

func badmorestackg0()

func badmorestackgsignal ¶

func badmorestackgsignal()

func badreflectcall ¶

func badreflectcall()

func badsignal ¶

func badsignal(sig uintptr, c *sigctxt)

This runs on a foreign stack, without an m or a g. No stack split.

func badsystemstack ¶

func badsystemstack()

func badunlockosthread ¶

func badunlockosthread()

func beforeIdle ¶

func beforeIdle(int64) (*g, bool)

func beforefork ¶

func beforefork()

func bgscavenge ¶

func bgscavenge(c chan int)

Background scavenger.

The background scavenger maintains the RSS of the application below the line described by the proportional scavenging statistics in the mheap struct.

func bgsweep ¶

func bgsweep(c chan int)

func binarySearchTree ¶

func binarySearchTree(x *stackObjectBuf, idx int, n int) (root *stackObject, restBuf *stackObjectBuf, restIdx int)

Build a binary search tree with the n objects in the list x.obj[idx], x.obj[idx+1], ..., x.next.obj[0], ... Returns the root of that tree, and the buf+idx of the nth object after x.obj[idx]. (The first object that was not included in the binary search tree.) If n == 0, returns nil, x.

func block ¶

func block()

func blockAlignSummaryRange ¶

func blockAlignSummaryRange(level int, lo, hi int) (int, int)

blockAlignSummaryRange aligns indices into the given level to that level's block width (1 << levelBits[level]). It assumes lo is inclusive and hi is exclusive, and so aligns them down and up respectively.

func blockableSig ¶

func blockableSig(sig uint32) bool

blockableSig reports whether sig may be blocked by the signal mask. We never want to block the signals marked _SigUnblock; these are the synchronous signals that turn into a Go panic. In a Go program--not a c-archive/c-shared--we never want to block the signals marked _SigKill or _SigThrow, as otherwise it's possible for all running threads to block them and delay their delivery until we start a new thread. When linked into a C program we let the C code decide on the disposition of those signals.

func blockevent ¶

func blockevent(cycles int64, skip int)

func blocksampled ¶

func blocksampled(cycles int64) bool

func bool2int ¶

func bool2int(x bool) int

bool2int returns 0 if x is false or 1 if x is true.

func breakpoint ¶

func breakpoint()

func bucketEvacuated ¶

func bucketEvacuated(t *maptype, h *hmap, bucket uintptr) bool

func bucketMask ¶

func bucketMask(b uint8) uintptr

bucketMask returns 1<<b - 1, optimized for code generation.

func bucketShift ¶

func bucketShift(b uint8) uintptr

bucketShift returns 1<<b, optimized for code generation.

func bulkBarrierBitmap ¶

func bulkBarrierBitmap(dst, src, size, maskOffset uintptr, bits *uint8)

bulkBarrierBitmap executes write barriers for copying from [src, src+size) to [dst, dst+size) using a 1-bit pointer bitmap. src is assumed to start maskOffset bytes into the data covered by the bitmap in bits (which may not be a multiple of 8).

This is used by bulkBarrierPreWrite for writes to data and BSS.

func bulkBarrierPreWrite ¶

func bulkBarrierPreWrite(dst, src, size uintptr)

bulkBarrierPreWrite executes a write barrier for every pointer slot in the memory range [src, src+size), using pointer/scalar information from [dst, dst+size). This executes the write barriers necessary before a memmove. src, dst, and size must be pointer-aligned. The range [dst, dst+size) must lie within a single object. It does not perform the actual writes.

As a special case, src == 0 indicates that this is being used for a memclr. bulkBarrierPreWrite will pass 0 for the src of each write barrier.

Callers should call bulkBarrierPreWrite immediately before calling memmove(dst, src, size). This function is marked nosplit to avoid being preempted; the GC must not stop the goroutine between the memmove and the execution of the barriers. The caller is also responsible for cgo pointer checks if this may be writing Go pointers into non-Go memory.

The pointer bitmap is not maintained for allocations containing no pointers at all; any caller of bulkBarrierPreWrite must first make sure the underlying allocation contains pointers, usually by checking typ.ptrdata.

Callers must perform cgo checks if writeBarrier.cgo.

func bulkBarrierPreWriteSrcOnly ¶

func bulkBarrierPreWriteSrcOnly(dst, src, size uintptr)

bulkBarrierPreWriteSrcOnly is like bulkBarrierPreWrite but does not execute write barriers for [dst, dst+size).

In addition to the requirements of bulkBarrierPreWrite callers need to ensure [dst, dst+size) is zeroed.

This is used for special cases where e.g. dst was just created and zeroed with malloc.

func bytes ¶

func bytes(s string) (ret []byte)

func bytesHash ¶

func bytesHash(b []byte, seed uintptr) uintptr

func c128equal ¶

func c128equal(p, q unsafe.Pointer) bool

func c128hash ¶

func c128hash(p unsafe.Pointer, h uintptr) uintptr

func c64equal ¶

func c64equal(p, q unsafe.Pointer) bool

func c64hash ¶

func c64hash(p unsafe.Pointer, h uintptr) uintptr

func call1024 ¶

func call1024(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call1048576 ¶

func call1048576(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call1073741824 ¶

func call1073741824(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call128 ¶

func call128(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call131072 ¶

func call131072(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call134217728 ¶

func call134217728(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call16 ¶

func call16(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

in asm_*.s not called directly; definitions here supply type information for traceback.

func call16384 ¶

func call16384(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call16777216 ¶

func call16777216(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call2048 ¶

func call2048(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call2097152 ¶

func call2097152(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call256 ¶

func call256(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call262144 ¶

func call262144(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call268435456 ¶

func call268435456(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call32 ¶

func call32(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call32768 ¶

func call32768(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call33554432 ¶

func call33554432(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call4096 ¶

func call4096(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call4194304 ¶

func call4194304(typ, fn, arg unsafe.Pointer, n, retoffset uint32)

func call512 ¶

func call512(typ, fn, arg unsafe.Pointer, n, retoffset uint32)