Thank you for this report @WangLeonard!

Kindly pinging @cherrymui @randall77.

Thanks @WangLeonard and @odeke-em.

/cc @4ad, who I believe this was authored a good portion of this in golang.org/cl/7142.

If you can't provide an example of failing code, could you please provide more details about the failure you encountered?

Current go documentation does not specify any memory order guarantees made by the atomic operations, but there is some agreements on the memory order. Please refer to this CL https://go-review.googlesource.com/c/go/+/189417/ and this discussion https://groups.google.com/forum/#!msg/golang-dev/vVkH_9fl1D8/azJa10lkAwAJ.

The reason why CAS uses exclusive mode Load/Store instructions is to guarantee multiple opeations of a single process are atomic accesses. But there is no obvious reason that we have to use exclusive load/store instructions for atomic Load/Store.
Could you share why do you think need to use exclusive mode instructions?

Can you help to check your code to confirm whether it follows the rule "you shouldn't mix atomic and non-atomic accesses for a given memory word."? By the way, we also don't guarantee the memory order among atomic and non-atomic access on different memory.

Additional notes: The following is a description of these instructions in the arm architecture reference manual.
LDAR: Load-Acquire Register derives an address from a base register value, loads a 32-bit word or 64-bit doubleword from memory, and writes it to a register
STLR: Store-Release Register stores a 32-bit word or a 64-bit doubleword to a memory location, from a register.
LDAXR: Load-Acquire Exclusive Register derives an address from a base register value, loads a 32-bit word or 64-bit doubleword from memory, and writes it to a register. The memory access is atomic. The PE marks the physical address being accessed as an exclusive access.
STLXR: Store-Release Exclusive Register stores a 32-bit word or a 64-bit doubleword to memory if the PE has exclusive access to the memory address, from two registers, and returns a status value of 0 if the store was successful, or of 1 if no store was performed.The memory access is atomic.

@toothrot @zhangfannie
my program looks like the above, but this demo can't reproduce my problem, this is just a simplified version， my application will be more complicated, add some cgo and syscall，more concurrent etc.

I added some positioning and found that when there is a problem with the application,
The state of wrap2's goroutine is Gwaiting,
The reason is waitReasonChanReceive,
The value of num is 1,
So suspected to be atomic.StoreInt32 failed.
my environment is arm64 V8 , linux.
For the problem variable, only one part of the program does not use atomic, but can be considered as a deadcode, will this affect the compiler?

I can't reproduce at Go1.13.1

package main

import (
        "log"
        "runtime"
        "sync/atomic"
        "time"
)

var num int32

func wrap1(ch chan bool) {
        go func() {
                for {
                        // if num is 0, change to 1.
                        if atomic.CompareAndSwapInt32(&num, 0, 1) {
                                ch <- true
                        }
                        // Do something.
                        log.Print("wrap1")
                        time.Sleep(time.Millisecond)
                }
        }()
}

func wrap2(ch chan bool) {
        runtime.LockOSThread()

        for range ch {
                time.Sleep(time.Millisecond)
                atomic.StoreInt32(&num, 0)
                log.Print("wrap2")
        }
}

func main() {
        ch := make(chan bool, 1)
        wrap1(ch)
        wrap2(ch)
}

2019/11/19 14:50:37 wrap1
2019/11/19 14:50:37 wrap1
2019/11/19 14:50:37 wrap2
2019/11/19 14:50:37 wrap1
2019/11/19 14:50:37 wrap1
2019/11/19 14:50:37 wrap2
……

@mengzhuo
Yes, this demo can't reproduce the problem, just to simplify the scenario that describes the application.
Now just guessing from some phenomena and states, and I changed StoreInt32 to CompareAndSwapInt32 to seem to solve this problem, I am still confirming.

@WangLeonard what is the problem?

The state of wrap2's goroutine is Gwaiting,
The reason is waitReasonChanReceive,
The value of num is 1,
So suspected to be atomic.StoreInt32 failed.

Why? It could be that the atomic store succeeded, then the CAS in wrap1 succeeded, but the scheduler hasn't woken the wrap2 goroutine.

I want to mention that GCC also generates LDAR/STLR for atomic load/store.

uint64_t Load64(uint64_t *p)              { return __atomic_load_n (p, __ATOMIC_SEQ_CST); }
void     Store64(uint64_t *p, uint64_t v) { __atomic_store_n (p, v, __ATOMIC_SEQ_CST); }

Load64:
.LFB0:
	.cfi_startproc
	ldar	x0, [x0]
	ret
	.cfi_endproc

Store64:
.LFB1:
	.cfi_startproc
	stlr	x1, [x0]
	ret
	.cfi_endproc

@cherrymui
I just suspect that StoreInt32 failed. This is a rather strange question.

For what you said the scheduler hasn't woken the wrap2 goroutine.
Because I used LockOSThread, I counted the number of executions of the three locations in the schedule() function:
before stoplockedm() 、 after stoplockedm()、startlockedm(gp)
found that:
before stoplockedm = after stoplockedm +1
before stoplockedm = startlockedm(gp)+1
And these three values no longer increase(more than a few hours) when a bug occurs.

So I guess chan is not written when a bug occurs,
wrap2 goroutine not put into the local runnable queue,
schedule() can not find this G to M execution.

When I can reproduce the problem or locate other reasons, I will provide more information.
Thank you!