Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to imply memory barrier in go? #43031

Closed
zhangshishen opened this issue Dec 6, 2020 · 3 comments
Closed

How to imply memory barrier in go? #43031

zhangshishen opened this issue Dec 6, 2020 · 3 comments

Comments

@zhangshishen
Copy link

What version of Go are you using (go version)?

$ go version 1.15

What operating system and processor architecture are you using (go env)?

amd64

What did you do?

I implemented an inter-process shared memory queue with Golang.
I use an atomic.Value as an IPC mutex, but has problem on memory access sequence.
eg:
process 1 has:

var lock atomic.Value = (*byte)(shared_memory_addr + 0)
var ptr *byte = (*byte)(shared_memory_addr + 1000)
*ptr = 1
lock.Store(1)

process 2 has:

var lock atomic.Value = (*byte)(shared_memory_addr + 0)
var ptr *byte = (*byte)(shared_memory_addr + 1000)
for lock.Load() != 1 {
      continue
}
printf("%d",*ptr) // maybe not 1

What did you expect to see?

the *ptr should always be 1, but it sometimes can be 0.

Should I use assembly smb() or sth to ensure the correct order?

Is there some function like C++ std::atomic_thread_fence in golang?

@randall77
Copy link
Contributor

You should not need a fence to make that work. If an (*atomic.Value).Loadobserves the write from an (*atomic.Value).Store, then all future loads after the (*atomic.Value).Load should see all the stores before the (*atomic.Value).Store.

Can you show us a full program that demonstrates the problem?

Make sure you're obeying the restrictions on atomic.Value (no copying, multiple stores all have to be the same type).

@zhangshishen
Copy link
Author

Full program is:

package mq

import (
	"gw/pkg/shm"
	"gw/pkg/logs"
	"errors"
	"fmt"
	"reflect"
	"sync/atomic"
	"unsafe"
)

const headerSize = 1024
const packetHeaderSize = 4 + 4


func GetUint32(ptr uintptr) uint32 {
	return atomic.LoadUint32((*uint32)(unsafe.Pointer(ptr)))
}

func SetUint32(ptr uintptr, value uint32) {
	atomic.StoreUint32((*uint32)(unsafe.Pointer(ptr)), value)
}

func getAlignedSize(l uint32) uint32 {
	return (l + 7) / 8 * 8
}

type ShmHeader struct {
	ptr uintptr
}

func (h *ShmHeader) GetWriteBytes() uint32 {
	return GetUint32(h.ptr)
}
func (h *ShmHeader) SetWriteBytes(b uint32) {
	SetUint32(h.ptr, b)
}
func (h *ShmHeader) GetWriteOffset() uint32 {
	return GetUint32(h.ptr + 4)
}
func (h *ShmHeader) SetWriteOffset(b uint32) {
	SetUint32(h.ptr+4, b)
}
func (h *ShmHeader) GetReadBytes() uint32 {
	return GetUint32(h.ptr + 8)
}
func (h *ShmHeader) SetReadBytes(b uint32) {
	SetUint32(h.ptr+8, b)
}
func (h *ShmHeader) GetReadOffset() uint32 {
	return GetUint32(h.ptr + 12)
}
func (h *ShmHeader) SetReadOffset(b uint32) {
	SetUint32(h.ptr+12, b)
}

type PacketHeader struct {
	ptr uintptr
}

func (h *PacketHeader) GetLen() uint32 {
	return GetUint32(h.ptr)
}
func (h *PacketHeader) SetLen(l uint32) {
	SetUint32(h.ptr, l)
}
func (h PacketHeader) GetSeq() uint32 {
	return GetUint32(h.ptr + 4)
}
func (h PacketHeader) SetSeq(l uint32) {
	SetUint32(h.ptr+4, l)
}

type MqShm struct {
	shm           *shm.Memory
	header        ShmHeader
	content       []byte
	contentSize   uint32
	maxPacketSize uint32
	diffMax       uint32
	seq           uint32
	seqRead       uint32

}

func NewMqShm(name string, sizeData int32, maxPacketSize uint32, isCreate bool) (*MqShm, error) {
	size := sizeData + headerSize
	var m *shm.Memory
	var err error

	if isCreate {
		shm.Delete(name)
		m, err = shm.Create(name, size)
	} else {
		m, err = shm.Open(name, size)
	}
	if err != nil {
		logs.Error("shm: create error:name=%s,size=%d", name, size)
		return nil, err
	}

	header := ShmHeader{ptr: m.GetPtr()}

	var content []byte
	hc := (*reflect.SliceHeader)(unsafe.Pointer(&content))
	hc.Cap = int(sizeData)
	hc.Len = int(sizeData)
	hc.Data = m.GetPtr() + headerSize

	ret := &MqShm{
		shm:           m,
		header:        header,
		content:       content,
		contentSize:   uint32(sizeData),
		maxPacketSize: maxPacketSize,
	}
	//logs.Info("shm: create:name=%s, size=%d", name, size)
	return ret, nil
}

func (s *MqShm) Stop() {
	//logs.Info("shm: stop")
	s.shm.Close()
}

func (s *MqShm) Start() error {
	return nil
}

func (s *MqShm) getHeadInfo() string {
	rOff := s.header.GetReadOffset()
	wOff := s.header.GetWriteOffset()
	diffOff := uint32((((int)(wOff)-(int)(rOff))%(int)(s.contentSize) + (int)(s.contentSize)) % (int)(s.contentSize))
	rByte := s.header.GetReadBytes()
	wByte := s.header.GetWriteBytes()
	diffByte := wByte - rByte

	sRet := fmt.Sprintf("diffOff=%d, diffByte=%d, roff=%d, rbytes=%d, woff=%d, wbytes=%d", diffOff, diffByte, rOff, rByte, wOff, wByte)
	return sRet
}

func (s *MqShm) Read(buf []byte) (int, uint32, error) {
	//1. check empty
	if s.IsEmpty() {

		return 0, 0, nil
	}
	//2. read data
	start := s.header.GetReadOffset()
	packet := PacketHeader{ptr: s.header.ptr + uintptr(headerSize+start)}
	plen := int(packet.GetLen())
	uplen := getAlignedSize(uint32(plen))
	seq := packet.GetSeq()
	if plen == 0 {
		serr := fmt.Sprintf("read packet 0 size: %s", s.getHeadInfo())
		logs.Error(serr)
		return 0, 0, errors.New(serr)
	}
	//logs.Info("shm: read len=%d, seq=%d, roff=%d", plen, seq, s.header.GetReadOffset())
	if len(buf) < plen {
		serr := fmt.Sprintf("read packet too large: size=%d %s", plen, s.getHeadInfo())
		logs.Error(serr)
		return 0, 0, errors.New(serr)
	}

	s.seqRead = seq
	copy(buf, s.content[start+packetHeaderSize:uplen+start+packetHeaderSize])
	//3.move read ptr
	s.header.SetReadBytes(s.header.GetReadBytes() + uplen)
	//logs.Info("shm3: read len=%d, seq=%d, roff=%d", plen, seq, s.header.GetReadOffset())

	roff := s.header.GetReadOffset() + uplen + packetHeaderSize
	if roff >= s.contentSize-s.maxPacketSize-packetHeaderSize {
		roff = 0
	}
	s.header.SetReadOffset(roff)
	//logs.Info("shm: read(%d) len=%d, at pos:%d, next off:%d", seq, plen, start, roff)
	return plen, seq, nil
}

func (s *MqShm) Write(buf []byte) (int, uint32, error) {
	//0. check param
	plen := len(buf)
	uplen := getAlignedSize(uint32(plen))
	if uplen > s.maxPacketSize {
		logs.Error("shm: write too large")
		return 0, 0, fmt.Errorf("write buffer too large:%d", plen)
	}
	if plen == 0 {
		logs.Error("shm: write 0 size")
		return 0, 0, fmt.Errorf("write 0 size")
	}
	//1. check full
	if s.IsFull() {
		//s.checkLogOnce("write full")
		return 0, 0, nil
	}
	//2. write data
	start := s.header.GetWriteOffset()
	packet := PacketHeader{ptr: s.header.ptr + uintptr(headerSize+start)}
	packet.SetLen(uint32(plen))
	s.seq++
	packet.SetSeq(s.seq)

	copy(s.content[start+packetHeaderSize:uplen+start+packetHeaderSize], buf)

	//3.move write ptr
	s.header.SetWriteBytes(s.header.GetWriteBytes() + uplen)
	woff := s.header.GetWriteOffset() + uplen + packetHeaderSize
	if woff >= s.contentSize-s.maxPacketSize-packetHeaderSize {
		woff = 0
	}
	s.header.SetWriteOffset(woff)

	return plen, s.seq, nil
}

func (s *MqShm) IsEmpty() bool {
	r := s.header.GetReadBytes()
	w := s.header.GetWriteBytes()
	diff := w - r
	rOff := s.header.GetReadOffset()
	wOff := s.header.GetWriteOffset()
	return diff == 0 || (diff > 0x80000000 && rOff == wOff)
}

func (s *MqShm) IsFull() bool {
	if s.IsEmpty() {
		return false
	}
	rOff := s.header.GetReadOffset()
	wOff := s.header.GetWriteOffset()
	diff := uint32((((int)(wOff)-(int)(rOff))%(int)(s.contentSize) + (int)(s.contentSize)) % (int)(s.contentSize))
	if s.diffMax < diff {
		s.diffMax = diff
	}
	diffMax := s.contentSize - (s.maxPacketSize+packetHeaderSize)*2
	ret := diff > diffMax
	if ret {
		//logs.Infof("shm: full: diff = %d, roff:%d, woff:%d", diff, rOff, wOff)
	}
	return ret
}

func (s *MqShm) GetMaxUsed() uint32 {
	return s.diffMax
}

I checked the disassembly of Read:

TEXT gw/internal/rtcmq.(*MqShm).Read(SB) gw/internal/rtcmq/mq_shm.go
  mq_shm.go:143		0x46f58c0		65488b0c2530000000		MOVQ GS:0x30, CX								
  mq_shm.go:143		0x46f58c9		488d4424c0			LEAQ -0x40(SP), AX								
  mq_shm.go:143		0x46f58ce		483b4110			CMPQ 0x10(CX), AX								
  mq_shm.go:143		0x46f58d2		0f8692040000			JBE 0x46f5d6a									
  mq_shm.go:143		0x46f58d8		4881ecc0000000			SUBQ $0xc0, SP									
  mq_shm.go:143		0x46f58df		4889ac24b8000000		MOVQ BP, 0xb8(SP)								
  mq_shm.go:143		0x46f58e7		488dac24b8000000		LEAQ 0xb8(SP), BP								
  mq_shm.go:145		0x46f58ef		488b8424c8000000		MOVQ 0xc8(SP), AX								
  mq_shm.go:145		0x46f58f7		48890424			MOVQ AX, 0(SP)									
  mq_shm.go:145		0x46f58fb		e8c0070000			CALL gw/internal/rtcmq.(*MqShm).IsEmpty(SB)		
  mq_shm.go:145		0x46f5900		807c240800			CMPB $0x0, 0x8(SP)								
  mq_shm.go:145		0x46f5905		0f8522040000			JNE 0x46f5d2d									
  mq_shm.go:150		0x46f590b		488b9c24c8000000		MOVQ 0xc8(SP), BX								
  mq_shm.go:150		0x46f5913		488b7308			MOVQ 0x8(BX), SI								
  mq_shm.go:52		0x46f5917		4883c60c			ADDQ $0xc, SI									
  mq_shm.go:18		0x46f591b		8b36				MOVL 0(SI), SI									
  mq_shm.go:151		0x46f591d		48c744245800000000		MOVQ $0x0, 0x58(SP)								
  mq_shm.go:151		0x46f5926		488b7b08			MOVQ 0x8(BX), DI								
  mq_shm.go:151		0x46f592a		448d8600040000			LEAL 0x400(SI), R8								
  mq_shm.go:151		0x46f5931		4c01c7				ADDQ R8, DI									
  mq_shm.go:151		0x46f5934		48897c2458			MOVQ DI, 0x58(SP)								
  mq_shm.go:63		0x46f5939		90				NOPL										
  mq_shm.go:18		0x46f593a		8b3f				MOVL 0(DI), DI									
  mq_shm.go:153		0x46f593c		90				NOPL										
  mq_shm.go:154		0x46f593d		4c8b442458			MOVQ 0x58(SP), R8								
  mq_shm.go:69		0x46f5942		4983c004			ADDQ $0x4, R8									
  mq_shm.go:18		0x46f5946		458b00				MOVL 0(R8), R8									
  mq_shm.go:152		0x46f5949		4189f9				MOVL DI, R9									
  mq_shm.go:155		0x46f594c		4d85c9				TESTQ R9, R9									
  mq_shm.go:155		0x46f594f		0f849f020000			JE 0x46f5bf4									
  mq_shm.go:152		0x46f5955		4c894c2450			MOVQ R9, 0x50(SP)								
  mq_shm.go:161		0x46f595a		4c8b9424d8000000		MOVQ 0xd8(SP), R10								
  mq_shm.go:161		0x46f5962		4d39ca				CMPQ R9, R10									
  mq_shm.go:161		0x46f5965		0f8cfa000000			JL 0x46f5a65									
  mq_shm.go:167		0x46f596b		44894338			MOVL R8, 0x38(BX)								
  mq_shm.go:168		0x46f596f		488b5320			MOVQ 0x20(BX), DX								
  mq_shm.go:168		0x46f5973		4c8b5b10			MOVQ 0x10(BX), R11								
  mq_shm.go:26		0x46f5977		83c707				ADDL $0x7, DI									
  mq_shm.go:26		0x46f597a		c1ef03				SHRL $0x3, DI									
  mq_shm.go:168		0x46f597d		8d0cfe				LEAL 0(SI)(DI*8), CX								
  mq_shm.go:168		0x46f5980		8d4908				LEAL 0x8(CX), CX								
  mq_shm.go:168		0x46f5983		4839d1				CMPQ DX, CX									
  mq_shm.go:168		0x46f5986		0f87d8030000			JA 0x46f5d64									
  mq_shm.go:168		0x46f598c		8d4608				LEAL 0x8(SI), AX								
  mq_shm.go:168		0x46f598f		4839c8				CMPQ CX, AX									
  mq_shm.go:168		0x46f5992		0f87c7030000			JA 0x46f5d5f									
  mq_shm.go:168		0x46f5998		4829c1				SUBQ AX, CX									
  mq_shm.go:168		0x46f599b		4939ca				CMPQ CX, R10									
  mq_shm.go:168		0x46f599e		4c0f4fd1			CMOVG CX, R10									
  mq_shm.go:168		0x46f59a2		89c1				MOVL AX, CX									
  mq_shm.go:168		0x46f59a4		4829d0				SUBQ DX, AX									
  mq_shm.go:168		0x46f59a7		48c1f83f			SARQ $0x3f, AX									
  mq_shm.go:168		0x46f59ab		4821c1				ANDQ AX, CX									
  mq_shm.go:168		0x46f59ae		498d040b			LEAQ 0(R11)(CX*1), AX								
  mq_shm.go:168		0x46f59b2		488b8c24d0000000		MOVQ 0xd0(SP), CX								
  mq_shm.go:168		0x46f59ba		4839c1				CMPQ AX, CX									
  mq_shm.go:168		0x46f59bd		756f				JNE 0x46f5a2e									
  mq_shm.go:46		0x46f59bf		488b4308			MOVQ 0x8(BX), AX								
  mq_shm.go:46		0x46f59c3		4883c008			ADDQ $0x8, AX									
  mq_shm.go:18		0x46f59c7		8b00				MOVL 0(AX), AX									
  mq_shm.go:170		0x46f59c9		8d04f8				LEAL 0(AX)(DI*8), AX								
  mq_shm.go:49		0x46f59cc		488b4b08			MOVQ 0x8(BX), CX								
  mq_shm.go:49		0x46f59d0		4883c108			ADDQ $0x8, CX									
  mq_shm.go:22		0x46f59d4		8701				XCHGL AX, 0(CX)									
  mq_shm.go:52		0x46f59d6		488b4308			MOVQ 0x8(BX), AX								
  mq_shm.go:52		0x46f59da		4883c00c			ADDQ $0xc, AX									
  mq_shm.go:18		0x46f59de		8b00				MOVL 0(AX), AX									
  mq_shm.go:173		0x46f59e0		8d04f8				LEAL 0(AX)(DI*8), AX								
  mq_shm.go:173		0x46f59e3		8d4008				LEAL 0x8(AX), AX								
  mq_shm.go:174		0x46f59e6		8b4b28				MOVL 0x28(BX), CX								
  mq_shm.go:174		0x46f59e9		2b4b2c				SUBL 0x2c(BX), CX								
  mq_shm.go:174		0x46f59ec		83c1f8				ADDL $-0x8, CX									
  mq_shm.go:174		0x46f59ef		39c8				CMPL CX, AX									
  mq_shm.go:177		0x46f59f1		b900000000			MOVL $0x0, CX									
  mq_shm.go:177		0x46f59f6		0f43c1				CMOVAE CX, AX									
  mq_shm.go:55		0x46f59f9		488b4b08			MOVQ 0x8(BX), CX								
  mq_shm.go:55		0x46f59fd		4883c10c			ADDQ $0xc, CX									
  mq_shm.go:22		0x46f5a01		8701				XCHGL AX, 0(CX)									
  mq_shm.go:179		0x46f5a03		4c898c24e8000000		MOVQ R9, 0xe8(SP)								
  mq_shm.go:179		0x46f5a0b		44898424f0000000		MOVL R8, 0xf0(SP)								
  mq_shm.go:179		0x46f5a13		0f57c0				XORPS X0, X0									
  mq_shm.go:179		0x46f5a16		0f118424f8000000		MOVUPS X0, 0xf8(SP)								
  mq_shm.go:174		0x46f5a1e		488bac24b8000000		MOVQ 0xb8(SP), BP								
  mq_shm.go:174		0x46f5a26		4881c4c0000000			ADDQ $0xc0, SP									
  mq_shm.go:174		0x46f5a2d		c3				RET										
  mq_shm.go:18		0x46f5a2e		4489442438			MOVL R8, 0x38(SP)								
  mq_shm.go:26		0x46f5a33		897c243c			MOVL DI, 0x3c(SP)								
  mq_shm.go:168		0x46f5a37		48890c24			MOVQ CX, 0(SP)									
  mq_shm.go:168		0x46f5a3b		4889442408			MOVQ AX, 0x8(SP)								
  mq_shm.go:168		0x46f5a40		4c89542410			MOVQ R10, 0x10(SP)								
  mq_shm.go:168		0x46f5a45		e8364797ff			CALL runtime.memmove(SB)							
  mq_shm.go:46		0x46f5a4a		488b9c24c8000000		MOVQ 0xc8(SP), BX								
  mq_shm.go:170		0x46f5a52		8b7c243c			MOVL 0x3c(SP), DI								
  mq_shm.go:179		0x46f5a56		448b442438			MOVL 0x38(SP), R8								
  mq_shm.go:179		0x46f5a5b		4c8b4c2450			MOVQ 0x50(SP), R9								
  mq_shm.go:168		0x46f5a60		e95affffff			JMP 0x46f59bf									
  mq_shm.go:162		0x46f5a65		48891c24			MOVQ BX, 0(SP)									
  mq_shm.go:162		0x46f5a69		e812fcffff			CALL gw/internal/rtcmq.(*MqShm).getHeadInfo(SB)	
  mq_shm.go:162		0x46f5a6e		488b442410			MOVQ 0x10(SP), AX								
  mq_shm.go:162		0x46f5a73		4889442460			MOVQ AX, 0x60(SP)								
  mq_shm.go:162		0x46f5a78		488b4c2408			MOVQ 0x8(SP), CX								
  mq_shm.go:162		0x46f5a7d		48898c2480000000		MOVQ CX, 0x80(SP)								
  mq_shm.go:162		0x46f5a85		488b542450			MOVQ 0x50(SP), DX								
  mq_shm.go:162		0x46f5a8a		48891424			MOVQ DX, 0(SP)									
  mq_shm.go:162		0x46f5a8e		e8ad6d91ff			CALL runtime.convT64(SB)							
  mq_shm.go:162		0x46f5a93		488b442408			MOVQ 0x8(SP), AX								
  mq_shm.go:162		0x46f5a98		4889442478			MOVQ AX, 0x78(SP)								
  mq_shm.go:162		0x46f5a9d		488b8c2480000000		MOVQ 0x80(SP), CX								
  mq_shm.go:162		0x46f5aa5		48890c24			MOVQ CX, 0(SP)									
  mq_shm.go:162		0x46f5aa9		488b4c2460			MOVQ 0x60(SP), CX								
  mq_shm.go:162		0x46f5aae		48894c2408			MOVQ CX, 0x8(SP)								
  mq_shm.go:162		0x46f5ab3		e8086e91ff			CALL runtime.convTstring(SB)							
  mq_shm.go:162		0x46f5ab8		488b442410			MOVQ 0x10(SP), AX								
  mq_shm.go:162		0x46f5abd		0f57c0				XORPS X0, X0									
  mq_shm.go:162		0x46f5ac0		0f11842498000000		MOVUPS X0, 0x98(SP)								
  mq_shm.go:162		0x46f5ac8		0f118424a8000000		MOVUPS X0, 0xa8(SP)									
  mq_shm.go:162		0x46f5ad0		488d0d29f10800			LEAQ runtime.rodata+479744(SB), CX						
  mq_shm.go:162		0x46f5ad7		48898c2498000000		MOVQ CX, 0x98(SP)								
  mq_shm.go:162		0x46f5adf		488b4c2478			MOVQ 0x78(SP), CX								
  mq_shm.go:162		0x46f5ae4		48898c24a0000000		MOVQ CX, 0xa0(SP)								
  mq_shm.go:162		0x46f5aec		488d0d8dfd0800			LEAQ runtime.rodata+482944(SB), CX						
  mq_shm.go:162		0x46f5af3		48898c24a8000000		MOVQ CX, 0xa8(SP)								
  mq_shm.go:162		0x46f5afb		48898424b0000000		MOVQ AX, 0xb0(SP)								
  mq_shm.go:162		0x46f5b03		488d052cbc1f00			LEAQ go.string.*+189518(SB), AX							
  mq_shm.go:162		0x46f5b0a		48890424			MOVQ AX, 0(SP)									
  mq_shm.go:162		0x46f5b0e		48c744240821000000		MOVQ $0x21, 0x8(SP)								
  mq_shm.go:162		0x46f5b17		488d842498000000		LEAQ 0x98(SP), AX								
  mq_shm.go:162		0x46f5b1f		4889442410			MOVQ AX, 0x10(SP)								
  mq_shm.go:162		0x46f5b24		48c744241802000000		MOVQ $0x2, 0x18(SP)								
  mq_shm.go:162		0x46f5b2d		48c744242002000000		MOVQ $0x2, 0x20(SP)								
  mq_shm.go:162		0x46f5b36		e8d5999eff			CALL fmt.Sprintf(SB)								
  mq_shm.go:162		0x46f5b3b		488b442430			MOVQ 0x30(SP), AX								
  mq_shm.go:162		0x46f5b40		4889442440			MOVQ AX, 0x40(SP)								
  mq_shm.go:162		0x46f5b45		488b4c2428			MOVQ 0x28(SP), CX								
  mq_shm.go:162		0x46f5b4a		48894c2470			MOVQ CX, 0x70(SP)								
  mq_shm.go:163		0x46f5b4f		90				NOPL										
  logs.go:87		0x46f5b50		488b1529bc9600			MOVQ gw/gopkg/logs.defaultLogger(SB), DX				
  logs.go:87		0x46f5b57		48891424			MOVQ DX, 0(SP)									
  logs.go:87		0x46f5b5b		48894c2408			MOVQ CX, 0x8(SP)								
  logs.go:87		0x46f5b60		4889442410			MOVQ AX, 0x10(SP)								
  logs.go:87		0x46f5b65		0f57c0				XORPS X0, X0									
  logs.go:87		0x46f5b68		0f11442418			MOVUPS X0, 0x18(SP)								
  logs.go:87		0x46f5b6d		48c744242800000000		MOVQ $0x0, 0x28(SP)								
  logs.go:87		0x46f5b76		e81560d7ff			CALL gw/gopkg/logs.(*Logger).Error(SB)				
  errors.go:59		0x46f5b7b		488d059e650f00			LEAQ runtime.rodata+902944(SB), AX						
  errors.go:59		0x46f5b82		48890424			MOVQ AX, 0(SP)									
  errors.go:59		0x46f5b86		e8b59a91ff			CALL runtime.newobject(SB)							
  errors.go:59		0x46f5b8b		488b7c2408			MOVQ 0x8(SP), DI								
  errors.go:59		0x46f5b90		488b442440			MOVQ 0x40(SP), AX								
  errors.go:59		0x46f5b95		48894708			MOVQ AX, 0x8(DI)								
  errors.go:59		0x46f5b99		833d70c4990000			CMPL $0x0, runtime.writeBarrier(SB)						
  errors.go:59		0x46f5ba0		7546				JNE 0x46f5be8									
  errors.go:59		0x46f5ba2		488b442470			MOVQ 0x70(SP), AX								
  errors.go:59		0x46f5ba7		488907				MOVQ AX, 0(DI)									
  mq_shm.go:164		0x46f5baa		48c78424e800000000000000	MOVQ $0x0, 0xe8(SP)								
  mq_shm.go:164		0x46f5bb6		c78424f000000000000000		MOVL $0x0, 0xf0(SP)								
  mq_shm.go:164		0x46f5bc1		488d05b8692f00			LEAQ go.itab.*errors.errorString,error(SB), AX					
  mq_shm.go:164		0x46f5bc8		48898424f8000000		MOVQ AX, 0xf8(SP)								
  mq_shm.go:164		0x46f5bd0		4889bc2400010000		MOVQ DI, 0x100(SP)								
  mq_shm.go:164		0x46f5bd8		488bac24b8000000		MOVQ 0xb8(SP), BP								
  mq_shm.go:164		0x46f5be0		4881c4c0000000			ADDQ $0xc0, SP									
  mq_shm.go:164		0x46f5be7		c3				RET										
  errors.go:59		0x46f5be8		488b442470			MOVQ 0x70(SP), AX								
  errors.go:59		0x46f5bed		e8ee3397ff			CALL runtime.gcWriteBarrier(SB)							
  errors.go:59		0x46f5bf2		ebb6				JMP 0x46f5baa									
  mq_shm.go:156		0x46f5bf4		48891c24			MOVQ BX, 0(SP)									
  mq_shm.go:156		0x46f5bf8		e883faffff			CALL gw/internal/rtcmq.(*MqShm).getHeadInfo(SB)	
  mq_shm.go:156		0x46f5bfd		488b442410			MOVQ 0x10(SP), AX								
  mq_shm.go:156		0x46f5c02		488b4c2408			MOVQ 0x8(SP), CX								
  mq_shm.go:156		0x46f5c07		48890c24			MOVQ CX, 0(SP)									
  mq_shm.go:156		0x46f5c0b		4889442408			MOVQ AX, 0x8(SP)								
  mq_shm.go:156		0x46f5c10		e8ab6c91ff			CALL runtime.convTstring(SB)							
  mq_shm.go:156		0x46f5c15		488b442410			MOVQ 0x10(SP), AX								
  mq_shm.go:156		0x46f5c1a		0f57c0				XORPS X0, X0									
  mq_shm.go:156		0x46f5c1d		0f11842488000000		MOVUPS X0, 0x88(SP)								
  mq_shm.go:156		0x46f5c25		488d0d54fc0800			LEAQ runtime.rodata+482944(SB), CX						
  mq_shm.go:156		0x46f5c2c		48898c2488000000		MOVQ CX, 0x88(SP)								
  mq_shm.go:156		0x46f5c34		4889842490000000		MOVQ AX, 0x90(SP)								
  mq_shm.go:156		0x46f5c3c		488d05b1091f00			LEAQ go.string.*+144140(SB), AX							
  mq_shm.go:156		0x46f5c43		48890424			MOVQ AX, 0(SP)									
  mq_shm.go:156		0x46f5c47		48c744240816000000		MOVQ $0x16, 0x8(SP)								
  mq_shm.go:156		0x46f5c50		488d842488000000		LEAQ 0x88(SP), AX								
  mq_shm.go:156		0x46f5c58		4889442410			MOVQ AX, 0x10(SP)								
  mq_shm.go:156		0x46f5c5d		48c744241801000000		MOVQ $0x1, 0x18(SP)								
  mq_shm.go:156		0x46f5c66		48c744242001000000		MOVQ $0x1, 0x20(SP)								
  mq_shm.go:156		0x46f5c6f		e89c989eff			CALL fmt.Sprintf(SB)								
  mq_shm.go:156		0x46f5c74		488b442430			MOVQ 0x30(SP), AX								
  mq_shm.go:156		0x46f5c79		4889442448			MOVQ AX, 0x48(SP)								
  mq_shm.go:156		0x46f5c7e		488b4c2428			MOVQ 0x28(SP), CX								
  mq_shm.go:156		0x46f5c83		48894c2468			MOVQ CX, 0x68(SP)								
  mq_shm.go:157		0x46f5c88		90				NOPL										
  logs.go:87		0x46f5c89		488b15f0ba9600			MOVQ gw/gopkg/logs.defaultLogger(SB), DX				
  logs.go:87		0x46f5c90		48891424			MOVQ DX, 0(SP)									
  logs.go:87		0x46f5c94		48894c2408			MOVQ CX, 0x8(SP)								
  logs.go:87		0x46f5c99		4889442410			MOVQ AX, 0x10(SP)								
  logs.go:87		0x46f5c9e		0f57c0				XORPS X0, X0									
  logs.go:87		0x46f5ca1		0f11442418			MOVUPS X0, 0x18(SP)								
  logs.go:87		0x46f5ca6		48c744242800000000		MOVQ $0x0, 0x28(SP)								
  logs.go:87		0x46f5caf		e8dc5ed7ff			CALL gw/gopkg/logs.(*Logger).Error(SB)				
  errors.go:59		0x46f5cb4		488d0565640f00			LEAQ runtime.rodata+902944(SB), AX						
  errors.go:59		0x46f5cbb		48890424			MOVQ AX, 0(SP)									
  errors.go:59		0x46f5cbf		e87c9991ff			CALL runtime.newobject(SB)							
  errors.go:59		0x46f5cc4		488b7c2408			MOVQ 0x8(SP), DI								
  errors.go:59		0x46f5cc9		488b442448			MOVQ 0x48(SP), AX								
  errors.go:59		0x46f5cce		48894708			MOVQ AX, 0x8(DI)								
  errors.go:59		0x46f5cd2		833d37c3990000			CMPL $0x0, runtime.writeBarrier(SB)						
  errors.go:59		0x46f5cd9		7546				JNE 0x46f5d21									
  errors.go:59		0x46f5cdb		488b442468			MOVQ 0x68(SP), AX								
  errors.go:59		0x46f5ce0		488907				MOVQ AX, 0(DI)									
  mq_shm.go:158		0x46f5ce3		48c78424e800000000000000	MOVQ $0x0, 0xe8(SP)								
  mq_shm.go:158		0x46f5cef		c78424f000000000000000		MOVL $0x0, 0xf0(SP)								
  mq_shm.go:158		0x46f5cfa		488d057f682f00			LEAQ go.itab.*errors.errorString,error(SB), AX					
  mq_shm.go:158		0x46f5d01		48898424f8000000		MOVQ AX, 0xf8(SP)								
  mq_shm.go:158		0x46f5d09		4889bc2400010000		MOVQ DI, 0x100(SP)								
  mq_shm.go:158		0x46f5d11		488bac24b8000000		MOVQ 0xb8(SP), BP								
  mq_shm.go:158		0x46f5d19		4881c4c0000000			ADDQ $0xc0, SP									
  mq_shm.go:158		0x46f5d20		c3				RET										
  errors.go:59		0x46f5d21		488b442468			MOVQ 0x68(SP), AX								
  errors.go:59		0x46f5d26		e8b53297ff			CALL runtime.gcWriteBarrier(SB)							
  errors.go:59		0x46f5d2b		ebb6				JMP 0x46f5ce3									
  mq_shm.go:147		0x46f5d2d		48c78424e800000000000000	MOVQ $0x0, 0xe8(SP)								
  mq_shm.go:147		0x46f5d39		c78424f000000000000000		MOVL $0x0, 0xf0(SP)								
  mq_shm.go:147		0x46f5d44		0f57c0				XORPS X0, X0									
  mq_shm.go:147		0x46f5d47		0f118424f8000000		MOVUPS X0, 0xf8(SP)								
  mq_shm.go:147		0x46f5d4f		488bac24b8000000		MOVQ 0xb8(SP), BP								
  mq_shm.go:147		0x46f5d57		4881c4c0000000			ADDQ $0xc0, SP									
  mq_shm.go:147		0x46f5d5e		c3				RET										
  mq_shm.go:168		0x46f5d5f		e84c3c97ff			CALL runtime.panicSliceB(SB)							
  mq_shm.go:168		0x46f5d64		e8273c97ff			CALL runtime.panicSliceAcap(SB)							
  mq_shm.go:168		0x46f5d69		90				NOPL										
  mq_shm.go:143		0x46f5d6a		e8d11297ff			CALL runtime.morestack_noctxt(SB)						
  mq_shm.go:143		0x46f5d6f		e94cfbffff			JMP gw/internal/rtcmq.(*MqShm).Read(SB)		
  :-1			0x46f5d74		cc				INT $0x3									
  :-1			0x46f5d75		cc				INT $0x3									
  :-1			0x46f5d76		cc				INT $0x3									
  :-1			0x46f5d77		cc				INT $0x3									
  :-1			0x46f5d78		cc				INT $0x3									
  :-1			0x46f5d79		cc				INT $0x3									
  :-1			0x46f5d7a		cc				INT $0x3									
  :-1			0x46f5d7b		cc				INT $0x3									
  :-1			0x46f5d7c		cc				INT $0x3									
  :-1			0x46f5d7d		cc				INT $0x3									
  :-1			0x46f5d7e		cc				INT $0x3									
  :-1			0x46f5d7f		cc				INT $0x3

It seems function copy(buf, s.content[start+packetHeaderSize:uplen+start+packetHeaderSize]) (runtime.memmove in disassembly) is after atomic memory store s.header.SetWriteOffset(woff) (XCHGL AX, 0(CX) in disassembly) which will cause error.

@randall77
Copy link
Contributor

This is not a complete program that I can run. There's no main or Test* function.
There are also imports, gw/pkg/logs and gw/pkg/shm, that I can't resolve, so I can't even compile it.

The memmove looks like it runs before the XCHGLs to me. The memmove is at 0x46f5a45. Subsequent to that, at 0x46f5a60 there's a jump to 0x46f59bf, which leads to the XCHGLs at 0x46f59d4 and 0x46f5a01.

@golang golang locked and limited conversation to collaborators Dec 10, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants