Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reflect: documentation of StructField.Offset is not clear about type embedding #61495

Closed
bronze1man opened this issue Jul 21, 2023 · 10 comments
Closed
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. Documentation Issues describing a change to documentation. FrozenDueToAge help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@bronze1man
Copy link
Contributor

bronze1man commented Jul 21, 2023

When I read the document I think StructField.Offset means the offset of the field of caller struct, not the offset of embeded struct.But it is the offset of embeded struct.
I think the document should be updated to clear it.

What version of Go are you using (go version)?

$ go version
go version go1.19.2 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/a/Library/Caches/go-build"
GOENV="/Users/a/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/a/go/pkg/mod"
GOOS="darwin"
GOPATH="/Users/a/go"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.19.2"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/mw/884f6hx53r56gd7zxsm1h8bm0000gp/T/go-build2286547601=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

https://go.dev/play/p/xae9pAtDcY7

What did you expect to see?

8 8

What did you see instead?

8 0

  • ps current solution of the origin problem (get correct offset of a field of struct by name with reflect):
func mustGetOffsetFormStruct(rs reflect.Type,fieldName string) uintptr{
	f,ok:=rs.FieldByName(fieldName)
	if ok==false{
		panic(`fieldName not in struct`)
	}
	if len(f.Index)==1{
		return f.Offset
	}
	offset:=uintptr(0)
	for _,index:=range f.Index{
		rf:=rs.Field(index)
		offset+=rf.Offset
		rs = rf.Type
	}
	return offset
}
@gopherbot gopherbot added compiler/runtime Issues related to the Go compiler and/or runtime. Documentation Issues describing a change to documentation. labels Jul 21, 2023
@zigo101
Copy link

zigo101 commented Jul 21, 2023

unsafe instead of reflect? Or both?

@bronze1man
Copy link
Contributor Author

bronze1man commented Jul 21, 2023

unsafe instead of reflect? Or both?

Both. I just try to use unsafe, reflect and generics together to write a library, it is faster than my manual code to solve the same problem in my benchmark. amazing!!!
I know it just a benchmark game. But it is still amazing.

func SliceToMapFieldAsKey[Key_t comparable,Item_t any](s []Item_t,fieldName string) map[Key_t]Item_t{
	var key Key_t
	rkeyt:=reflect.TypeOf(key)
	m:=make(map[Key_t]Item_t,len(s))
	t1:=reflect.TypeOf(s)
	t1e:=t1.Elem()
	if t1e.Kind()!=reflect.Struct{
		panic(`only support slice of struct now.`)
	}
	f,ok:=t1e.FieldByName(fieldName)
	if ok==false{
		panic(`SliceToMap1 fieldName not found`)
	}
	if f.Type.ConvertibleTo(rkeyt)==false{
		panic(`SliceToMap1 fieldName type not match`)
	}
	offset:=mustGetOffsetFormStruct(t1e,fieldName)
	for i:=range s{
		key=*(*Key_t)(unsafe.Pointer(uintptr(unsafe.Pointer(&s[i]))+offset))
		m[key] = s[i]
	}
	return m
}
func mustGetOffsetFormStruct(rs reflect.Type,fieldName string) uintptr{
	f,ok:=rs.FieldByName(fieldName)
	if ok==false{
		panic(`fieldName not in struct`)
	}
	if len(f.Index)==1{
		return f.Offset
	}
	offset:=uintptr(0)
	for _,index:=range f.Index{
		rf:=rs.Field(index)
		offset+=rf.Offset
		rs = rf.Type
	}
	return offset
}

@heschi heschi added this to the Backlog milestone Jul 21, 2023
@heschi heschi added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 21, 2023
@ianlancetaylor
Copy link
Member

Do you have any suggestions for how to make the comment clearer?

It seems to me that the confusion is not so much about Offset as about the fact that FieldByName can return a field in an embedded struct.

@bronze1man
Copy link
Contributor Author

bronze1man commented Jul 22, 2023

Do you have any suggestions for how to make the comment clearer?

offset within defined struct, in bytes. If the field is embed in struct, it is the offset within the embedded struct. 

It may be better to add another func like reflect.GetOffsetFormStructByName(rs reflect.Type,fieldName string) (uintptr,bool) then add common with To get the offset of a embedded field within struct use GetOffsetFormStructByName.

It seems to me that the confusion is not so much about Offset as about the fact that FieldByName can return a field in an embedded struct.

I do not think so. I first use the StructField.Index, it tell me the index of embedded level. So I do not think StructField is return a field in an embedded struct

@mknyszek mknyszek changed the title reflect: document of StructField.Offset is not clear about embed reflect: documentation of StructField.Offset is not clear about type embedding Jul 26, 2023
@eduardbme
Copy link
Contributor

eduardbme commented Jul 27, 2023

Wouldn't it be better to have a similar behavior with the StructField.Index method and return an array of offsets ?

Thus:

  • the last element of an array is always an offset within a struct
  • rest (prior to the last) elements are the offsets of the embedded structs

In this way, if an array contains one element, it's just an offset within the struct, if more - the struct is embedded. And you can leverage or just ignore that info and go with the last element, which is always an offset within the struct.

@bronze1man
Copy link
Contributor Author

bronze1man commented Jul 27, 2023

Wouldn't it be better to have a similar behavior with the StructField.Index method and return an array of offsets ?

I do not think this design is better, it will confuse/surprise me again.
I think add a field or method give me the answer of the offset within the caller struct is best.
In my use case (try to use unsafe to make reflect code run faster like github.com/viant/xunsafe ), an offset within the caller struct is useful and the offsets of the embedded structs is useless.

@ianlancetaylor
Copy link
Member

I don't see any reason to add new fields or functions or methods. All the information anybody might want is available and correct. As far as I can tell, this is a documentation issue that should be addressed by writing better documentation.

qiulaidongfeng added a commit to qiulaidongfeng/go that referenced this issue Jul 29, 2023
Mention that StructField.Offset can represent the offset of the embedded structure.

Fixes golang#61495

Change-Id: I9abaf3613c797006b803dcb1dbee16f25ffb7516
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/514035 mentions this issue: reflect: update StructField.Offset document

@bronze1man
Copy link
Contributor Author

Thanks a lot .
I think we should also mention this information on StructField.Offset not only on Type.FieldByName.
Do we need to add this information on Type.Field , Type.FieldByIndex, Type.FieldByNameFunc?

@ianlancetaylor
Copy link
Member

I don't think we need a common on StructField.Offset. I think the existing comment there is exactly right. The only confusing case is when you look up a field and don't realize that it is a promoted field, and that is specific to the lookup.

We don't need any comment on Type.Field or Type.FieldByIndex. I think those cases are clear. But a common on Type.FieldByNameFunc would be appropriate, as that can also return a promoted field.

@github-project-automation github-project-automation bot moved this from Todo to Done in Go Compiler / Runtime Aug 2, 2023
@golang golang locked and limited conversation to collaborators Aug 1, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. Documentation Issues describing a change to documentation. FrozenDueToAge help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants