Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os/user: LookupGroup does not identify all cases of UnknownGroupError #40334

Open
anonymouse64 opened this issue Jul 21, 2020 · 8 comments
Open
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@anonymouse64
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.14.5 linux/amd64

Does this issue reproduce with the latest release?

I only tested Go 1.14.5, I did not test Go 1.14.6 or tip.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/user/.cache/go-build"
GOENV="/home/user/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/user/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/snap/go/6090"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/snap/go/6090/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build704426793=/tmp/go-build -gno-record-gcc-switches"

What did you do?

https://play.golang.org/p/qEi8JbjG5-4 when built on Linux and run inside a Ubuntu 20.10 chroot inside a Ubuntu 16.04 LXD container inside an Ubuntu 16.04 VM (this setup comes from a cloud building environment) fails to identify the returned error as a user.UnknownGroupError.

I suspect there is something about the implementation that is specific to the chroot, as running the same program outside of the chroot (inside the LXD container) successfully returns. See:

# ./golang-group-lookup 
successful error handling
# chroot /tmp/tmp.evWDSKSvUt/ /golang-group-lookup
error, not matchable (type *errors.errorString), user: lookup groupname thisdoesnotexist: no such process
# uname -a
Linux first-marlin-ubuntu-bartender 4.4.0-185-generic #215-Ubuntu SMP Mon Jun 8 21:53:19 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

It's possible something about getgrpnam_r in the version of glibc in Ubuntu 20.10 has changed to return a different error code, but as per the man page of getgrpnam_r under "Notes", there are numerous possible error codes that could be used to identify user.UnknownGroupError:

The formulation given above under "RETURN VALUE" is from POSIX.1-2001. It does not call "not found" an error, hence does not specify what value errno might have in this situation. But that makes it impossible to recognize errors. One might argue that according to POSIX errno should be left unchanged if an entry is not found. Experiments on various UNIX-like systems shows that lots of different values occur in this situation: 0, ENOENT, EBADF, ESRCH, EWOULDBLOCK, EPERM and probably others.

As such, I would expect Go to do a "best effort" job at identifying errors returned as user.UnknownGroupError, including at least the documented ENOENT, ESRCH, EBADF, EPERM cases mentioned in the man page (in this case it is ESRCH that is being returned).

Finally, note that the same problem hypothetically affects user.Lookup and the associated user.UnknownUserError, but getpwnam_r is not acting up the same way getgrpnam_r is here.

What did you expect to see?

I expected the program when run inside the chroot to work the same as outside of the chroot.

What did you see instead?

Inside the chroot it fails to identify the returned error as user.UnknownGroupError.

@anonymouse64
Copy link
Author

Further debugging of the issue revealed the issue to actually be related to the fact that we are in the chroot, and hence we do not have things like systemd userdb to respond, however the chroot still had this in it's /etc/nsswitch.conf:

# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# If you have the `glibc-doc-reference' and `info' packages installed, try:
# `info libc "Name Service Switch"' for information about this file.

passwd:         files systemd
group:          files systemd
shadow:         files
gshadow:        files

hosts:          files dns
networks:       files

protocols:      db files
services:       db files
ethers:         db files
rpc:            db files

netgroup:       nis

notice group: files systemd.

So it seems to be that really we have an issue with NSS in this chroot that gets bubbled up to getgrpnam_r as ESRCH, which Go then doesn't recognize as user.UnknownGroupError.

@toothrot toothrot changed the title user.LookupGroup does not identify all cases of UnknownGroupError os/user: LookupGroup does not identify all cases of UnknownGroupError Jul 22, 2020
@toothrot toothrot added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 22, 2020
@toothrot toothrot added this to the Backlog milestone Jul 22, 2020
@toothrot
Copy link
Contributor

/cc @bradfitz @kevinburke

@kevinburke
Copy link
Contributor

Interesting... I'll try to take a look later this week. I don't think this is important enough to get through the code freeze so we'd be targeting 1.15.1 or 1.16 (another way of saying that the fix will not show up in a public release for a while).

@xnox
Copy link

xnox commented Jul 24, 2020

Checking through nss_systemd code, I do not see it returning anything unusual on the nss module level. It returns things like NSS_STATUS_UNAVAIL NSS_STATUS_NOTFOUND NSS_STATUS_TRYAGAIN. But I didn't check how that translates between libc & nss interface, and golang & libc.

@kevinburke
Copy link
Contributor

I'm having trouble reproducing. I am running the same program you are:

package main

import (
	"fmt"
	"os/user"
)

func main() {
	_, err := user.LookupGroup("thisdoesnotexist")
	fmt.Printf("%#v\n", err)
	switch err.(type) {
	case user.UnknownGroupError:
		fmt.Println("successful error handling")
	default:
		fmt.Printf("error, not matchable (type %T), %v\n", err, err)
	}
}

in tmp/test.go, then compiling the Go binary in a Linux container:

$ docker run --tty -v $(PWD)/tmp:/go --interactive meterup/ubuntu-golang:latest /usr/local/go/bin/go build -ldflags '-extldflags "-static"' -o /go/test/test-program ./

then sharing the compiled binary into a Docker container:

$ docker run --tty -v $(PWD)/tmp:/go --interactive meterup/ubuntu-golang:latest /bin/bash --login

running ldd to find remaining links in the binary:

$ ldd /go/test/test-program
	linux-vdso.so.1 (0x00007ffe801af000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007feed8a62000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007feed8671000)
	/lib64/ld-linux-x86-64.so.2 (0x00007feed8c81000)

copying these files into the chroot:

# mkdir /go/test/lib/x86_64-linux-gnu 
# cp -r --dereference /lib/x86_64-linux-gnu/* /go/test/lib/x86_64-linux-gnu

(and similar for the other files)

then, finally, running the program:

chroot /go/test /test-program
&errors.errorString{s:"user: lookup groupname thisdoesnotexist: no such file or directory"}
error, not matchable (type *errors.errorString), user: lookup groupname thisdoesnotexist: no such file or directory

Running with strace (you have to install it into that container):

[pid   456] openat(AT_FDCWD, "/etc/group", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
[pid   456] futex(0x55f6b8, FUTEX_WAKE_PRIVATE, 1) = 1
[pid   457] <... futex resumed> )       = 0
[pid   457] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2321, tv_nsec=911496659}) = 0
[pid   457] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2321, tv_nsec=911587159}) = 0
[pid   456] write(1, "&errors.errorString{s:\"user: loo"..., 92 <unfinished ...>
[pid   457] futex(0xc00003e948, FUTEX_WAKE_PRIVATE, 1&errors.errorString{s:"user: lookup groupname thisdoesnotexist: no such file or directory"}

So it looks like my program is trying to read /etc/group and failing (a failure I would expect in the chroot). My understanding is /etc/group parse is fallback logic if cgo is not enabled, though I am compiling on Linux with a linux target, so cgo should be enabled... I don't know strace well enough to determine whether getgrnam_r is the one that is trying and failing to read /etc/group, ie. cgo is enabled but it's still failing to read the file, though I am concerned that I am getting different output than you got.

@kevinburke
Copy link
Contributor

Odd... I copied the contents of os/user to a third party package so I could add print statements to it, and when I do that, it compiles and runs successfully.

$ docker run --tty -v $(PWD)/tmp:/go --interactive meterup/ubuntu-golang:latest /usr/local/go/bin/go build -ldflags '-extldflags "-static"' -o /go/test/test-program ./
# _/go
/tmp/go-link-139166518/000002.o: In function `mygetgrouplist':
/go/user2/getgrouplist_unix.go:16: warning: Using 'getgrouplist' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/tmp/go-link-139166518/000001.o: In function `mygetgrgid_r':
/go/user2/cgo_lookup_unix.go:38: warning: Using 'getgrgid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/tmp/go-link-139166518/000001.o: In function `mygetgrnam_r':
/go/user2/cgo_lookup_unix.go:43: warning: Using 'getgrnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/tmp/go-link-139166518/000001.o: In function `mygetpwnam_r':
/go/user2/cgo_lookup_unix.go:33: warning: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/tmp/go-link-139166518/000001.o: In function `mygetpwuid_r':
/go/user2/cgo_lookup_unix.go:28: warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
# chroot /go/test /test-program
calling lookup Group using mygetgrnam_r
lookup err: <nil>
"thisdoesnotexist"
successful error handling

@anonymouse64
Copy link
Author

I can share a reproducer environment when I get a chance, sorry been busy with other things... should be soon though

@mvo5
Copy link
Contributor

mvo5 commented Jan 19, 2022

Fwiw, I can easily reproduce this by installing the sssd package on Ubuntu which will add the sss to the nsswitch.conf. The fix for this seems fairly straightforward
0001-user-handle-ENOENT-from-getpw-uid-nam-_r.patch.gz

(tested the issue with sssd on 20.04 and 21.10 and the fix as well).

Alternatively the fmt.Errorf() return could use %w instead of the current %v to make it possible for the upper layer to reliably identify this.

mvo5 added a commit to mvo5/snappy that referenced this issue Jan 19, 2022
When the `sssd` package is installed on a Ubuntu system the
`TestUserMaybeSudoUser` test will fail because then `user.Lookup`
returns a generic error
```
user: lookup username guy: no such file or directory
```
because in the go layer the getpw{uid,nam}_r() returned ENOENT.

This is already reported upstream as
golang/go#40334

This commit works around this upstream issue by checking for
ENOENT on the error from user.Lookup(). It's not nice but the
best I could come up to to handle this case. On my 21.10 system
sssd is installed as part of `ubuntu-desktop-minimal`.
jakule added a commit to gravitational/teleport that referenced this issue Jun 1, 2023
As described here golang/go#40334 incorrect system configuration can lead user.LookupGroup to return ENOENT. We've already added a workaround #18981, but this one place seems to be forgotten. I also added ESRCH to ignored errors as newer glibc returns it.

The error was exposed after updating our docker image to Ubuntu 20.04 #26905 which includes newer Glibc.
github-merge-queue bot pushed a commit to gravitational/teleport that referenced this issue Jun 1, 2023
As described here golang/go#40334 incorrect system configuration can lead user.LookupGroup to return ENOENT. We've already added a workaround #18981, but this one place seems to be forgotten. I also added ESRCH to ignored errors as newer glibc returns it.

The error was exposed after updating our docker image to Ubuntu 20.04 #26905 which includes newer Glibc.
github-actions bot pushed a commit to gravitational/teleport that referenced this issue Jun 1, 2023
As described here golang/go#40334 incorrect system configuration can lead user.LookupGroup to return ENOENT. We've already added a workaround #18981, but this one place seems to be forgotten. I also added ESRCH to ignored errors as newer glibc returns it.

The error was exposed after updating our docker image to Ubuntu 20.04 #26905 which includes newer Glibc.
github-actions bot pushed a commit to gravitational/teleport that referenced this issue Jun 1, 2023
As described here golang/go#40334 incorrect system configuration can lead user.LookupGroup to return ENOENT. We've already added a workaround #18981, but this one place seems to be forgotten. I also added ESRCH to ignored errors as newer glibc returns it.

The error was exposed after updating our docker image to Ubuntu 20.04 #26905 which includes newer Glibc.
github-actions bot pushed a commit to gravitational/teleport that referenced this issue Jun 1, 2023
As described here golang/go#40334 incorrect system configuration can lead user.LookupGroup to return ENOENT. We've already added a workaround #18981, but this one place seems to be forgotten. I also added ESRCH to ignored errors as newer glibc returns it.

The error was exposed after updating our docker image to Ubuntu 20.04 #26905 which includes newer Glibc.
github-merge-queue bot pushed a commit to gravitational/teleport that referenced this issue Jun 2, 2023
As described here golang/go#40334 incorrect system configuration can lead user.LookupGroup to return ENOENT. We've already added a workaround #18981, but this one place seems to be forgotten. I also added ESRCH to ignored errors as newer glibc returns it.

The error was exposed after updating our docker image to Ubuntu 20.04 #26905 which includes newer Glibc.
github-merge-queue bot pushed a commit to gravitational/teleport that referenced this issue Jun 2, 2023
As described here golang/go#40334 incorrect system configuration can lead user.LookupGroup to return ENOENT. We've already added a workaround #18981, but this one place seems to be forgotten. I also added ESRCH to ignored errors as newer glibc returns it.

The error was exposed after updating our docker image to Ubuntu 20.04 #26905 which includes newer Glibc.
github-merge-queue bot pushed a commit to gravitational/teleport that referenced this issue Jun 2, 2023
As described here golang/go#40334 incorrect system configuration can lead user.LookupGroup to return ENOENT. We've already added a workaround #18981, but this one place seems to be forgotten. I also added ESRCH to ignored errors as newer glibc returns it.

The error was exposed after updating our docker image to Ubuntu 20.04 #26905 which includes newer Glibc.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

5 participants