embed: go1.16beta1 http.FileServer runtime error: invalid memory address or nil pointer dereference #43682

vomnes · 2021-01-13T23:35:42Z

What version of Go are you using (`go version`)?

$ go version
go version go1.16beta1 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go1.16beta1 env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOOS="darwin"
GOPROXY="https://proxy.golang.org,direct"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOVCS=""
GOVERSION="go1.16beta1"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/fn/61cn31hn7r1bbd_hbpmfjqdm0000gp/T/go-build1615016027=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I'm using the "embed" package available with go1.16beta1.

I'm trying to implement a web server using http.ListenAndServe to render my Angular build Website.

embed was working fine until I replace the http.FS by http.Dir.
Now, http.Dir inside http.FileServer is working fine but http.FSand so embed are not working anymore.
I have a crash http: panic serving [::1]:50241: runtime error: invalid memory address or nil pointer dereference either calling localhost:8080 on my web browser or just need to wait few seconds to receive the crash from the server.

Reproduce the behaviour

Need to have a folder 'a' containing for example a file such as 'index.html'.

Step 1

package main

import (
	"embed"
	"net/http"
)

func main() {
        // go:embed a
	var server embed.FS
        http.ListenAndServe(":8080", http.FileServer(http.FS(server))) // <-- http.FS
}

go1.16beta1 run main.go

Step 2

package main

import "net/http"

func main() {
        http.ListenAndServe(":8080", http.FileServer(http.Dir("a"))) // <-- http.Dir
}

go1.16beta1 run main.go

Step 3

package main

import (
	"embed"
	"net/http"
)

func main() {
        // go:embed a
	var server embed.FS
        http.ListenAndServe(":8080", http.FileServer(http.FS(server))) // <-- http.FS
}

go1.16beta1 run main.go

Step 3

The server should not working anymore and return a crash.
http: panic serving [::1]:50241: runtime error: invalid memory address or nil pointer dereference
See below for more details.

What did you expect to see?

The file rendered in the web browser.

What did you see instead?

2021/01/14 00:08:00 http: panic serving [::1]:50138: runtime error: invalid memory address or nil pointer dereference
goroutine 5 [running]:
net/http.(*conn).serve.func1(0xc00007c000)
        /Users/username/sdk/go1.16beta1/src/net/http/server.go:1824 +0x153
panic(0x125dc80, 0x1446a90)
        /Users/username/sdk/go1.16beta1/src/runtime/panic.go:971 +0x47a
embed.FS.readDir(0x0, 0x12a16a5, 0x1, 0x14484a0, 0x160b280, 0x0)
        /Users/username/sdk/go1.16beta1/src/embed/embed.go:266 +0x26
embed.FS.Open(0x0, 0x12a16a5, 0x1, 0x8, 0x1484768, 0x64, 0x203000)
        /Users/username/sdk/go1.16beta1/src/embed/embed.go:285 +0xce
net/http.ioFS.Open(0x12f58e0, 0x0, 0xc0000160b4, 0x1, 0x24, 0x160b280, 0xc0000579c0, 0x100eedb)
        /Users/username/sdk/go1.16beta1/src/net/http/fs.go:760 +0x68
net/http.serveFile(0x12f8cd0, 0xc00018e000, 0xc000180000, 0x12f5ba0, 0xc000098cd0, 0xc0000160b4, 0x1, 0xc00007a001)
        /Users/username/sdk/go1.16beta1/src/net/http/fs.go:597 +0x84
net/http.(*fileHandler).ServeHTTP(0xc000098ce0, 0x12f8cd0, 0xc00018e000, 0xc000180000)
        /Users/username/sdk/go1.16beta1/src/net/http/fs.go:848 +0x9c
net/http.serverHandler.ServeHTTP(0xc0000fc000, 0x12f8cd0, 0xc00018e000, 0xc000180000)
        /Users/username/sdk/go1.16beta1/src/net/http/server.go:2887 +0xa3
net/http.(*conn).serve(0xc00007c000, 0x12f9100, 0xc00002a080)
        /Users/username/sdk/go1.16beta1/src/net/http/server.go:1952 +0x8ed
created by net/http.(*Server).Serve
        /Users/username/sdk/go1.16beta1/src/net/http/server.go:3013 +0x39b

The text was updated successfully, but these errors were encountered:

ianlancetaylor · 2021-01-14T00:14:53Z

Marking as a release blocker for 1.16, as it seems that it might be a bug in the new go:embed support.

bcmills · 2021-01-14T01:57:09Z

CC @jayconrod @rsc

andig · 2021-01-14T07:23:04Z

3 looks identicial to 1. are you saying one works and one doesn‘t?

vomnes · 2021-01-14T09:18:26Z

3 looks identicial to 1. are you saying one works and one doesn‘t?

Yes, they are the same; It's just the steps to reproduce the bug.
Step 1: Work, Step 2: Work, Step 3: Doesn't work anymore

jayconrod · 2021-01-14T17:27:46Z

It looks like the embed.FS isn't initialized because there's a space between // and go:embed. They have to be together. This works:

package main

import (
	"embed"
	"net/http"
)

//go:embed a
var server embed.FS

func main() {
	http.ListenAndServe(":8080", http.FileServer(http.FS(server))) // <-- http.FS
}

(Note also that in 1.16, the //go:embed directive can only apply to global variables. Allowing local variables 1.16beta1 led to some unexpected behaviors.)

This seems like it might come up a lot. Maybe http could panic with a clearer message?

bcmills · 2021-01-14T17:35:19Z

Should it be a compile-time error to instantiate a value of type embed.FS without a corresponding go:embed directive?

If not, should the embed.FS methods explicitly check that the FS has been initialized, or should the zero embed.FS be a valid empty filesystem?

bcmills · 2021-01-14T17:36:55Z

Ah, I see that's addressed in the documentation:

When declared without a //go:embed directive, an FS is an empty file system.

So the fact that Open is panicking here seems to be out-of-spec.

bcmills · 2021-01-14T17:37:25Z

That said, I think at the very least go vet should emit a warning for package-level variables of type embed.FS that lack go:embed directives.

jayconrod · 2021-01-14T17:40:37Z

Should it be a compile-time error to instantiate a value of type embed.FS without a corresponding go:embed directive?

Not necessarily, especially for local variables and fields. It's essentially a reference type, so I could see copying it into a variable that's uninitialized.

So the fact that Open is panicking here seems to be out-of-spec.

That seems like the thing to fix. All the other methods call that.

That said, I think at the very least go vet should emit a warning for package-level variables of type embed.FS that lack go:embed directives.

+1 to that. Should that be a new check or part of something existing? I think other pragmas have the same comment restriction, and we might want to check them all.

bcmills · 2021-01-14T17:50:48Z

Filed the vet check as proposal #43698.

gopherbot · 2021-01-14T18:20:40Z

Change https://golang.org/cl/283852 mentions this issue: embed: treat uninitialized FS as empty

vomnes · 2021-01-14T21:11:19Z

Indeed, it's the space before go:embed that cause the problem without there are no crash anymore.

As described in the FS documentation. This prevents http.FS and other clients from panicking when the go:embed directive is missing. For #43682 Related #43698 Change-Id: Iecf26d229a099e55d24670c3119cd6c6d17ecc6e Reviewed-on: https://go-review.googlesource.com/c/go/+/283852 Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>

ianlancetaylor · 2021-01-21T00:49:03Z

This appears to have been fixed, so closing.

ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 14, 2021

ianlancetaylor added this to the Go1.16 milestone Jan 14, 2021

ianlancetaylor added the release-blocker label Jan 14, 2021

bcmills mentioned this issue Jan 14, 2021

cmd/vet: warn about "// go:" comments (with space) #43698

Open

ianlancetaylor closed this as completed Jan 21, 2021

bcmills mentioned this issue Feb 2, 2021

Panic when trying to serve http with embedded assets #43761

Closed

golang locked and limited conversation to collaborators Jan 21, 2022

gopherbot added the FrozenDueToAge label Jan 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

embed: go1.16beta1 http.FileServer runtime error: invalid memory address or nil pointer dereference #43682

embed: go1.16beta1 http.FileServer runtime error: invalid memory address or nil pointer dereference #43682

vomnes commented Jan 13, 2021 •

edited

ianlancetaylor commented Jan 14, 2021

bcmills commented Jan 14, 2021

andig commented Jan 14, 2021

vomnes commented Jan 14, 2021

jayconrod commented Jan 14, 2021

bcmills commented Jan 14, 2021

bcmills commented Jan 14, 2021

bcmills commented Jan 14, 2021

jayconrod commented Jan 14, 2021

bcmills commented Jan 14, 2021

gopherbot commented Jan 14, 2021

vomnes commented Jan 14, 2021

ianlancetaylor commented Jan 21, 2021

embed: go1.16beta1 http.FileServer runtime error: invalid memory address or nil pointer dereference #43682

embed: go1.16beta1 http.FileServer runtime error: invalid memory address or nil pointer dereference #43682

Comments

vomnes commented Jan 13, 2021 • edited

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

Reproduce the behaviour

Step 1

Step 2

Step 3

Step 3

What did you expect to see?

What did you see instead?

ianlancetaylor commented Jan 14, 2021

bcmills commented Jan 14, 2021

andig commented Jan 14, 2021

vomnes commented Jan 14, 2021

jayconrod commented Jan 14, 2021

bcmills commented Jan 14, 2021

bcmills commented Jan 14, 2021

bcmills commented Jan 14, 2021

jayconrod commented Jan 14, 2021

bcmills commented Jan 14, 2021

gopherbot commented Jan 14, 2021

vomnes commented Jan 14, 2021

ianlancetaylor commented Jan 21, 2021

vomnes commented Jan 13, 2021 •

edited

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?