Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

embed: FS root directory is always the package directory, while os.DirFS may have another root #43431

Closed
alicebob opened this issue Dec 30, 2020 · 8 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@alicebob
Copy link
Contributor

alicebob commented Dec 30, 2020

What version of Go are you using (go version)?

go version devel +2ff33f5e44 Thu Dec 17 16:03:19 2020 +0000 linux/amd64

(that's 1.16beta1)

Does this issue reproduce with the latest release?

yes

What did you do?

I want to serve static documents in a webserver using the new embed package, but with an option to use local files.

https://play.golang.org/p/hyBgSUBUhf3

There is a subdir ./docroot/ with an index.html file:

myuser@mymachine:/tmp/root$ ls -R
.:
docroot/  go.mod  main.go  root

./docroot:
index.html

When I use the os.DirFS() version (./root -docroot ./docroot) I get the index.html nicely at http://localhost:6602/index.html. If I use the embed version (./root) it ends up at http://localhost:6602/docroot/index.html

What did you expect to see?

I expect either none or both to have the ./docroot/ subdir in there. This way I can't use http.StripPrefix(). But maybe I missed something else.

What did you see instead?

The embed.FS and os.DirFS seem to behave inconsistently.

If embed.FS implemented fs.SubFS I think I could make this work by using docroot.Sub("./docroot/")

@changkun
Copy link
Member

changkun commented Dec 30, 2020

You can do something like this:

//go:embed docroot/*
var root embed.FS

// myFS implements fs.FS
type myFS struct {
	content embed.FS
}

func (c myFS) Open(name string) (fs.File, error) {
	return c.content.Open(path.Join("docroot", name))
}

Then:

r.Handle("/", http.FileServer(http.FS(myFS{root})))

@dmitshur dmitshur changed the title included embed.FS subdir embed: FS root directory is always the package directory, while os.DirFS may have another root Dec 30, 2020
@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Dec 30, 2020
@dmitshur dmitshur added this to the Backlog milestone Dec 30, 2020
@dmitshur
Copy link
Contributor

It seems to me that embed.FS and os.DirFS are working as intended, you will need to arrange your code to either handle different roots, or have the same root by running your program with ./root -docroot=. instead of ./root -docroot=./docroot.

Since it's a new API, I'll let @rsc consider if anything more should be done.

@alicebob
Copy link
Contributor Author

Thanks for the quick responses.

Is there anything against embed.FS implementing fs.SubFS? That would make it easier to deal with this case.

@alicebob
Copy link
Contributor Author

@changkun solution would work great. I ended up adding a single file inside the docroot:

$ cat docroot/fs.go 
package docroot

import "embed"

//go:embed *
var FS embed.FS

and importing that. It's not pretty, but works.

(as far as I'm concerned this issue can be closed)

@changkun
Copy link
Member

changkun commented Dec 31, 2020

@alicebob Indeed. I noticed the behavior you reported in this thread while I was trying to convert my static website here.

I didn't report this simply because I don't know that is the actually expected behavior from these new set of APIs. The design draft is just a draft, as far as I partially skimmed with the ongoing discussions, the current implementation is already apart from the draft design (lack of documentation).

For more instance, I'd like to directly embed everything in a repository (and of course exclude the .git folder):

$ l
total 16
drwxr-xr-x   5 changkun  staff   160B Dec 31 13:56 .
drwxr-xr-x  29 changkun  staff   928B Dec 31 13:49 ..
drwxr-xr-x   9 changkun  staff   288B Dec 31 13:50 .git
-rw-r--r--   1 changkun  staff    18B Dec 31 13:56 go.mod
-rw-r--r--   1 changkun  staff   102B Dec 31 13:54 main.go

$ cat main.go
package main

import "embed"

//go:embed *
var static embed.FS

func main() {
        println(static)
}

With the (undocumented) 37588ff, the .git should be ignored and have a successful build.

But unfortunately, build such a simple program can result in the following compile error:

$ go build
pattern *: cannot embed directory .git: invalid name .git

@earthboundkid
Copy link
Contributor

Am I misunderstanding or does this use of fs.Sub do what you want? https://github.com/carlmjohnson/exembed/blob/main/web/main.go

@changkun
Copy link
Member

Am I misunderstanding or does this use of fs.Sub do what you want? https://github.com/carlmjohnson/exembed/blob/main/web/main.go

Oh, how cloud I miss the fs.Sub, thanks for the hint. It would be great to have an example like this in a *_example.go and demonstrate in the document.

@alicebob
Copy link
Contributor Author

alicebob commented Jan 1, 2021

@carlmjohnson awesome, yes that solves everything.

I misread the output of go doc: I saw SubFS interface, which embed.FS doesn't implement, but I missed the pure Sub() function.

@alicebob alicebob closed this as completed Jan 1, 2021
@golang golang locked and limited conversation to collaborators Jan 1, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

5 participants