Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tour: offline version of 'A Tour of Go' broken with modules #44243

Closed
fzipp opened this issue Feb 12, 2021 · 12 comments
Closed

x/tour: offline version of 'A Tour of Go' broken with modules #44243

fzipp opened this issue Feb 12, 2021 · 12 comments
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@fzipp
Copy link
Contributor

fzipp commented Feb 12, 2021

What version of Go are you using (go version)?

$ go version
go version go1.16rc1 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/fzipp/Library/Caches/go-build"
GOENV="/Users/fzipp/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/fzipp/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/fzipp/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16rc1"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/lc/1h4lth_d68zfy0rvwrv37tkm0000gn/T/go-build3688970254=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I tried to install the offline tour of Go via go get as described here and here and run it.

$ go get golang.org/x/tour
go: downloading golang.org/x/tour v0.0.0-20201207214521-004403599411
go: downloading golang.org/x/tools v0.0.0-20190312164927-7b79afddac43
go: downloading golang.org/x/net v0.0.0-20190311183353-d8887717615a
$ tour

What did you expect to see?

Opening a web browser displaying my local version of the tour.

What did you see instead?

$ tour
2021/02/12 22:53:03 Couldn't find tour files: could not find go-tour content; check $GOROOT and $GOPATH

Of course the files can't be found in $GOPATH, because go get downloaded it into the module cache.

@gopherbot gopherbot added this to the Unreleased milestone Feb 12, 2021
@dmitshur
Copy link
Contributor

Thanks for reporting. I'll take a look.

@dmitshur dmitshur self-assigned this Feb 12, 2021
@dmitshur dmitshur added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. modules labels Feb 12, 2021
@gopherbot
Copy link

Change https://golang.org/cl/291849 mentions this issue: tour: use embed and io/fs

@dmitshur
Copy link
Contributor

It's possible to fix this either by adjusting the code to look for tour files in module cache, or by embedding them in the binary. I chose with the latter and sent CL 291849.

@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Feb 13, 2021
@fzipp
Copy link
Contributor Author

fzipp commented Feb 13, 2021

Thanks, the CL version starts locally.
There's still a problem with tour snippets that import "golang.org/x/tour/..." packages:

http://127.0.0.1:3999/moretypes/18
http://127.0.0.1:3999/methods/25
import "golang.org/x/tour/pic"
compile3.go:3:8: no required module provides package golang.org/x/tour/pic: working directory is not part of a module

http://127.0.0.1:3999/moretypes/23
import "golang.org/x/tour/wc"

http://127.0.0.1:3999/methods/22
import "golang.org/x/tour/reader"

http://127.0.0.1:3999/concurrency/8
import "golang.org/x/tour/tree"

@gopherbot
Copy link

Change https://golang.org/cl/291889 mentions this issue: tour: build snippets in module mode

@dmitshur
Copy link
Contributor

Right, that will happen if the tour source code isn't available in GOPATH/src/golang.org/x/tour. To remove that requirement, we need to update tour to start building snippets in module mode (and supply a go.mod file that provides a version of the golang.org/x/tour module).

I've sent CL 291889 for that.

@dmitshur dmitshur added this to In Progress in Go Release Team Feb 23, 2021
@gopherbot
Copy link

Change https://golang.org/cl/298529 mentions this issue: tour: look for content in module cache, not GOROOT/misc/tour

gopherbot pushed a commit to golang/tour that referenced this issue Mar 4, 2021
We want to start building snippets in module mode rather than legacy
legacy GOPATH mode, to make tour work well when there's no tour code
in GOPATH/src and to be more future-proof. Building in module mode
means we need to specify a go.mod file to use for each snippet.

The tour web UI design predates the module mode and its go.mod files.
Txtar-based multi-file support was added to the playground (that the
tour uses for executing snippets) in golang.org/issue/32040, but the
web UI hasn't been updated to have first-class separate <textarea>
elements or tabs for editing multiple files.

For now, we can get by with providing a default go.mod file for all
tour snippets, since they only need some packages in the tour module.

Update to a newer version of the golang.org/x/tools/playground/socket
package which includes multi-file snippet support added in CL 204237.

For golang/go#44243.

Change-Id: Ib05725c52fbe43658b2f22ac728b1e901e635824
Reviewed-on: https://go-review.googlesource.com/c/tour/+/291889
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
@dmitshur
Copy link
Contributor

dmitshur commented Mar 4, 2021

It's possible to fix this either by adjusting the code to look for tour files in module cache, or by embedding them in the binary. I chose with the latter and sent CL 291849.

We can't quite start relying on Go 1.16+ APIs while Go 1.15 is still supported (not without causing much code duplication), so I sent CL 298529 to get the module cache fix in the interim.

Go Release Team automation moved this from In Progress to Done Mar 5, 2021
gopherbot pushed a commit to golang/website that referenced this issue May 26, 2021
We want to start building snippets in module mode rather than legacy
legacy GOPATH mode, to make tour work well when there's no tour code
in GOPATH/src and to be more future-proof. Building in module mode
means we need to specify a go.mod file to use for each snippet.

The tour web UI design predates the module mode and its go.mod files.
Txtar-based multi-file support was added to the playground (that the
tour uses for executing snippets) in golang.org/issue/32040, but the
web UI hasn't been updated to have first-class separate <textarea>
elements or tabs for editing multiple files.

For now, we can get by with providing a default go.mod file for all
tour snippets, since they only need some packages in the tour module.

Update to a newer version of the golang.org/x/tools/playground/socket
package which includes multi-file snippet support added in CL 204237.

For golang/go#44243.

Change-Id: Ib05725c52fbe43658b2f22ac728b1e901e635824
Reviewed-on: https://go-review.googlesource.com/c/tour/+/291889
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
X-Tour-Commit: b72029adf3393d3635828fedcddd5ae7053d3250
gopherbot pushed a commit to golang/website that referenced this issue May 26, 2021
Tour content hasn't been available in GOROOT since Go 1.12¹, so stop
looking for it there.

A more useful fallback place to find tour content is the module cache.
When the golang.org/x/tour is in the build list, it will be found via
build.Import. It will not be found, however, when the tour binary is
run outside of a module. This situation is more common as of Go 1.16,
since the default GO111MODULE value changed to on². If the version of
the tour binary is known, access the content for that version from the
module cache.

In the future, this complex and imperfect machinery will be replaced
with simply embedding the needed files in the binary. We can't do that
quite yet (not without duplicating a lot of code) because Go 1.15 is
supported, and because Go 1.14 is used for deployment.

It's not neccessary to provide a custom socket.Environ (which served
to provide a custom GOPATH value pointing inside GOROOT; that path
is being removed here). Delete it along with some other unused code.

¹ https://golang.org/doc/go1.12#tour
² https://golang.org/doc/go1.16#modules

Fixes golang/go#44243.

Change-Id: Ia239700170b8614d21cafe9617b51dddf3e337d5
Reviewed-on: https://go-review.googlesource.com/c/tour/+/298529
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
X-Tour-Commit: b3263fcf7749f3600c8928d5f7d4e8a66a34b74f
@gopherbot
Copy link

Change https://golang.org/cl/334969 mentions this issue: tour: look for content in x/website module

@jfesler
Copy link

jfesler commented Jul 16, 2021

This works fine for running the binaries on the same host as you build the binaries.

https://github.com/google/go-licenses does the same trick. Said binaries were not useful anywhere else. I ended up having to duplicate the license data into a place I can embed, to produce a portable binary.

@dmitshur
Copy link
Contributor

dmitshur commented Jul 16, 2021

@jfesler Thanks for that information. Indeed, looking for content in the module cache can only work on a machine where Go is installed.

Go 1.17 will come out pretty soon, at which point Go 1.15 will become unsupported and we can start relying on //go:embed for embedding assets into the binary itself (CL 342711), which won't have that limitation.

gopherbot pushed a commit to golang/website that referenced this issue Jul 17, 2021
As of CL 323897, the tour command and its static content are
contained in a module with a slightly different module path.
Update findRoot accordingly so it looks in the right place.

Updates golang/go#44243.

Change-Id: I5c979789440464ca3f9f99799bcc8e73db0769a3
Reviewed-on: https://go-review.googlesource.com/c/website/+/334969
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alberto Donizetti <alb.donizetti@gmail.com>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/342711 mentions this issue: tour: use embed and io/fs

gopherbot pushed a commit to golang/website that referenced this issue Aug 17, 2021
Embed content, static and template directories directly in the tour
binary. This way it will work even if the package source is deleted.
The content in those directories is not large, the binary size goes
up from 14.2 MB to 15.2 MB in my testing (with Go 1.17 darwin/arm64).

Start using the Go 1.16 runtime for App Engine deployment.

Updates golang/go#44243.

Change-Id: I35fb32961cdc1edec1f8f8c0fc0193b07cef9acd
Reviewed-on: https://go-review.googlesource.com/c/website/+/342711
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
passionSeven added a commit to passionSeven/website that referenced this issue Oct 18, 2022
We want to start building snippets in module mode rather than legacy
legacy GOPATH mode, to make tour work well when there's no tour code
in GOPATH/src and to be more future-proof. Building in module mode
means we need to specify a go.mod file to use for each snippet.

The tour web UI design predates the module mode and its go.mod files.
Txtar-based multi-file support was added to the playground (that the
tour uses for executing snippets) in golang.org/issue/32040, but the
web UI hasn't been updated to have first-class separate <textarea>
elements or tabs for editing multiple files.

For now, we can get by with providing a default go.mod file for all
tour snippets, since they only need some packages in the tour module.

Update to a newer version of the golang.org/x/tools/playground/socket
package which includes multi-file snippet support added in CL 204237.

For golang/go#44243.

Change-Id: Ib05725c52fbe43658b2f22ac728b1e901e635824
Reviewed-on: https://go-review.googlesource.com/c/tour/+/291889
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
X-Tour-Commit: b72029adf3393d3635828fedcddd5ae7053d3250
passionSeven added a commit to passionSeven/website that referenced this issue Oct 18, 2022
Tour content hasn't been available in GOROOT since Go 1.12¹, so stop
looking for it there.

A more useful fallback place to find tour content is the module cache.
When the golang.org/x/tour is in the build list, it will be found via
build.Import. It will not be found, however, when the tour binary is
run outside of a module. This situation is more common as of Go 1.16,
since the default GO111MODULE value changed to on². If the version of
the tour binary is known, access the content for that version from the
module cache.

In the future, this complex and imperfect machinery will be replaced
with simply embedding the needed files in the binary. We can't do that
quite yet (not without duplicating a lot of code) because Go 1.15 is
supported, and because Go 1.14 is used for deployment.

It's not neccessary to provide a custom socket.Environ (which served
to provide a custom GOPATH value pointing inside GOROOT; that path
is being removed here). Delete it along with some other unused code.

¹ https://golang.org/doc/go1.12#tour
² https://golang.org/doc/go1.16#modules

Fixes golang/go#44243.

Change-Id: Ia239700170b8614d21cafe9617b51dddf3e337d5
Reviewed-on: https://go-review.googlesource.com/c/tour/+/298529
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
X-Tour-Commit: b3263fcf7749f3600c8928d5f7d4e8a66a34b74f
passionSeven added a commit to passionSeven/website that referenced this issue Oct 18, 2022
As of CL 323897, the tour command and its static content are
contained in a module with a slightly different module path.
Update findRoot accordingly so it looks in the right place.

Updates golang/go#44243.

Change-Id: I5c979789440464ca3f9f99799bcc8e73db0769a3
Reviewed-on: https://go-review.googlesource.com/c/website/+/334969
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alberto Donizetti <alb.donizetti@gmail.com>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
passionSeven added a commit to passionSeven/website that referenced this issue Oct 18, 2022
Embed content, static and template directories directly in the tour
binary. This way it will work even if the package source is deleted.
The content in those directories is not large, the binary size goes
up from 14.2 MB to 15.2 MB in my testing (with Go 1.17 darwin/arm64).

Start using the Go 1.16 runtime for App Engine deployment.

Updates golang/go#44243.

Change-Id: I35fb32961cdc1edec1f8f8c0fc0193b07cef9acd
Reviewed-on: https://go-review.googlesource.com/c/website/+/342711
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
@golang golang locked and limited conversation to collaborators Jun 23, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done.
Projects
Archived in project
Development

No branches or pull requests

4 participants