x/tour: offline version of 'A Tour of Go' broken with modules #44243

fzipp · 2021-02-12T22:02:28Z

What version of Go are you using (`go version`)?

$ go version
go version go1.16rc1 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/fzipp/Library/Caches/go-build"
GOENV="/Users/fzipp/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/fzipp/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/fzipp/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16rc1"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/lc/1h4lth_d68zfy0rvwrv37tkm0000gn/T/go-build3688970254=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I tried to install the offline tour of Go via go get as described here and here and run it.

$ go get golang.org/x/tour
go: downloading golang.org/x/tour v0.0.0-20201207214521-004403599411
go: downloading golang.org/x/tools v0.0.0-20190312164927-7b79afddac43
go: downloading golang.org/x/net v0.0.0-20190311183353-d8887717615a
$ tour

What did you expect to see?

Opening a web browser displaying my local version of the tour.

What did you see instead?

$ tour
2021/02/12 22:53:03 Couldn't find tour files: could not find go-tour content; check $GOROOT and $GOPATH

Of course the files can't be found in $GOPATH, because go get downloaded it into the module cache.

The text was updated successfully, but these errors were encountered:

dmitshur · 2021-02-12T22:45:15Z

Thanks for reporting. I'll take a look.

gopherbot · 2021-02-13T04:45:37Z

Change https://golang.org/cl/291849 mentions this issue: tour: use embed and io/fs

dmitshur · 2021-02-13T04:46:49Z

It's possible to fix this either by adjusting the code to look for tour files in module cache, or by embedding them in the binary. I chose with the latter and sent CL 291849.

fzipp · 2021-02-13T06:26:58Z

Thanks, the CL version starts locally.
There's still a problem with tour snippets that import "golang.org/x/tour/..." packages:

http://127.0.0.1:3999/moretypes/18
http://127.0.0.1:3999/methods/25
import "golang.org/x/tour/pic"
compile3.go:3:8: no required module provides package golang.org/x/tour/pic: working directory is not part of a module

http://127.0.0.1:3999/moretypes/23
import "golang.org/x/tour/wc"

http://127.0.0.1:3999/methods/22
import "golang.org/x/tour/reader"

http://127.0.0.1:3999/concurrency/8
import "golang.org/x/tour/tree"

gopherbot · 2021-02-13T23:20:08Z

Change https://golang.org/cl/291889 mentions this issue: tour: build snippets in module mode

dmitshur · 2021-02-13T23:20:15Z

Right, that will happen if the tour source code isn't available in GOPATH/src/golang.org/x/tour. To remove that requirement, we need to update tour to start building snippets in module mode (and supply a go.mod file that provides a version of the golang.org/x/tour module).

I've sent CL 291889 for that.

gopherbot · 2021-03-04T02:09:13Z

Change https://golang.org/cl/298529 mentions this issue: tour: look for content in module cache, not GOROOT/misc/tour

We want to start building snippets in module mode rather than legacy legacy GOPATH mode, to make tour work well when there's no tour code in GOPATH/src and to be more future-proof. Building in module mode means we need to specify a go.mod file to use for each snippet. The tour web UI design predates the module mode and its go.mod files. Txtar-based multi-file support was added to the playground (that the tour uses for executing snippets) in golang.org/issue/32040, but the web UI hasn't been updated to have first-class separate <textarea> elements or tabs for editing multiple files. For now, we can get by with providing a default go.mod file for all tour snippets, since they only need some packages in the tour module. Update to a newer version of the golang.org/x/tools/playground/socket package which includes multi-file snippet support added in CL 204237. For golang/go#44243. Change-Id: Ib05725c52fbe43658b2f22ac728b1e901e635824 Reviewed-on: https://go-review.googlesource.com/c/tour/+/291889 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>

dmitshur · 2021-03-04T02:17:11Z

It's possible to fix this either by adjusting the code to look for tour files in module cache, or by embedding them in the binary. I chose with the latter and sent CL 291849.

We can't quite start relying on Go 1.16+ APIs while Go 1.15 is still supported (not without causing much code duplication), so I sent CL 298529 to get the module cache fix in the interim.

We want to start building snippets in module mode rather than legacy legacy GOPATH mode, to make tour work well when there's no tour code in GOPATH/src and to be more future-proof. Building in module mode means we need to specify a go.mod file to use for each snippet. The tour web UI design predates the module mode and its go.mod files. Txtar-based multi-file support was added to the playground (that the tour uses for executing snippets) in golang.org/issue/32040, but the web UI hasn't been updated to have first-class separate <textarea> elements or tabs for editing multiple files. For now, we can get by with providing a default go.mod file for all tour snippets, since they only need some packages in the tour module. Update to a newer version of the golang.org/x/tools/playground/socket package which includes multi-file snippet support added in CL 204237. For golang/go#44243. Change-Id: Ib05725c52fbe43658b2f22ac728b1e901e635824 Reviewed-on: https://go-review.googlesource.com/c/tour/+/291889 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> X-Tour-Commit: b72029adf3393d3635828fedcddd5ae7053d3250

Tour content hasn't been available in GOROOT since Go 1.12¹, so stop looking for it there. A more useful fallback place to find tour content is the module cache. When the golang.org/x/tour is in the build list, it will be found via build.Import. It will not be found, however, when the tour binary is run outside of a module. This situation is more common as of Go 1.16, since the default GO111MODULE value changed to on². If the version of the tour binary is known, access the content for that version from the module cache. In the future, this complex and imperfect machinery will be replaced with simply embedding the needed files in the binary. We can't do that quite yet (not without duplicating a lot of code) because Go 1.15 is supported, and because Go 1.14 is used for deployment. It's not neccessary to provide a custom socket.Environ (which served to provide a custom GOPATH value pointing inside GOROOT; that path is being removed here). Delete it along with some other unused code. ¹ https://golang.org/doc/go1.12#tour ² https://golang.org/doc/go1.16#modules Fixes golang/go#44243. Change-Id: Ia239700170b8614d21cafe9617b51dddf3e337d5 Reviewed-on: https://go-review.googlesource.com/c/tour/+/298529 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> X-Tour-Commit: b3263fcf7749f3600c8928d5f7d4e8a66a34b74f

gopherbot · 2021-07-16T00:44:21Z

Change https://golang.org/cl/334969 mentions this issue: tour: look for content in x/website module

jfesler · 2021-07-16T00:55:33Z

This works fine for running the binaries on the same host as you build the binaries.

https://github.com/google/go-licenses does the same trick. Said binaries were not useful anywhere else. I ended up having to duplicate the license data into a place I can embed, to produce a portable binary.

dmitshur · 2021-07-16T02:29:53Z

@jfesler Thanks for that information. Indeed, looking for content in the module cache can only work on a machine where Go is installed.

Go 1.17 will come out pretty soon, at which point Go 1.15 will become unsupported and we can start relying on //go:embed for embedding assets into the binary itself (CL 342711), which won't have that limitation.

As of CL 323897, the tour command and its static content are contained in a module with a slightly different module path. Update findRoot accordingly so it looks in the right place. Updates golang/go#44243. Change-Id: I5c979789440464ca3f9f99799bcc8e73db0769a3 Reviewed-on: https://go-review.googlesource.com/c/website/+/334969 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alberto Donizetti <alb.donizetti@gmail.com> Trust: Dmitri Shuralyov <dmitshur@golang.org>

gopherbot · 2021-08-17T01:32:48Z

Change https://golang.org/cl/342711 mentions this issue: tour: use embed and io/fs

Embed content, static and template directories directly in the tour binary. This way it will work even if the package source is deleted. The content in those directories is not large, the binary size goes up from 14.2 MB to 15.2 MB in my testing (with Go 1.17 darwin/arm64). Start using the Go 1.16 runtime for App Engine deployment. Updates golang/go#44243. Change-Id: I35fb32961cdc1edec1f8f8c0fc0193b07cef9acd Reviewed-on: https://go-review.googlesource.com/c/website/+/342711 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> Trust: Dmitri Shuralyov <dmitshur@golang.org>

We want to start building snippets in module mode rather than legacy legacy GOPATH mode, to make tour work well when there's no tour code in GOPATH/src and to be more future-proof. Building in module mode means we need to specify a go.mod file to use for each snippet. The tour web UI design predates the module mode and its go.mod files. Txtar-based multi-file support was added to the playground (that the tour uses for executing snippets) in golang.org/issue/32040, but the web UI hasn't been updated to have first-class separate <textarea> elements or tabs for editing multiple files. For now, we can get by with providing a default go.mod file for all tour snippets, since they only need some packages in the tour module. Update to a newer version of the golang.org/x/tools/playground/socket package which includes multi-file snippet support added in CL 204237. For golang/go#44243. Change-Id: Ib05725c52fbe43658b2f22ac728b1e901e635824 Reviewed-on: https://go-review.googlesource.com/c/tour/+/291889 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> X-Tour-Commit: b72029adf3393d3635828fedcddd5ae7053d3250

Tour content hasn't been available in GOROOT since Go 1.12¹, so stop looking for it there. A more useful fallback place to find tour content is the module cache. When the golang.org/x/tour is in the build list, it will be found via build.Import. It will not be found, however, when the tour binary is run outside of a module. This situation is more common as of Go 1.16, since the default GO111MODULE value changed to on². If the version of the tour binary is known, access the content for that version from the module cache. In the future, this complex and imperfect machinery will be replaced with simply embedding the needed files in the binary. We can't do that quite yet (not without duplicating a lot of code) because Go 1.15 is supported, and because Go 1.14 is used for deployment. It's not neccessary to provide a custom socket.Environ (which served to provide a custom GOPATH value pointing inside GOROOT; that path is being removed here). Delete it along with some other unused code. ¹ https://golang.org/doc/go1.12#tour ² https://golang.org/doc/go1.16#modules Fixes golang/go#44243. Change-Id: Ia239700170b8614d21cafe9617b51dddf3e337d5 Reviewed-on: https://go-review.googlesource.com/c/tour/+/298529 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> X-Tour-Commit: b3263fcf7749f3600c8928d5f7d4e8a66a34b74f

As of CL 323897, the tour command and its static content are contained in a module with a slightly different module path. Update findRoot accordingly so it looks in the right place. Updates golang/go#44243. Change-Id: I5c979789440464ca3f9f99799bcc8e73db0769a3 Reviewed-on: https://go-review.googlesource.com/c/website/+/334969 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alberto Donizetti <alb.donizetti@gmail.com> Trust: Dmitri Shuralyov <dmitshur@golang.org>

Embed content, static and template directories directly in the tour binary. This way it will work even if the package source is deleted. The content in those directories is not large, the binary size goes up from 14.2 MB to 15.2 MB in my testing (with Go 1.17 darwin/arm64). Start using the Go 1.16 runtime for App Engine deployment. Updates golang/go#44243. Change-Id: I35fb32961cdc1edec1f8f8c0fc0193b07cef9acd Reviewed-on: https://go-review.googlesource.com/c/website/+/342711 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> Trust: Dmitri Shuralyov <dmitshur@golang.org>

gopherbot added this to the Unreleased milestone Feb 12, 2021

dmitshur self-assigned this Feb 12, 2021

dmitshur added NeedsInvestigation modules labels Feb 12, 2021

dmitshur added NeedsFix and removed NeedsInvestigation labels Feb 13, 2021

gopherbot closed this as completed in golang/tour@b3263fc Mar 5, 2021

rsc unassigned dmitshur Jun 23, 2022

heschi moved this to Done in Go Release Sep 27, 2022

heschi added this to Go Release Sep 27, 2022

golang locked and limited conversation to collaborators Jun 23, 2023

gopherbot added the FrozenDueToAge label Jun 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

x/tour: offline version of 'A Tour of Go' broken with modules #44243

x/tour: offline version of 'A Tour of Go' broken with modules #44243

fzipp commented Feb 12, 2021

dmitshur commented Feb 12, 2021

gopherbot commented Feb 13, 2021

dmitshur commented Feb 13, 2021

fzipp commented Feb 13, 2021

gopherbot commented Feb 13, 2021

dmitshur commented Feb 13, 2021

gopherbot commented Mar 4, 2021

dmitshur commented Mar 4, 2021

gopherbot commented Jul 16, 2021

jfesler commented Jul 16, 2021

dmitshur commented Jul 16, 2021 •

edited

Loading

gopherbot commented Aug 17, 2021

x/tour: offline version of 'A Tour of Go' broken with modules #44243

x/tour: offline version of 'A Tour of Go' broken with modules #44243

Comments

fzipp commented Feb 12, 2021

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

dmitshur commented Feb 12, 2021

gopherbot commented Feb 13, 2021

dmitshur commented Feb 13, 2021

fzipp commented Feb 13, 2021

gopherbot commented Feb 13, 2021

dmitshur commented Feb 13, 2021

gopherbot commented Mar 4, 2021

dmitshur commented Mar 4, 2021

gopherbot commented Jul 16, 2021

jfesler commented Jul 16, 2021

dmitshur commented Jul 16, 2021 • edited Loading

gopherbot commented Aug 17, 2021

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

dmitshur commented Jul 16, 2021 •

edited

Loading