go1.14 is not one of the actively supported versions (go1.15, go1.16). Can you reproduce the issue with one of the supported versions?

go1.14 is not one of the actively supported versions (go1.15, go1.16). Can you reproduce the issue with one of the supported versions?

Unfortunately, I have to cope with whats delivered by (stable) distros. Even 1.14 is only delivered by additional backport repos, no 1.15 available.

Like @cagedmantis said, 1.14.x is no longer supported, so even if a bug was spotted here, at most there would be a backport to 1.15.x.

I imagine that what's happening here is that @ is not allowed in package paths, as it clashes with the modules syntax to specify a version, like go get foo/bar@v1.2.3.

Consider escaping that special character, just like you might usually do for slashes or spaces.

@bcmills where are these restrictions documented? I don't see it in go help importpath or go help packages.

I believe the current set of restrictions are #45549 and @ hasn't been allowed since modules were introduced in 1.11

cc @jayconrod @matloob

Building w/ 'go build' and setting GOPATH to a directory that happens to have @ in its name.

Please provide concrete steps to reproduce the observed behavior using a supported version of the go tool (1.16.3 or 1.15.11).

The posted go env output (GOPATH="/usr/lib/go-1.13") does not appear to correspond to “setting GOPATH”. The specific details are very relevant to the validation that the go command performs, so if we are to address the underlying use-case we need to know how to reproduce it accurately.

I believe the current set of restrictions are #45549 and @ hasn't been allowed since modules were introduced in 1.11

cc @jayconrod @matloob

I'm confused - this wasn't supposed to be a module name, but a plain path name in GOPATH.

Meanwhile I've found a workaround (pretty dirty hack) to build moby, but autogenerating go.mod via 'go mod init'.
BUT: I need to point GOPATH to an unwritable directly, so it can't download anything - otherwise lots of errors break the build.
This is very ugly, but at least seems to work (somehow).

Is there any way for preventing any downloads and remote access completely ?

--mtx

Based on the command you posted earlier, I expect you'll still see this error (with a slightly different message) in Go 1.16 because of this argument:

/home/nekrad/src/deb-pkg/pkg/moby-engine@master.git/cmd/dockerd

To avoid confusion and ambiguous commands, go build disallows @ characters in arguments except when they're used in path@version arguments in module mode.

This isn't a problem for import paths, since the language spec does not allow @ characters in import declarations. It may be a problem (as in this case) for relative and absolute directory paths, but I think it's better to keep the current strict behavior rather than change anything.

I'm confused - this wasn't supposed to be a module name, but a plain path name in GOPATH.

/home/nekrad/src/deb-pkg/pkg/moby-engine@master.git/cmd/dockerd is a directory path. You may be able to build dockerd using its normal package path instead, but it looks like this has been customized enough that I'm not sure what that would be. cmd/dockerd maybe?

Is there any way for preventing any downloads and remote access completely ?

In module-aware mode, go mod vendor will build a vendor directory (fetching what it needs from the network), then you can build offline with -mod=vendor (which may be enabled automatically, depending on the go version in go.mod). There are a few ways to build offline, but that's probably the easiest.

To avoid confusion and ambiguous commands, go build disallows @ characters in arguments except when they're used in path@version arguments in module mode.

The go command should (and does!) allow arguments that contain the @ character if they unambiguously refer to a directory rather than a package import path:

$ go version
go version devel go1.17-1108cbe60 Thu May 6 02:21:55 2021 +0000 linux/amd64

$ mkdir foo@bar

$ cd foo@bar

foo@bar$ go mod init example.com/bar
go: creating new go.mod: module example.com/bar

foo@bar$ cat > main.go
package main

func main() {}

foo@bar$ pwd
/tmp/tmp.eEQh4n8Unb/foo@bar

foo@bar$ go build $(pwd)

foo@bar$ ls
bar  go.mod  main.go

foo@bar$

However, in GOPATH mode when outside of GOPATH/src, the go command synthesizes an import path for the working directory based on its file path. It appears that when the directory includes an @ symbol, that synthesized import path also includes an @ symbol and is then rejected as an argument:

foo@bar$ go list .
example.com/bar

foo@bar$ export GO111MODULE=off

foo@bar$ go list .
_/tmp/tmp.eEQh4n8Unb/foo@bar

foo@bar$ go build .

foo@bar$ go build $(pwd)
package /tmp/tmp.eEQh4n8Unb/foo@bar: can only use path@version syntax with 'go get' and 'go install' in module-aware mode

To avoid confusion and ambiguous commands, go build disallows @ characters in arguments except when they're used in path@version arguments in module mode.

This is a huge problem for automated build engines, if a programming language / compiler imposes restrictions on the source root directories. This also applies to the inherent "src" prefix. That's a heavy violation of layers that make maintaining automatic build infrastructures a pretty hard problem.

In my case, the generic deb/rpm packaging engine (*1) puts the source trees (git repos) into subdirs that may have '@' in their names, when explicit package versions (note: package versions have nothing do to w/ go module versions). If the go compiler (just one of a dozen different compilers used in a distro context) has it's own funny ideas how source directory names may look like, I have to do invasive changes in the packaging engine and the surrounding infrastructures.

This isn't a problem for import paths, since the language spec does not allow @ characters in import declarations. It may be a problem (as in this case) for relative and absolute directory paths, but I think it's better to keep the current strict behavior rather than change anything.

By "import path" you mean the arguments of the import statement ?
These are not of my concern, but the place where the whole code tree is placed in the file system.

I'm confused - this wasn't supposed to be a module name, but a plain path name in GOPATH.

/home/nekrad/src/deb-pkg/pkg/moby-engine@master.git/cmd/dockerd is a directory path. You may be able to build dockerd using its normal package path instead, but it looks like this has been customized enough that I'm not sure what that would be. cmd/dockerd maybe?

I played a lot with GOPATH, but always lead to lots of error messages, some indicate clashes with the standard library (don't recall the exact messages). So, I tried to generate go.mod (via go mod init), which also just works under certain special circumstances. (see below)

The upstream builds docker within a docker container (exactly what I CAN NOT do) and copying around their source tree so /go/src. Without having 'src' prefix in the path name does not seem to work - go wont find the correct sources.
According to the official documentation (*2), that sounds pretty much mandatory - haven't found any documentation on how to do that w/o having that special source tree layout.

Is there any way for preventing any downloads and remote access completely ?

In module-aware mode, go mod vendor will build a vendor directory (fetching what it needs from the network), then you can build offline with -mod=vendor (which may be enabled automatically, depending on the go version in go.mod). There are a few ways to build offline, but that's probably the easiest.

My problem here is 'go mod init'. Documentation suggest that this can be used to create a go.mod from vendor.conf (which does exist in the docker/moby source). BUT: it tries to download a lot of stuff (another thing I CAN NOT have in a distro build context) and uses that stuff instead of from vendor dir - which in turn breaks a lot of things. I can prevent that by pointing GOPATH to an unwritable directory - a very weird hack.

Is there any way to prevent 'go mod init' from doing any downloads and just take what is already there ?
Or is there any other way of creating a go.mod file from the existing sources.

Finally, all I wanna do is building exactly the sources in that tree, using the distro's toolchain (and it's standard library), and no external interactions.

--mtx

*1) https://github.com/metux/deb-pkg
*2) https://golang.org/doc/gopath_code

For better or worse, cmd/go is a very opinionated tool. This make it easier for different projects to work together, since strong conventions are enforced. However, if you have a lot of external constraints (sounds like that's the case here), you may run into problems.

It seems like a fairly simple change is possible: cmd/go could allow @ in arguments if they appear to be absolute or relative file paths (starting with ./ or ../). Version suffixes have never been allowed on file paths, so there doesn't seem much risk of confusion here.

Is there any way to prevent 'go mod init' from doing any downloads and just take what is already there ?
Or is there any other way of creating a go.mod file from the existing sources.

You can use replace directives to stitch multiple local modules together into a build (note that you will also need to require every module that provides a directly-imported package or, for Go 1.17 or later, a transitively-imported package).

Then you can set GOPROXY=off in the environment to prevent the go command from downloading other modules.

This sort of stitching-together should be easier using Go 1.17 (which is still under development) with a go.mod file that specifies go 1.17 or later. That will avoid loading go.mod files for module dependencies that don't provide imported packages, which may substantially decrease the number of replace directives that you need in order to produce a fully-local configuration.

Just a note that we should never accept paths that start with @ characters, as the system linker and other tools can treat them specially, as response files (https://docs.microsoft.com/en-us/windows/win32/midl/response-files).

We could maybe make a more explicit check to reject the pathis with '@'s mode, but we don't plan to work on this because it's a GOPATH mode issue.