Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: allow go run . in module mode without a go.mod file #36513

Open
go101 opened this issue Jan 12, 2020 · 40 comments
Open

cmd/go: allow go run . in module mode without a go.mod file #36513

go101 opened this issue Jan 12, 2020 · 40 comments
Labels
NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@go101
Copy link

go101 commented Jan 12, 2020

What version of Go are you using (go version)?

$ go version
go version go1.14beta1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What did you do?

  • Create a simple hello world project with only one main.go file.
  • Set GO111MODULE=on
  • Run go run . in the project folder.

What did you expect to see?

Run okay.

What did you see instead?

go: cannot find main module; see 'go help modules'

or for a git repository

go: cannot find main module, but found .git/config in /path/to/projects/helloworld
	to create a module there, run:
	go mod init
@mvdan
Copy link
Member

mvdan commented Jan 12, 2020

Why are you setting GO111MODULE=on? A module requires a go.mod file, so it seems like you have conflicting objectives here.

In any case, it seems like you disagree with #32027. That discussion happened a while ago, though.

@TapirLiu
Copy link
Contributor

Some people might have set it on globally before for some other reasons.
I just found this issue from a PR of one my project.

@TapirLiu
Copy link
Contributor

TapirLiu commented Jan 12, 2020

The current design also causes inconveniences in writing some Go tutorials with a simple Go file (and might have caused for some old tutorials). They must remind readers to create a go.mod file if GO111MODULE=on is set.

If some new gophers have set GO111MODULE=on before reading such old tutorials, the result might frustrate them at least for awhile.

@mvdan
Copy link
Member

mvdan commented Jan 12, 2020

I think it depends on what exactly the user is doing. If they need third party dependencies, requiring a go.mod sounds correct.

If they don't need third party dependencies, i.e. they use the standard library alone, I agree that a go.mod is largely not needed right now. You could work around that by asking the user to set GO111MODULE=off.

But it would still be a good idea to use a go.mod file in the long term, even if it's just for simple examples. It will be required to use newer language features, for example. See https://golang.org/doc/go1.13#language.

@TapirLiu
Copy link
Contributor

TapirLiu commented Jan 12, 2020

Yes, the go.mod file have some benefits, but not for new gophers to learn Go.

At least, the error message should not depend on whether or not there is a .git subfolder. The help message should also be printed for both cases. And even for the git repository case, the printed help message is also too unfriendly for new gophers. It is better to provide an example, such as go mod init my.module.

@jayconrod
Copy link
Contributor

I'd recommend against setting GO111MODULE=on globally. That was useful for working on modules checked out into GOPATH in Go 1.12 or earlier, but since GO111MODULE=auto was changed in 1.13, I don't think it makes sense now. Maybe in a few specific cases for tools, but not generally. Hopefully people who have it set know the implications, so I wouldn't expect that tutorials need to be written for them. If any documentation recommends new users set GO111MODULE=on, we should fix that.

This specific change in behavior was #32027. In short, if a user has GO111MODULE=on set explicitly, and there's no go.mod file in the current directory or in any parent, you can only build or import packages in the standard library or packages specified as .go files on the command line. So for example, this works:

$ cat >hello.go <<EOF
package main

import "fmt"

func main() {
  fmt.Println("hello")
}
EOF

$ go run hello.go
hello

Import paths in modules other than std and cmd won't work. That includes relative imports and patterns.

$ go run
go run: no go files listed
$ go run .
go: cannot find main module; see 'go help modules'
$ go list golang.org/x/tools/gopls
can't load package: cannot find module providing package golang.org/x/tools/gopls: working directory is not part of a module
$ go list ./...
go: cannot find main module; see 'go help modules'

The rationale for this change was that early on, people would set GO111MODULE=on, not create a go.mod file, then find that everything is very slow: we need to look up module paths and versions for every import on every command, since there's nowhere to write down dependencies.

I don't think this necessarily means go build . or go run . should not work, but fixing this would probably involve adding a significant special case, and it would be risky to do that this late in the cycle.

@bcmills @matloob WDYT? We might want to do this if/when GOPATH mode is deprecated, since GO111MODULE=on will be the only mode after that.

@jayconrod jayconrod added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Jan 13, 2020
@jayconrod jayconrod added this to the Unplanned milestone Jan 13, 2020
@bcmills
Copy link
Contributor

bcmills commented Jan 22, 2020

At the very least, we would need to decide what language version to use in module mode for a package in a directory outside of any module.

We would also need to determine the effective import path for such a package for debug info and build caching. (Perhaps the same import path that we generate in GOPATH mode today?)

@go101
Copy link
Author

go101 commented Jan 27, 2021

It looks this problem has been fixed in go1.16beta1.

@go101
Copy link
Author

go101 commented Feb 5, 2021

A new problem if go.mod doesn't present:

// main.go

package main

import _ "embed"

//go:embed main.go
var b []byte

func main() {
	println(string(b))
}

// error: pattern main.go: cannot embed irregular file main.go

The embedding feature is not usable without a go.mod file.

@derrickb
Copy link

derrickb commented Apr 4, 2021

Just want to say I came back to a go project that I (thankfully) rarely need to touch more than once per year, and found it really confusing to get back into it and just build something.

go version go1.16.3 windows/amd64

No matter what I had set for GO111MODULE, I kept getting an error about go.mod not being preset. I refer to one external package (an ancient image resizing one that is no longer maintained), and both the vscode extension and the go compiler were tripping up over this module file thing. Googling wasn't super helpful here. I eventually got past this by googling and stumbling through setting up a module, but I now have more files and a package.json-like setup that I don't really understand.

I second the OP, this isn't really user-friendly for simple projects.

@go101
Copy link
Author

go101 commented Apr 5, 2021

@derrickb it looks running simple go files doesn't need a go.mod file with Go toolchian v1.16. Maybe yours is another problem?

@ghost
Copy link

ghost commented Apr 5, 2021

@go101
go run main.go works, but go run . doesn't.

@go101
Copy link
Author

go101 commented Apr 5, 2021

Yes. Sorry, I forgot that.

@derrickb
Copy link

derrickb commented Apr 5, 2021

@go101 I need to mess around with it more... my project is just three small .go files and using go build but it required a mod file no matter what combination of env variables I had.

@ghost
Copy link

ghost commented Apr 5, 2021

@derrickb go build *.go should work. It's not very newbie-friendly, though.

@go101
Copy link
Author

go101 commented Apr 5, 2021

go build *.go doesn't work on Windows.

@ghost
Copy link

ghost commented Apr 6, 2021

Yes, on windows you'd have to list the files individually: go build a.go b.go main.go ...

@jayconrod
Copy link
Contributor

There's been some renewed attention on this issue, since GO111MODULE defaults to on in 1.16. It's not clear to me what folks want to change though.

In module-aware mode, if there is no go.mod file, you can still go build (or go run, etc) a package comprised of individual .go files listed on the command line (go build a.go b.go). The package can only import packages from the standard library.

This last restriction was added in Go 1.14 (#24031). Since there's no go.mod file, there's no place for the go command to save which version of each dependency is used, so before that, it went out to the network to look up every missing import. This was slow and non-deterministic, and we didn't feel that it was a good experience for beginners. An alternative would be using modules already downloaded into the module cache, but there's potential to use unexpectedly old versions, so we decided against that.

For this issue, it seems reasonable to me to let go build and other commands accept an absolute or relative path (go build . or go build ./foo) that would refer to the package in that directory. The package still wouldn't be able to import anything outside the standard library.

Does that sound reasonable?

@eliasnaur
Copy link
Contributor

For this issue, it seems reasonable to me to let go build and other commands accept an absolute or relative path (go build . or go build ./foo) that would refer to the package in that directory. The package still wouldn't be able to import anything outside the standard library.

Does that sound reasonable?

Yes, please. The distinction between a list of .go files and a path, with respect to the no-go.mod mode, is surprising to me; I learned about it from your message.

@thepudds
Copy link
Contributor

thepudds commented Apr 6, 2021

Hi @jayconrod I agree this would be a nice and friendly improvement.

For this issue, it seems reasonable to me to let go build and other commands accept an absolute or relative path (go build . or go build ./foo) that would refer to the package in that directory.

One clarifying question: would this also apply to commands like go build without any arguments? (I think that might be implied by what you wrote, but just trying to make it explicit, vs. perhaps there is some reason that wouldn’t work that is not immediately springing to mind for me 😅…)

@jayconrod
Copy link
Contributor

@thepudds Yes, go build would be equivalent to go build ., which should work.

@jayconrod
Copy link
Contributor

Discussed this with @bcmills, @matloob, and @_rsc. I think we're all agreed this is a good idea.

One clarification, go run . should still apply build constraints, same as if you'd run go run . inside a module. So it won't be exactly equivalent to go run *.go, which does not apply build constraints.

@jayconrod jayconrod changed the title cmd/go: it is inconvenient and unfriendly to require a simple program must have a go.mod file cmd/go: allow go run . in module mode without a go.mod file Apr 13, 2021
@jayconrod jayconrod added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. labels Apr 13, 2021
@jayconrod jayconrod modified the milestones: Unplanned, Backlog Apr 13, 2021
@pawan-saxena

This comment has been minimized.

@mjaggard
Copy link

Well this has caused my first experience of Go to be awful. I just checked out a project (https://github.com/rancher/migration-tools) and hoped to run it but I have no clue how. I figured that I needed to run glide install but then I can't find any incantation that will run the actual project. I know the project likely needs updating but it should be possible to run old programs.

@derrickb
Copy link

Yeah this made go a little less "pick up and go" which is such a shame, as that was a great feature of the language and tools.

@bcmills
Copy link
Contributor

bcmills commented Jul 14, 2021

@mjaggard, what were the commands that you ran and what was the output produced by those commands?

Even today, go run . within a checkout of a repo ought to suggest running go mod init, and once that's done go run . should work. Did go run . not suggest go mod init, or did you run into other problems from that point?

@mjaggard
Copy link

$ go mod init
go: cannot determine module path for source directory /Users/mat/dev/migration-tools (outside GOPATH, module path must be specified)

Example usage:
	'go mod init example.com/m' to initialize a v0 or v1 module
	'go mod init example.com/m/v2' to initialize a v2 module

Run 'go help mod init' for more information.
$ go mod init .
go: invalid module path ".": is a local import path

@uzxmx
Copy link

uzxmx commented Nov 20, 2021

Vote for such "pick up and go" feature, which makes go run main.go like an interpreter.

Take the below code for an example:

package main

import (
	"fmt"
	"github.com/vishvananda/netlink"
)

func main() {
	list, _ := netlink.LinkList()
	for _, l := range list {
		fmt.Println(l.Attrs().Name)
	}
}

For go1.13, when I run go run main.go, it works without go.mod (with GO111MODULE set to on).

For go1.14.1, it fails without go.mod:

main.go:5:2: cannot find module providing package github.com/vishvananda/netlink: working directory is not part of a module

And for go1.17.3, it also fails without go.mod:

main.go:5:2: no required module provides package github.com/vishvananda/netlink: go.mod file not found in current directory or any parent directory; see 'go help modules'

@Alexader
Copy link

Alexader commented Jan 2, 2022

Vote for such "pick up and go" feature, which makes go run main.go like an interpreter.

Take the below code for an example:

package main

import (
	"fmt"
	"github.com/vishvananda/netlink"
)

func main() {
	list, _ := netlink.LinkList()
	for _, l := range list {
		fmt.Println(l.Attrs().Name)
	}
}

For go1.13, when I run go run main.go, it works without go.mod (with GO111MODULE set to on).

For go1.14.1, it fails without go.mod:

main.go:5:2: cannot find module providing package github.com/vishvananda/netlink: working directory is not part of a module

And for go1.17.3, it also fails without go.mod:

main.go:5:2: no required module provides package github.com/vishvananda/netlink: go.mod file not found in current directory or any parent directory; see 'go help modules'

Yes! It is pretty annoying when you want to test out some code snippets and all you can do is to setup a small project for this. I think it is good to support this kind of script-style exection.

@chrislusf

This comment was marked as off-topic.

@madneal

This comment was marked as duplicate.

@bartgrantham
Copy link

$ go mod init
go: cannot determine module path for source directory /Users/mat/dev/migration-tools (outside GOPATH, module path must be specified)

Example usage:
	'go mod init example.com/m' to initialize a v0 or v1 module
	'go mod init example.com/m/v2' to initialize a v2 module

Run 'go help mod init' for more information.
$ go mod init .
go: invalid module path ".": is a local import path

It's no surprise that @mjaggard's first experience with Go was bad. I would be impressed if he stuck around.

And if you look up go mod init you get this page which talks about "Start a module that others can use".

Imagine a poor beginner programmer (or at least Go newbie) in this situation and how suddenly they are out of their depth. All they wanted to do was try an interesting piece of code they found online and now Go is asking them to choose a name for their module to share with others.

Or in my case, an experienced Go programmer 5 minutes ago. And now I'm sitting here asking "why do I need to choose a throwaway name to initialize a module just so I can go run foobar.go?" And I'm looking at the module naming guidelines and thinking about how anything involving modules adds this pretty large activation energy to getting started with small Go programs.

(not to mention the cognitive overhead of having to page back in all the go module arcana such as go mod parameters, go.mod syntax, odd import syntax, etc.)

The solution is simple: go mod init _ && go mod tidy But this kind of formality feels completely arbitrary unless you're an experienced Go programmer (and sometimes even then).

@yogo1212
Copy link

I hate to be that guy.. making the module requirement a default and thus forcing the duplication of packages all over my hard drive wasn't the best of ideas :-(

I made the mistake of updating Go on a server where the code was assembled in a folder during deployment.
Making builds re-use the previously downloaded packages is now very, very hacky.

@kpym
Copy link

kpym commented Apr 16, 2023

The package can only import packages from the standard library.

This last restriction was added in Go 1.14 (#24031). Since there's no go.mod file, there's no place for the go command to save which version of each dependency is used, so before that, it went out to the network to look up every missing import. This was slow and non-deterministic, and we didn't feel that it was a good experience for beginners. An alternative would be using modules already downloaded into the module cache, but there's potential to use unexpectedly old versions, so we decided against that.

@jayconrod It should be possible to find a compromise between "fast and deterministic" and "easy to run snippets and use go as a scripting language".

For example, we can imagine that if the run command finds a "not so old" version of the requested non-standard library in the cache, it will use it, otherwise it will download the latest available version and possibly make it available in the cache.

What "not so old" means and whether "on the fly" packages should be cached could be configurable, with good reasonable defaults for newbies.

There could also be a flag like --imports=(only cached|no cached|no|auto).

@yogo1212
Copy link

@kpym I very much appreciate this line of thinking.

In my eyes, the mandate of reproducible builds and of deterministic dependency management goes against the idea of "simplicity". Those are important for organizational software deployment - not so much for almost every other use case.
Like debugging dependencies, for instance. I can't easily switch to a local version in order to place a log.Println or altering a value (not that it happens often).
Increased build times don't weigh as much to me as the time I have to actively spend setting up the debug build and watching the size of the module cache.

Having come to terms with the situation, a command line option for controlling would be simple enough addition (imo).
At the same time, I don't think it's a good idea to add another option building on yet another concept.
There are GO111MODULE and -mod for controlling the module cache use that affect go run as well as go build.

Building on your idea, your issue is better addressed with another option for -mod?

-mod=relaxed tells the compiler to pick the latest available version of a package (ignoring go.mod) from the vendor directory and the module cache, preferring the vendor directory. If the package was not found, it will be downloaded to the module cache.

That would not directly solve the issue with go run . as -mod is only used in module-aware mode and . doesn't contain a go.mod file (as per the issue).
The original issue topic probably falls more into the -modfile= territory.
Personally, I would prefer a mechanism that works without go.mod as well but that would probably require finding a simpler paradigm - maybe one that unifies GO111MODULE and -mod.

@dimitarvp
Copy link

dimitarvp commented Jul 6, 2023

Just stumbled upon this today. I want to have a number of local Golang playground scripts and the inability to just go run whatever.go immediately after creating the file is displeasing. I understand the arguments given by @jayconrod here:

Since there's no go.mod file, there's no place for the go command to save which version of each dependency is used, so before that, it went out to the network to look up every missing import. This was slow and non-deterministic, and we didn't feel that it was a good experience for beginners. An alternative would be using modules already downloaded into the module cache, but there's potential to use unexpectedly old versions, so we decided against that.

...and I agree that should be the default behavior. But I would appreciate a go run CLI flag that basically says "use whatever you can find in the module cache and download the rest". (An extra, 2nd, CLI flag that says "always fetch the latest versions of the dependencies from the internet" can be useful as well.)

Having a no-fuss go run -just-do-it whatever.go that downloads and/or uses cached modules would be awesome. I believe it's well understood that when you want control you absolutely should go the go mod init scope/name and go mod tidy route. But for quick scripting we should have an escape hatch.

Failing all that, I suppose I can make my own Golang workspace with subdirs for all my playground scripts and use something like just to run them via aliases e.g. just pubsub or just multichannel etc.

(I also tried GO111MODULE=off go run whatever.go but that only works if you did go get . in the same directory beforehand.)

@NewUserHa
Copy link

NewUserHa commented Aug 6, 2023

Since there's no go.mod file, there's no place for the go command to save which version of each dependency is used, so before that, it went out to the network to look up every missing import. This was slow and non-deterministic, and we didn't feel that it was a good experience for beginners. An alternative would be using modules already downloaded into the module cache, but there's potential to use unexpectedly old versions, so we decided against that.

go run test.go should use only local modules(including std and 3rd depencies) in $goroot/current folder, and could just tip users 'not found any' or 'found exists local version x.x.x'. if it's 'not found', it's the users who should go to use go get ... in any folder by themselves.
or users can even go get whatever they want ahead before they write any code for speeding up the latter go building

//
use set GO111MODULE=auto can also not solve this, since it's searching in ...\go\src\... gopath style instead of ...\go\pkg\... gomodule style path

@ErikSwan
Copy link

ErikSwan commented Aug 7, 2023

Go team, please prioritize this!

As a brand new Go user I ended up here after trying to run a few simple script-like pieces of code I found online. A new user just trying to build and run a few simple examples should not have to Google inscrutable error messages and read through a GitHub issue to try to figure out how to get it working. The beginner experience should not be this painful!

Why is this issue languishing in the Unplanned/Backlog milestones? Please prioritize it, for the sake of new users and for making Go more of a joy to use, especially in the "pick up and go" type of use case.

@mvdan
Copy link
Member

mvdan commented Aug 7, 2023

@ErikSwan I appreciate the enthusiasm, but there are over eight thousand open issues right now, and a limited number of hands. If they were to listen to the requests to prioritize some issues over others, we'd make even less progress on the overall number of issues. If you really care about an issue in particular which is currently in the backlog, the best way to help is to contribute :)

@MrNocTV
Copy link

MrNocTV commented Nov 13, 2023

This should be possible,
I have a legacy project without a go.mod file and it's impossible to run it now. :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests