Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: Go 2: cmd/go: allow relative imports #20883

Closed
cch123 opened this issue Jul 2, 2017 · 34 comments
Closed

proposal: Go 2: cmd/go: allow relative imports #20883

cch123 opened this issue Jul 2, 2017 · 34 comments
Labels
NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. Proposal v2 A language change or incompatible library change
Milestone

Comments

@cch123
Copy link
Contributor

cch123 commented Jul 2, 2017

I think this is a useful feature for library writers, sometimes dependency can not be avoided for a lib.
eg. a proxy lib may use a sql parser written by third party.

The library package layout may look like follows:

myproxy: host on github or sth
  main.go
  └── import "github.com/xxx/sqlparser"
  └── import "github.com/yyy/sometcppoolpackage"

If I want to make the dependency invisible to lib endusers, currently I have to move the dependencies to internal package, and manually change the import path to, maybe:

import "github.com/myname/dbproxy/internal/github.com/xxx/sqlparser"

It's quite ugly and inconvenient.

If internal can be imported easily, then import path will be,

import "internal/github.com/xxx/sqlparser"

quite clear

I noticed that some libs also have inner package, and they currently layout their project like the following:

mqtool is hosted on github.com/xxx/mqtool

mqtool/
  main.go
   └─ import "github.com/xxx/mqtool/cli"
  otherfile.go
  └─ import "github.com/xxx/mqtool/cli"
  └─ import "github.com/xxx/mqtool/logutil"
  ... many many file
  otherfileend.go
  └─ import "github.com/xxx/mqtool/bytes2"

If someone wants to fork this mqtool lib to fix bugs or maintain it as his own, then he will meet a lot problems (to modify endless import paths).

Someone may say that he can git clone this project to the correct path, but go get can not support such case, which is really weird (because go get will put this project to the forked path, not the original path).

If internal package can be found in relative path, I think things can go much better :-)

Then the above project may become like this:

mqtool is hosted on github.com/xxx/mqtool

mqtool/
  main.go
   └─ import "internal/cli"
  otherfile.go
  └─ import "internal/cli"
  └─ import "internal/logutil"
  ... many many file
  otherfileend.go
  └─ import "internal/bytes2"
@gopherbot gopherbot added this to the Proposal milestone Jul 2, 2017
@rsc
Copy link
Contributor

rsc commented Jul 17, 2017

This is not really about internal at all, since you could have the same situation with non-internal subdirectories you want to import. For better or worse we've decided that having fully-qualified paths is useful for being able to understand imports when you see them. It's not clear we want to break that now.

@rsc rsc changed the title proposal: make internal package can be found in the relative path proposal: cmd/go: allow relative "internal/..." imports Jul 17, 2017
@rsc rsc closed this as completed Jul 17, 2017
@leonklingele
Copy link
Contributor

Can this be reopened as a proposal for Go 2 or should I create another proposal?

I'm especially asking for something similar to this:

mqtool is hosted on github.com/xxx/mqtool

mqtool/
  main.go
   └─ import "internal/cli"
  otherfile.go
  └─ import "internal/cli"
  └─ import "internal/logutil"
  ... many many file
  otherfileend.go
  └─ import "internal/bytes2"

It doesn't make sense to hard-code the platform / domain I'm going to host my code on inside the code itself.

@ianlancetaylor
Copy link
Contributor

I'll reopen for Go 2.

@ianlancetaylor ianlancetaylor changed the title proposal: cmd/go: allow relative "internal/..." imports proposal: Go 2: cmd/go: allow relative "internal/..." imports Aug 16, 2017
@ianlancetaylor ianlancetaylor added v2 A language change or incompatible library change and removed Proposal-Declined labels Aug 16, 2017
@leonklingele
Copy link
Contributor

While I'm not a fan of internal/.., something like this ("project-relative imports") should be considered for Go 2. internal could be a package name, can it? So this could potentially break existing code.
Instead, I'd prefer import "/x/y" (leading slash indicates project-relative import, / is the root of the project)

Hard-coding the platform / domain one intends to host code under should not be part of the code itself, moving code between different hosting providers (or even renaming the project) should be easier and should not require a.) changes to the source code itself (s/old-domain/new-domain/) b.) the use of a self-hosted go-import-providing server: example which gets the source from diverse hosting platforms but uses a single, static custom domain name.

This proposal basically is a requirement for proposal: incrementally modify the Go toolchain to work without GOPATH

@dlsniper
Copy link
Contributor

I would discourage relative imports to begin with. They can lead to unexpected results. The current model of paths being GOPATH dependent works really well and can be adapted towards a package oriented workflow rather easily.

This proposal basically is a requirement for proposal: incrementally modify the Go toolchain to work without GOPATH

I would disagree about that.

@leonklingele
Copy link
Contributor

I would discourage relative imports to begin with. They can lead to unexpected results

For example?

I would disagree about that.

Why? How else would the Go compiler toolchain know if an import should be retrieved from $GOPATH / vendor/ (i.e. external packages) or from the current project?

@johnDoe2018
Copy link

I'd share another example where this could be useful. Let's assume there is a project with the following structure:


github.com/janeRoe/myapp

    • main.go - imports models, handlers, and routes:
import (
	"github.com/janeRoe/myapp/handlers"
	"github.com/janeRoe/myapp/models"
	"github.com/janeRoe/myapp/routes"
)
    • handlers/
    • models/
    • routes/

I want to fork the project and make a few changes to, say, routes. I click GH "Fork" button and get a copy of the project that now lives under github.com/johnAnonDoe/myapp. But the main.go's imports still point to the old "github.com/janeRoe/myapp/...". That means the following requirements cannot be satisfied at the same time:

  1. Changes to the routes/ subpackage must affect my fork (which, in case of absolute import paths, requires update of the main.go).
  2. My fork should be go getable.
  3. It should be easy to merge the changes back to the initial repo.

@ianlancetaylor ianlancetaylor changed the title proposal: Go 2: cmd/go: allow relative "internal/..." imports proposal: Go 2: cmd/go: allow relative imports Feb 27, 2018
@ianlancetaylor
Copy link
Contributor

Because currently Go packages are distinguished by import paths, support for relative import paths would mean that very early on in the build process the relative imports would have to be converted to full paths.

As far as I can see the only time it is useful to have a relative import path is when you are going to move a set of packages around. That seems to happen relatively rarely, and it would be straightforward to write a tool to move a set of packages while rewriting the import paths. I'm not aware of any such tool today, but if we had one would there still be a need for relative import paths?

@ianlancetaylor ianlancetaylor added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Feb 27, 2018
@cch123
Copy link
Contributor Author

cch123 commented Mar 1, 2018

@ianlancetaylor , if I fork other people's project on github, and want to contribute back to this project.It seems there is still no way to keep the forked project go gettable and the import paths inside this forked project to be same with the original's.

If I want to use it as my own, I have to rewrite the import path, and if I want to contribute, I have to rewrite the import path again. So I think it's not quite friendly to the current github workflow.(but if you can provide a rewriting tool, this annoying procedure can be more convenient than now.

@ianlancetaylor
Copy link
Contributor

@cch123 Seems like you may be assuming that the other person's project is already using relative imports, which it most likely is not.

@cch123
Copy link
Contributor Author

cch123 commented Mar 2, 2018

@ianlancetaylor ,for example,I forked etcd project,this file https://github.com/coreos/etcd/blob/master/discovery/discovery.go contains imports of the same project's paths:

	"github.com/coreos/etcd/client"
	"github.com/coreos/etcd/pkg/transport"
	"github.com/coreos/etcd/pkg/types"

After forked to my own repo, the import path should be:

	"github.com/cch123/etcd/client"
	"github.com/cch123/etcd/pkg/transport"
	"github.com/cch123/etcd/pkg/types"

If I don't change the import paths and then execute

go get github.com/cch123/etcd

Then the project will appear at $GOPATH/src/github.com/cch123/etcd

But the import paths inside this project are not correct. I'm not assuming other people is using relative imports, but I mean if we could support some way(eg.relative import? or some way of internal usage) to unbind the import path with the host address, then I will not be forced to change the import paths back and forth.

The original issue is that I want to make use of internal feature to bind dependencies together with my library. Because dependencies on site like github cannot avoid risks to be deleted by original author(like the sad story of leftpad in npm community). So I think feature of this could make my library more safe in such situation. But if I make these dependencies as my library internals , it will also meet the import path problem. I have to change all the import paths of these dependencies if they use the import path like the above case.

@dlsniper
Copy link
Contributor

dlsniper commented Mar 2, 2018

@cch123 I recommend you read http://blog.sgmansfield.com/2016/06/working-with-forks-in-go/ on how to work with forks if you want to contribute changes to a project. If you want to use your fork instead of the original place, dep supports this (and I think vgo may support this at some point, if not already).

Relative import paths is something I would love to see gone in Go2 as they are a constant source of pain.

@cch123 cch123 closed this as completed Mar 2, 2018
@cch123 cch123 reopened this Mar 2, 2018
@nkovacs
Copy link

nkovacs commented Mar 2, 2018

I agree with @cch123 forking packages with subpackages in them is a huge pain.
Changing origins is not a solution (you can't go get them). Dep's support for forks is a band-aid.

You can actually abuse the vendor system to solve this problem. Just put your subpackages into vendor, and you've got project-relative imports, you can fork the package easily without having to rewrite everything. Obviously that's an ugly hack, but I'd like to see something like that as a solution to this problem, maybe not full relative imports.

@dlsniper
Copy link
Contributor

dlsniper commented Mar 2, 2018

I agree with @cch123 forking packages with subpackages in them is a huge pain.
Changing origins is not a solution (you can't go get them). Dep's support for forks is a band-aid.

@nkovacs this is not a band-aid, nor would relative import paths solve anything in this case. dep allows something that's considered basic functionality for package managers. There is nothing new or different for Go compared to other languages.

Just put your subpackages into vendor, and you've got project-relative imports, [...] Obviously that's an ugly hack

This is not why we have the vendor directory. We have it for reproducible builds. It allows guaranteed that, if committed, everyone gets the same identical source code regardless of environment.

but I'd like to see something like that as a solution to this problem, maybe not full relative imports.

I don't understand how relative imports solve any problem that you have.

@nkovacs
Copy link

nkovacs commented Mar 2, 2018

If I fork a php package, I can publish it on packagist without first having to change a bunch of imports inside it.
If I fork a go package with subpackages, I have to change all the imports to its subpackages in it if I want people to go get it, or govendor fetch it, or dep ensure it, or whatever.

Both composer and npm allow you to specify a temporary fork instead of the canonical package. But in both ecosystems you can just publish your forked package without having to rewrite a bunch of imports inside it, because they're all relative. You can't do that in go.

This is not why we have the vendor directory. We have it for reproducible builds. It allows guaranteed that, if committed, everyone gets the same identical source code regardless of environment.

Of course not. I said you can abuse it to fix the problem.

I don't understand how relative imports solve any problem that you have.

I wouldn't have to do this: nkovacs/ifacemaker@c327e8d
Or this: nkovacs/counterfeiter@d0d3dfc

@dominikh
Copy link
Member

dominikh commented Mar 2, 2018

If I fork a go package with subpackages, I have to change all the imports to its subpackages in it if I want people to go get it, or govendor fetch it, or dep ensure it, or whatever.

Which, compared to the effort of forking, modifying and maintaining a project, takes absolutely zero effort.

@dlsniper
Copy link
Contributor

dlsniper commented Mar 2, 2018

But in both ecosystems you can just publish your forked package without having to rewrite a bunch of imports inside it, because they're all relative. You can't do that in go.

To give you the counter-point for the PHP example. You would have to override the autoload process to tell it that your project provides now the reference for the namespace you want to use, rather than the original package, see https://github.com/Seldaek/monolog/blob/master/composer.json#L46-L48

Now Go being different than PHP is is allowed to behave in a different way. If you ask me, instructing dep to fetch the source from a different place is a lot more cleaner and obvious than digging in the package.json and figuring out where the autoloader will perform the operation for loading certain package.

And even in PHP or in Go world, allowing relative import paths suffer greatly when introducing symlinks into play (which at least for me in the PHP world have been a source of needlessly spent hours of debugging why something doesn't work in a particular setup of a developer / environment or in bug reports that I had to handle).

As far as I can tell, the whole argument is: please allow relative import paths so we don't have to run a search & replace operation for forks we want to use as main source but we don't want to use dep and we don't want to have a commit for that operation and we'll never backport changes to upstream (due to the import path change). Would this be an accurate depiction of the case?

@myitcv
Copy link
Member

myitcv commented Apr 27, 2018

Relative imports barely work today. They don't work inside of GOPATH. They only really work when compiling individual files. I think this proposal is about allowing relative imports inside of GOPATH.

👍 - that (a single file via go run) happens to be exactly the scenario I "tested" to see to what extent things worked today. Indeed they don't work within GOPATH - thanks, I hadn't realised that distinction.

Title is fine as is therefore. Apologies for the noise.

@metakeule
Copy link

Since the functionality is already available via the vendor directory, it would be sufficient to allow the import of vendor/internal/...'. As of Go2 the vendor` directory could then be renamed to something more matching (maybe just 'r' for 'relative').

@nyetwurk

This comment has been minimized.

@dominikh

This comment has been minimized.

@golang golang locked as too heated and limited conversation to collaborators Sep 9, 2018
@davecheney

This comment has been minimized.

@bcmills

This comment has been minimized.

@sebastianhaberey
Copy link

I, too, ran into the issues pointed out by @johnDoe2018 and @cch123 recently. I tried the proposed solution involving dep ensure, but I could not get it to work for internal packages. I tend to agree with @cch123 that this dep functionality is a bit of a band-aid. From the documentation:

source rules are generally brittle and should only be used when there is no other recourse.

I am convinced that there are good reasons to completely remove relative paths from Go and so I'll leave that discussion to people more familiar with Go. On the other hand I think that forking repositories, creating pull requests and validating them on CI systems are part of daily developer life. The current need to adress packages inside my own project by their full repository path seems to conflict with that workflow.

@gopherbot gopherbot added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Sep 3, 2019
@tv42
Copy link

tv42 commented Nov 26, 2019

The underlying problem seems to be solved by go modules and their support for "github-forking etcd", with a git clone fetching whatever clone you want, while avoiding all the downsides of relative imports. And writing code using forked libraries is supported with replace directive in go.mod.

@surajbarkale
Copy link

How about treating import paths that start with / to be relative to the root of module? So the following code blocks are equivallant?

	"github.com/coreos/etcd/client"
	"github.com/coreos/etcd/pkg/transport"
	"github.com/coreos/etcd/pkg/types"

With module relative import

	"/client"
	"/pkg/transport"
	"/pkg/types"

Since the module name is present from go.mod, this will not require any significant changes to the import machinery.

rjarry added a commit to rjarry/aerc that referenced this issue Nov 14, 2021
I'm not sure what are the implications but it seems required.

Link: golang/go#20883
Signed-off-by: Robin Jarry <robin@jarry.cc>
@wadeperrigo
Copy link

I would say relative paths are extremely important whenever developing an application. If my understanding is correct from this conversation any "module" I create needs to be moved to the go PATH rather than simply referencing it from "./myModule" or "./myModule.go". This seems extremely inconvenient and while not being able to auto-fork or pull down from a github repository as my goal, but rather just use modules I make locally - it seems relative pathing would make life so much easier. (IE: Look at JS)

@ianlancetaylor
Copy link
Contributor

With our current module ecosystem, our use of replace directives in go.mod files, the introduction of go.work files, the existence of the gomvpkg command, we're at a point where it's clear that are not going to start supporting relative imports. Closing this issue.

@awy
Copy link

awy commented Jan 3, 2024

It feels to me as though the proposal to use import paths starting with slash as same-module imports would be amenable to Go 1, at the very least by providing a suitable directive in go.mod if necessary. I don't see why this needs to be deferred to Go 2.

One would then have 3 styles of import:

  • import pkg // standard Go library
  • import domain/path/module/pkg // external module
  • import /pkg // relative to root of current module

The module fork-then-PR use-case (or even fork without PRs) is the most persuading of the need for such functionality.

Renaming or relocating a module may be unusual in some patterns of use but overall, not sufficiently unusual that the pain need not be alleviated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. Proposal v2 A language change or incompatible library change
Projects
None yet
Development

No branches or pull requests