Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: support self-hosted fossil in 'go get' #25811

Closed
schollz opened this issue Jun 10, 2018 · 15 comments
Closed

cmd/go: support self-hosted fossil in 'go get' #25811

schollz opened this issue Jun 10, 2018 · 15 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@schollz
Copy link

schollz commented Jun 10, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.10.3 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/zns/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/zns/go"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build451328689=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I am self-hosting fossils using a standalone server. I have a very simple "hello, world" example here: https://fossil.schollz.com/hello-world which I am trying to go get. This is a functional fossil, as you can clone and run it:

> fossil clone https://fossil.schollz.com/hello-world hello-world.fossil
> fossil open hello-world.fossil
> go run main.go
hello, world

I then tried to go get this fossil.

What did you expect to see?

I expected to be able to run

> go get fossil.schollz.com/hello-world.fossil

which would automatically download hello-world as a fossil into my $GOPATH.

What did you see instead?

Despite it being run using the canonical fossil server command, I am unable to go get it. Here are the attempts:

>  go get fossil.schollz.com/hello-world
package fossil.schollz.com/hello-world: unrecognized import path "fossil.schollz.com/hello-world" 
(parse https://fossil.schollz.com/hello-world?go-get=1: no go-import meta tags ())
> go get fossil.schollz.com/hello-world.fossil
package fossil.schollz.com/hello-world.fossil: cannot download, fossil.schollz.com/hello-world 
uses insecure protocol

It seems that the current fossil support only supports chiselapp.com - I refer to this change that added support.

Is there not yet support for self-hosted fossils?

@AlexRouSg
Copy link
Contributor

The correct command is go get -insecure fossil.schollz.com/hello-world.fossil
Your host is defaulting to http and go get doesn't like that so to force it to use http you need to pass the -insecure flag

@schollz
Copy link
Author

schollz commented Jun 11, 2018

Thanks for your help. It actually still doesn't work for me though.

> go get -insecure fossil.schollz.com/hello-world.fossil
can't load package: package fossil.schollz.com/hello-world.fossil: 
no Go files in /home/zns/go/src/fossil.schollz.com/hello-world.fossil

I don't think it's processing the Go files correctly, as there is a go file in the main branch at the root folder.

But it does seem to create a directory, although there is also nothing in it (including the fossil repo):

> tree $GOPATH/src/fossil.schollz.com
/home/zns/go/src/fossil.schollz.com
└── hello-world.fossil

1 directory, 0 files

@AlexRouSg
Copy link
Contributor

Using raw fossil commands results in errors, is the repo setup correctly and can you clone manually with fossil?

@AlexRouSg
Copy link
Contributor

AlexRouSg commented Jun 11, 2018

Sorry, I haven't used fossil before and was using the wrong url. Looks like go get is too.

cc @ksshannon @ianlancetaylor

go get is calling fossil clone fossil.schollz.com/hello-world however the correct usage would be fossil clone http://fossil.schollz.com/hello-world or fossil clone https://fossil.schollz.com/hello-world

@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 11, 2018
@bcmills bcmills added this to the Go1.12 milestone Jun 11, 2018
@ksshannon
Copy link
Contributor

@schollz, there could be a few thinks going on here. If you have a public URL I can test against, it'd probably be easier. You should set the tag to something like:

<meta name="go-import" content="fossil.schollz.com/hello-world fossil https://fossil.schollz.com">

or something similar.

This is done a little differently depending on if your fossil is before or after the drafts enhancement was implemented.

I'd be glad to dig into it, but I need to be able to replicate. For posterity, please report your fossil version as well. We'll get it figured out.

@AlexRouSg
Copy link
Contributor

AlexRouSg commented Jun 12, 2018

@ksshannon The URL is public btw, ohh and I tried 2.6 direct download from https://www.fossil-scm.org/xfer/uv/download.html

@ksshannon
Copy link
Contributor

@AlexRouSg, I believe it's the head metadata. It works the same with git hosting.

@AlexRouSg
Copy link
Contributor

@ksshannon The metadata error would explain the error with go get -insecure fossil.schollz.com/hello-world, however go get -insecure fossil.schollz.com/hello-world.fossil doesn't complain about metadata and progress until fossil errors out.

@AlexRouSg
Copy link
Contributor

AlexRouSg commented Jun 12, 2018

go get -insecure -v fossil.schollz.com/hello-world.fossil
# cd .; fossil 
Usage: fossil COMMAND ...
   or: fossil help           -- for a list of common commands
   or: fossil help COMMAND   -- for help with the named command

Commands and filenames may be passed on to fossil from a file
by using:

    fossil --args FILENAME ...

Each line of the file is assumed to be a filename unless it starts
with '-' and contains a space, in which case it is assumed to be
another flag and is treated as such. --args FILENAME may be used
in conjunction with any other flags.
# cd .; fossil 
Usage: fossil COMMAND ...
   or: fossil help           -- for a list of common commands
   or: fossil help COMMAND   -- for help with the named command

Commands and filenames may be passed on to fossil from a file
by using:

    fossil --args FILENAME ...

Each line of the file is assumed to be a filename unless it starts
with '-' and contains a space, in which case it is assumed to be
another flag and is treated as such. --args FILENAME may be used
in conjunction with any other flags.
fossil.schollz.com/hello-world.fossil (download)
# cd .; fossil clone fossil.schollz.com/hello-world /home/asdf/go/src/fossil.schollz.com/hello-world.fossil/.fossil
unknown repository: fossil.schollz.com/hello-world
package fossil.schollz.com/hello-world.fossil: exit status 1

@ksshannon
Copy link
Contributor

@AlexRouSg go get -insecure fossil.schollz.com/hello-world.fossil is a 404, same as wget http://fossil.sholiz.com/hello-world.fossil so I may need to clean up some of the std out dump, but I think that is a different issue. I'll look further if the metadata tag doesn't solve this issue, and then we'll move on.

@AlexRouSg
Copy link
Contributor

AlexRouSg commented Jun 12, 2018

@ksshannon Yes it's a 404, but looking at the -v output, it calls fossil clone fossil.schollz.com/hello-world so go get is stripping the .fossil.

This is going into undocumented territory ... someone should clarify whether go get myurl.com/repo.vcs needs to actually exist or if it's just a way to denote vcs and the url used is myurl.com/repo

If it's the latter then the issue is fossil clone myurl.com/repo is not a valid fossil command, it requires the protocol part as well and should instead run fossil clone http://myurl.com/repo or fossil clone https://myurl.com/repo

@ksshannon
Copy link
Contributor

@AlexRouSg No offense, but I'd like to solve @schollz 's issue first. We can file more issues after if we need to fix fossil support. I'm sure there are issues, but I'm not sure they should fall under this one.

@schollz
Copy link
Author

schollz commented Jun 12, 2018

@ksshannon Thanks for helping too.

> fossil version
This is fossil version 2.6 [9718f3b078] 2018-05-04 12:56:42 UTC

The URL (https://fossil.schollz.com/hello-world) is indeed live and a working fossil. You can try it yourself with:

> fossil clone https://fossil.schollz.com/hello-world hello-world.fossil
> fossil open hello-world.fossil
> go run main.go
hello, world

Let me know if I can provide any other information, or if you can point me in the right direction I can try digging too. I'm not so familiar with the Go source yet though.

@ksshannon
Copy link
Contributor

@schollz read the docs at:

https://golang.org/cmd/go/#hdr-Remote_import_paths

And set the meta tag it mentions for you repository. You should be able to add it at:

https://fossil.schollz.com/hello-world/setup_skinedit

@schollz
Copy link
Author

schollz commented Jun 13, 2018

@ksshannon Awesome. It works.

Here's the header I added:

<meta name="go-import" content="fossil.schollz.com/hello-world fossil https://fossil.schollz.com/hello-world">

It is now go get-able:

> go get -u -v fossil.schollz.com/hello-world
Fetching https://fossil.schollz.com/hello-world?go-get=1
Parsing meta tags from https://fossil.schollz.com/hello-world?go-get=1 (status code 200)
get "fossil.schollz.com/hello-world": found meta tag get.metaImport{Prefix:"fossil.schollz.com/hello-world", VCS:"fossil", RepoRoot:"https://fossil.schollz.com/hello-world"} at https://fossil.schollz.com/hello-world?go-get=1
fossil.schollz.com/hello-world (download)
> tree $GOPATH/src/fossil.schollz.com
/home/zns/go/src/fossil.schollz.com
hello-world
  main.go
  README.md

1 directory, 2 files

Thanks for your help @ksshannon and @AlexRouSg

@schollz schollz closed this as completed Jun 13, 2018
@golang golang locked and limited conversation to collaborators Jun 13, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

5 participants