Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/wiki,cmd/go: document how to fetch private repos with Yubikey SSH authentication #49515

Open
ckcr4lyf opened this issue Nov 11, 2021 · 9 comments
Open

x/wiki,cmd/go: document how to fetch private repos with Yubikey SSH authentication #49515

ckcr4lyf opened this issue Nov 11, 2021 · 9 comments
Labels
Documentation help wanted modules NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Unreleased

Comments

@ckcr4lyf
Copy link

ckcr4lyf commented Nov 11, 2021 •
edited

From the changelog of Go 1.17, I see:

Password prompts

The go command by default now suppresses SSH password prompts and Git Credential Manager prompts when fetching Git repositories using SSH, as it already did previously for other Git password prompts. Users authenticating to private Git repos with password-protected SSH may configure an ssh-agent to enable the go command to use password-protected SSH keys.

I use private repos with SSH and have correctly set up my .netrc and all. I use a yubikey with PIN for SSH authentication.

Go 1.16 behavior

I try and get a lib / run go mod vendor, the git client will prompt me for my yubikey PIN in order to authenticate with the remote server, once I enter it correctly, it all works well. (EDIT: Probably the SSH client which git calls, but either way, it worked fine)

go mod vendor
Enter PIN for 'PIV Card Holder pin (PIV_II)': 
Enter PIN for 'PIV Card Holder pin (PIV_II)': 
Enter PIN for 'PIV Card Holder pin (PIV_II)': 
go: downloading gitlab.com/[redacted]
go: downloading gitlab.com/[redacted]
go: downloading gitlab.com/[redacted]

Note: There are multiple prompts, I enter my pin for each one.

Go 1.17 behavior

I try and run go mod vendor or something, and it will fail with this:

go: gitlab.com/[redacted]: invalid version: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /home/raghu/go/pkg/mod/cache/vcs/[redacted]: exit status 128:
	git@gitlab.com: Permission denied (publickey,keyboard-interactive).
	fatal: Could not read from remote repository.
	
	Please make sure you have the correct access rights
	and the repository exists.

I am 99% sure this is due to the change in Go 1.17 to disable "password prompts", since if I revert to Go 1.16 all works as expected.

I am able to use my Yubikey w/ PIN prompt for Git and other SSH stuff just fine. The changelog suggests using an ssh-agent for password-protected keys, but in my case, the key resides on my Yubikey, and I need to enter the pin in order to perform cryptographic operations with it. Since the key is not accessible, I cannot cache it within SSH agent. (Additionally, I do not want to cache my yubikey pin).

This change introduces in Go 1.17 seems like a nerf to security, since for SSH repos, it seems to force you to have an SSH key which is cached by ssh-agent.

If there is no way to disable this with a flag on my end or something, I think it should be reverted for Go 1.18, as @bcmills suggested it is being "given a try" in Go 1.17 - #44904 (comment)
@ckcr4lyf
Copy link
Author

FYI: I currently have bypassed this by setting by env var

GIT_SSH_COMMAND="ssh -o ControlMaster=no"

as hinted by the source here -> https://github.com/golang/go/blob/master/src/cmd/go/internal/get/get.go#L156

I think it should at least be added to the docs (or just changelog) to make it easier

@bcmills bcmills changed the title Go 1.17 - Unable to fetch private repo with Yubikey SSH authentication cmd/go: unable to fetch private repo with Yubikey SSH authentication Nov 12, 2021
@bcmills bcmills added modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Nov 12, 2021
@bcmills
Copy link
Contributor

bcmills commented Nov 12, 2021

This change introduces in Go 1.17 seems like a nerf to security, since for SSH repos, it seems to force you to have an SSH key which is cached by ssh-agent.

The go command intentionally disables password prompts because it fetches repositories in parallel — there isn't really a controlled way for you to see which repo is asking for your credential. (That is admittedly more of a concern when PasswordAuthentication is enabled in your ssh client configuration, but we still don't want to train users to get in the habit of entering passwords without a clear destination.)

The long-term solution for Yubikey authentication in particular might be to use the ssh ControlMaster setting. You could establish the ssh session prior to running the go command, and then use that connection to fetch any needed repos, and finally tear it down again. Unfortunately, we also currently disable ControlMaster for #13453 — but that may be feasible to fix a different way.

@ckcr4lyf
Copy link
Author

there isn't really a controlled way for you to see which repo is asking for your credential.

That's a great point. Since all the private repos I'm using are based on the same auth, the thought had never crossed my mind, but I can see where this change came from.

I think using the environment variable is a decent workaround for now, since it allows for using Yubikey SSH auth w/ Go 1.17

@ckcr4lyf ckcr4lyf mentioned this issue Nov 15, 2021
Closed
@bcmills
Copy link
Contributor

bcmills commented Nov 15, 2021 •
edited

I did a bit more reading, and here's my suggestion (based on https://ldpreload.com/blog/ssh-control):

  1. Explicitly set GOPRIVATE for your private module path(s) (as I assume you have already done).
  2. You may also need to set a Git insteadOf rule in ~/.gitconfig as described in cmd/go: private repos on hard-coded hosting providers require special configuration #26134 (comment).
    • (Since gitlab.com is not a hard-coded hosting provider, that might not be necessary.)
  3. Add ControlPath ~/.ssh/control-%C to your .ssh/config file.
  4. Run ssh -M -N -f git@gitlab.com. That should prompt you for Yubikey authentication to GitLab, then return you the shell prompt (leaving the SSH connection open in the background).
  5. Run whatever go command(s) you need to download the needed module contents.
  6. Finally, run ssh -O exit git@gitlab.com to terminate the open SSH connection.

If you don't want to adjust your global .ssh/config, you could instead do:

  1. In the shell in which you will run the go command: 
    export GIT_SSH_COMMAND='ssh -o ControlPath=~/.ssh/control-%C -o BatchMode=yes`
  2. Run ssh -o ControlPath=~/.ssh/control-%C -M -N -f git@gitlab.com to prompt for authentication.
  3. Run whatever go command(s) you need to download the needed module contents.
  4. Finally, run ssh -o ControlPath=~/.ssh/control-%C -O exit git@gitlab.com to terminate the open SSH connection.

If you don't want to leave a connection open in the background indefinitely if you forget step (5), you could omit the -f flag to the ssh command and instead set the ControlPersist option to a reasonably short timeout.

Or:

  1. In a different terminal, run ssh -M -N git@gitlab.com.
  2. Run whatever go command(s) you need to download the needed module contents.
  3. In the terminal running the ssh command from step (4), press ⌃C to terminate the open session.

@bcmills
Copy link
Contributor

bcmills commented Nov 15, 2021

@ckcr4lyf, please give the above a try and let me know if it works. (If so, perhaps I can add some more formal documentation on this configuration.)

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Nov 15, 2021
@bcmills bcmills added this to the Backlog milestone Nov 15, 2021
@ckcr4lyf
Copy link
Author

@bcmills , thanks so much for the response. I can confirm it works perfectly!

I tested it with the latter option of a "temporary setup", and was able to fetch multiple private repos (for the same SSH auth) with no issues (including prompts). I tried the "permanent" solution w/ SSH config, and that also works great!

I believe this is a viable solution, since it allows for secure SSH-auth (e.g. via Yubikey + pin) explicitly, which can further be terminated once the go operations are complete.

Cheers, and thanks for the help (not sure if I should close the issue or not)

@bcmills bcmills added Documentation and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Nov 18, 2021
@bcmills bcmills changed the title cmd/go: unable to fetch private repo with Yubikey SSH authentication cmd/go: document how to fetch private repos with Yubikey SSH authentication Nov 18, 2021
@bcmills
Copy link
Contributor

bcmills commented Nov 18, 2021

Thanks for confirming! I'll leave this issue open for documentation — we certainly do want to support private repos that require two-factor authentication, and this setup and workflow is not trivial to figure out.

@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Nov 18, 2021
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Nov 18, 2021
@bcmills bcmills mentioned this issue Dec 14, 2021
Open
@Gerst20051
Copy link

FYI: I currently have bypassed this by setting by env var

GIT_SSH_COMMAND="ssh -o ControlMaster=no"

as hinted by the source here -> https://github.com/golang/go/blob/master/src/cmd/go/internal/get/get.go#L156

I think it should at least be added to the docs (or just changelog) to make it easier

here is a permalink to that line so it doesn't get lost when that file gets updated

go/src/cmd/go/internal/get/get.go

Line 156 in 0f3becf

os.Setenv("GIT_SSH_COMMAND", "ssh -o ControlMaster=no -o BatchMode=yes")

@bcmills bcmills mentioned this issue May 18, 2022
Closed
@bcmills bcmills self-assigned this Jan 19, 2023
@bcmills bcmills modified the milestones: Backlog, Go1.21 Jan 19, 2023
@gopherbot gopherbot modified the milestones: Go1.21, Go1.22 Aug 8, 2023
@bcmills bcmills modified the milestones: Go1.22, Unreleased Feb 1, 2024
@bcmills bcmills changed the title cmd/go: document how to fetch private repos with Yubikey SSH authentication x/wiki: document how to fetch private repos with Yubikey SSH authentication Feb 1, 2024
@bcmills bcmills changed the title x/wiki: document how to fetch private repos with Yubikey SSH authentication x/wiki,cmd/go: document how to fetch private repos with Yubikey SSH authentication Feb 1, 2024
@bcmills
Copy link
Contributor

bcmills commented Mar 15, 2024

@matloob, @samthanawalla: this issue should be a pretty straightforward writeup of the options discussed in #49515 (comment).

@bcmills bcmills removed their assignment Mar 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Documentation help wanted modules NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
4 participants
@Gerst20051 @bcmills @ckcr4lyf @gopherbot