Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/crypto/ssh: concurrent call ssh.Dial will fail #27140

Open
wangwd1991 opened this issue Aug 22, 2018 · 16 comments
Open

x/crypto/ssh: concurrent call ssh.Dial will fail #27140

wangwd1991 opened this issue Aug 22, 2018 · 16 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@wangwd1991
Copy link

wangwd1991 commented Aug 22, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.8.3

Does this issue reproduce with the latest release?

https://github.com/golang/crypto.git
commit : 614d502

What operating system and processor architecture are you using (go env)?

centos 7

What did you do?

auth := make([]ssh.AuthMethod, 0)
auth = append(auth, ssh.Password("123456"))
config := &ssh.ClientConfig {
	User: "root",
	Auth: auth,
	HostKeyCallback: func(hostname string, remote net.Addr, key ssh.PublicKey) error {
		return nil
	},
	Timeout: 30 * time.Second,
}
for i:=0;i<2;i++ {
	go func() {
		client, err := ssh.Dial("tcp", "ip:port", config)
		fmt.Println(err) // the 2th thread (maybe 3th or other litter value) will error
		time.Sleep(5 * time.Second)
		// error is not here, it just a test code
		client.Close()
	}()
}

What did you expect to see?

run pass

What did you see instead?

ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain

the second thread error , other machines maybe 3 or ...

@meirf meirf changed the title concurrent call ssh.Dial will fail x/crypto/ssh: concurrent call ssh.Dial will fail Aug 23, 2018
@gopherbot gopherbot added this to the Unreleased milestone Aug 23, 2018
@meirf
Copy link
Contributor

meirf commented Aug 23, 2018

@wangwd1991:

  • does this help you?
  • can you elaborate what you mean by "the second thread error , other machines maybe 3 or ..." ? please show the exact output.

@wangwd1991
Copy link
Author

@meirf
Thanks for your fast response.

1、That doesn't help me. The single thread, it works fine.

2、Sorry to unclear description.
“the second thread error , other machines maybe 3 or ..” means the cycle of the 'for' loop.
Using different number to reproduce the problem with different addr (ssh server)

  **for i:=0;i<2;i++** { //maybe 3 for dialing other server
     go func() {

Or. Should I use a 'lock' to call ssh.Dial then create ssh session to avoid this problem?

@crvv
Copy link
Contributor

crvv commented Aug 23, 2018

I can't reproduce this. The code is

package main

import (
	"fmt"
	"net"
	"time"

	"golang.org/x/crypto/ssh"
)

func main() {
	auth := []ssh.AuthMethod{ssh.Password("my password")}
	config := &ssh.ClientConfig{
		User: "my username",
		Auth: auth,
		HostKeyCallback: func(hostname string, remote net.Addr, key ssh.PublicKey) error {
			return nil
		},
		Timeout: 30 * time.Second,
	}
	for i := 0; i < 2; i++ {
		i := i
		go func() {
			client, err := ssh.Dial("tcp", "localhost:22", config)
			if err != nil {
				fmt.Println(i, err) // the 2th thread (maybe 3th or other litter value) will error
			} else {
				fmt.Println(i, "success")
				client.Close()
			}
		}()
	}
	select {}
}

And I got

1 success
0 success

If you can reproduce your problem with my code, please paste the exact output.

@wangwd1991
Copy link
Author

@crvv
I run with the result:
1 success
0 ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain

I think you can try with for i := 0; i < 3; i ++ { // with 3 or more.

@wangwd1991
Copy link
Author

And My server /etc/ssh/sshd_config

MaxSession 10
PasswordAuthentication yes

@agnivade
Copy link
Contributor

/cc @hanwen

@hanwen
Copy link
Contributor

hanwen commented Aug 23, 2018

code looks OK to me. You could try to run with the race detector, but can you confirm it fails with all types of SSH servers?

@wangwd1991
Copy link
Author

@hamaxx
I increase the number of loop , then go run -race main.go ? :

[root@my-server ssh]# go run -race main.go
15 ssh: handshake failed: read tcp ip1:port1->ip2:port2: read: connection reset by peer
17 ssh: handshake failed: EOF
19 ssh: handshake failed: EOF
9 success
0 success
6 ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain
1 ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain
4 ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain
11 ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain
2 ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain
8 ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain
7 ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain
18 ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain
16 ssh: handshake failed: ssh: unable to authenticate, attempted methods [password none], no supported methods remain
3 ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain
10 ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain
5 ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain
13 ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain
14 ssh: handshake failed: ssh: unable to authenticate, attempted methods [password none], no supported methods remain
12 ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain

My env:
my ssh is OpenSSH_6.6.1p1
Server : a VM CentOS Linux release 7.1.1503 (Core).
a Physical Machine CentOS Linux release 7.2.1511 (Core).

What I guess is that, client send the ssh package to the server quickly, the server will refuse or discard the package?

@hanwen
Copy link
Contributor

hanwen commented Aug 23, 2018

probably. Maybe there is rate limiting for password logins to avoid brute-force attacks?

@wangwd1991
Copy link
Author

@hanwen
Thanks for your reply.

Although, I should use it with thread safe.

Closing this.

@songtianyi
Copy link

songtianyi commented Jun 21, 2019

Same problem #32705 when connecting a firewall device.
When connecting the firewall device with a single thread, no matter the frequency you connect, there is only a small probability that error happens. BUT when connect concurrently, the error will happen for sure!

@katiehockman
Copy link
Contributor

/cc @hanwen re-opened this since another user had an issue. Can you PTAL?

@songtianyi
Copy link

songtianyi commented Jun 24, 2019

auth fail but returned nothing from server side
See client_auth.go#L350

@rustyx
Copy link

rustyx commented Jan 24, 2020

@songtianyi are you saying that client_auth.go#L350 is causing this issue, or just commenting?

@songtianyi
Copy link

songtianyi commented Jan 27, 2020

@songtianyi are you saying that client_auth.go#L350 is causing this issue, or just commenting?

Nope. What i'm pointing out here is the code client_auth.go#L350 does not return any error message when return auth_failed.

@mentaLwz
Copy link

Does this problem have been solved now? Or any other methods to deal with this?

pjbgf pushed a commit to pjbgf/source-controller that referenced this issue Mar 25, 2022
The underlying SSH connections are kept open and are reused
across several SSH sessions. This is due to upstream issues in
which concurrent/parallel SSH connections may lead to instability.

golang/go#51926
golang/go#27140
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
pjbgf pushed a commit to pjbgf/source-controller that referenced this issue Mar 25, 2022
The underlying SSH connections are kept open and are reused
across several SSH sessions. This is due to upstream issues in
which concurrent/parallel SSH connections may lead to instability.

golang/go#51926
golang/go#27140
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
pjbgf pushed a commit to pjbgf/source-controller that referenced this issue Mar 25, 2022
The underlying SSH connections are kept open and are reused
across several SSH sessions. This is due to upstream issues in
which concurrent/parallel SSH connections may lead to instability.

golang/go#51926
golang/go#27140
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
pjbgf pushed a commit to pjbgf/source-controller that referenced this issue Mar 25, 2022
The underlying SSH connections are kept open and are reused
across several SSH sessions. This is due to upstream issues in
which concurrent/parallel SSH connections may lead to instability.

golang/go#51926
golang/go#27140
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
pjbgf pushed a commit to pjbgf/source-controller that referenced this issue Mar 25, 2022
The underlying SSH connections are kept open and are reused
across several SSH sessions. This is due to upstream issues in
which concurrent/parallel SSH connections may lead to instability.

golang/go#51926
golang/go#27140
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
pjbgf pushed a commit to pjbgf/source-controller that referenced this issue Mar 28, 2022
The underlying SSH connections are kept open and are reused
across several SSH sessions. This is due to upstream issues in
which concurrent/parallel SSH connections may lead to instability.

golang/go#51926
golang/go#27140
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
@seankhliao seankhliao added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests