Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/crypto/acme: Client.FetchCert hangs even with timeout context #38790

Closed
gholt opened this issue May 1, 2020 · 3 comments
Closed

x/crypto/acme: Client.FetchCert hangs even with timeout context #38790

gholt opened this issue May 1, 2020 · 3 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Unreleased

Comments

@gholt
Copy link

gholt commented May 1, 2020

What version of Go are you using (go version)?

$ go version
go version go1.13.6 linux/amd64

Does this issue reproduce with the latest release?

Unknown as I'm unable to run go 1.14 right this moment. This shouldn't have anything to do with Go itself though, it's about the acme library.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="redacted/.cache/go-build"
GOENV="redacted/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY="internal stuff redacted"
GONOSUMDB="internal stuff redacted"
GOOS="linux"
GOPATH="redacted/go"
GOPRIVATE="internal stuff redacted"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="redacted/play/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build704151629=/tmp/go-build -gno-record-gcc-switches"

What did you do?

package main

import (
        "context"
        "time"

        "golang.org/x/crypto/acme"
)

func main() {
        ctx, cancel := context.WithTimeout(context.Background(), time.Second)
        defer cancel()
        (&acme.Client{}).FetchCert(ctx, "garbage", true)
}

What did you expect to see?

Program to exit at some point, preferably with an error due to the garbage url given.

What did you see instead?

Program never exits.

Additional Information

It seems that the acme code is buggy around its use of the cacheMu mutex. https://github.com/golang/crypto/blob/master/acme/acme.go#L143 will grab the lock, but can call getRegRFC which can then call post, etc. until it tries to grab the lock again, and deadlocks.
@gholt gholt changed the title Deadlock in acme library x/crypto: Deadlock in acme library May 1, 2020
@gopherbot gopherbot added this to the Unreleased milestone May 1, 2020
@andybons andybons changed the title x/crypto: Deadlock in acme library x/crypto/acme: Client.FetchCert hangs even with timeout context May 1, 2020
@andybons andybons added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label May 1, 2020
@andybons
Copy link
Member

andybons commented May 1, 2020

@FiloSottile @katiehockman

@rolandshoemaker
Copy link
Member

Looks like a combination of a deadlock and an infinite loop. FetchCert causes the following execution chain FetchCert -> fetchCertRFC -> postAsGet -> post -> postNoRetry -> accountKID -> getRegRFC -> post -> postNoRetry -> accountKID and then deadlocks waiting for cacheMu to become available. If you fix the locking you just get an infinite loop of accountKID -> getRegRFC -> post -> postNoRetry -> accountKID.

This happens because both c.kid and c.Key are not populated, which makes it impossible to lookup the KID (or do anything else really). Fix seems to be to just check for c.Key == nil in postNoRetry if key == nil.

@gopherbot
Copy link

Change https://golang.org/cl/233164 mentions this issue: x/crypto/acme: fix deadlock when Client.Key is nil

@golang golang locked and limited conversation to collaborators Aug 17, 2022
LewiGoddard pushed a commit to LewiGoddard/crypto that referenced this issue Feb 16, 2023
@rolandshoemaker
acme: fix deadlock when Client.Key is nil
e1ab2f2 
When methods that use POSTs are called on a acme.Client which has a
nil Key field it will cause a deadlock due to an infinite loop in
the code that looks up the account KID. This change adds a check for
the key being nil, and errors out if that is the case. Also adds a
test for this behavior.

Fixes golang/go#38790

Change-Id: I65ff6bbbade7ed2d85306895904a976089730bbf
Reviewed-on: https://go-review.googlesource.com/c/crypto/+/233164
Trust: Roland Shoemaker <roland@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
BiiChris pushed a commit to BiiChris/crypto that referenced this issue Sep 15, 2023
@rolandshoemaker
acme: fix deadlock when Client.Key is nil
6b84722 
When methods that use POSTs are called on a acme.Client which has a
nil Key field it will cause a deadlock due to an infinite loop in
the code that looks up the account KID. This change adds a check for
the key being nil, and errors out if that is the case. Also adds a
test for this behavior.

Fixes golang/go#38790

Change-Id: I65ff6bbbade7ed2d85306895904a976089730bbf
Reviewed-on: https://go-review.googlesource.com/c/crypto/+/233164
Trust: Roland Shoemaker <roland@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
4 participants
@andybons @gholt @rolandshoemaker @gopherbot