New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
x/crypto/acme: CreateOrderCert returning Order's status ('valid') is not acceptable for finalization #42278
Comments
The problem with ignoring this error with something like:
is that we don't have the order to fetch the certificate. |
CC @rolandshoemaker, @x1ddos, @FiloSottile per owners. |
Let's Encrypt will only set the status of an order to 'valid' if the certificate has already been issued. It's plausible if you are doing parallel validations of identical name sets you may be running into orders that are being reused at the server side. Are you checking if Order.CertURL is not empty if WaitOrder returns an order with a 'valid' status? |
Oh sorry I misread the initial issue. This seems like expected behavior, if WaitOrder returns a valid order you should be using FetchCert rather than CreateOrderCert. |
@rolandshoemaker I'm not doing parallel validations or fetches. I'm doing validations and then calling Also, to clarify, this seems like an edge case, we've been using the same code in production for close to a year and just ran into this problem last week a few times and it has since stopped happening. |
CreateOrderCert doesn't do anything fancy, all it does is POST the finalization request, poll for the order status to change to 'valid', and fetch the resulting certificate. You will only get the 'wrong status' error if a. you've already called CreateOrderCert for the finalization URL before, or b. there is an issue on Let's Encrypt's end in computing order statuses. Note that if you are creating two orders with the same set of names Let's Encrypt will reuse the order which would cause this exact problem (this is regardless of the CSR you use in CreateOrderCert). For example if you try to create two orders for ["example.com", "b.example.com"] before calling CreateOrderCert on the first order then Let's Encrypt will return the same order object, so you'll only be able to call CreateOrderCert once, the second time will return the error you are seeing as the order will have already been finalized. |
I see inside of the POST for the finalization request that it has some retry logic. Is there any way I can debug what happened in there for future occurrences? |
Oh hrm, I guess another possible explanation here is that LE is timing out internally, causing a 500 status code to be returned to the client but still actually completing the issuance. Since we receive a 500 we retry the call, resulting in the unexpected status error because the finalization actually happened despite us never seeing it happen. I'm still not sure adding a case to ignore the error and refetch the order is the correct solution to this problem, since it introduces a somewhat complex codepath that relies on matching against a textual error that is specific to Let's Encrypt's implementation. I think I'm still of the opinion this is better handled in user applications. |
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
The build that is seeing the error is using:
and I don't see any fixes related to acme since that.
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
After calling
WaitOrder
for the order to be ready and authorizations to succeed, we callCreateOrderCert
with the order'sFinalizeURL
and a freshly created CSR. We only callCreateOrderCert
once.What did you expect to see?
I expect it to ignore this error when calling the finalize URL and continue to fetch the certificate since the order is Valid.
What did you see instead?
CreateOrderCert
is returning:I imagine this is because of some retry logic that is not accounting for the fact that it's already valid. This just started happening today, I'm not sure if something changed on Let's Encrypt's end either.
The text was updated successfully, but these errors were encountered: