Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crypto/x509: unexpected name mismatch error #31440

Open
sneis opened this issue Apr 12, 2019 · 14 comments
Open

crypto/x509: unexpected name mismatch error #31440

sneis opened this issue Apr 12, 2019 · 14 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Unfortunate
Milestone

Comments

@sneis
Copy link

sneis commented Apr 12, 2019

What version of Go are you using (go version)?

Sorry, I'm not actually using Go itself, but I'm using etcd (https://github.com/etcd-io/etcd; apparently written in go) and got an error about subject and issuer names not matching in my certificate chain, which I believe can be tracked down to go's crypto/x509 implementation.

What did you do?

I tried to verify a certificate chain, where the Issuer DN of sub-CA certificate was specified using the ASN.1 type UTF8String, while the subject DN of the (renewed) CA certificate used PRINTABLESTRING.

What did you expect to see?

RFC 5280 (dated May 2008), section 7.1 says:
RFC 3280 required only binary comparison of attribute values encoded in UTF8String, however, this specification requires a more comprehensive handling of comparison.
And then it goes on to give details on how to compare DN's and then refers to RFC 4518 on how to compare strings of different ASN.1 types. To me, this sounds like at least comparing the exact same ASCII string when encoded as PRINTABLESTRING in one instance and as UTF8STRING in the other still should result in recognizing the two string as equal and thus the certificate chain should be validated successfully (as is the case e.g. for OpenSSL).

What did you see instead?

I got a name mismatch, just as if the ASN.1 encoding of the DN's would be "blindly" compared byte by byte as described by the outdated RFC 3280 (crypto/x509/verify.go:570 seems to indicate that this actually is the case). Of course, for now, there's the obvious workaround of ensuring the ASN.1 encodings of DN's are identical, but in the long run, that shouldn't be necessary (and if you're switching to a different software (version) for generating certificates, ensuring continuing compatibility might not be trivial).

@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Apr 12, 2019
@bcmills
Copy link
Contributor

bcmills commented Apr 12, 2019

CC @FiloSottile @agl

@bcmills
Copy link
Contributor

bcmills commented Apr 12, 2019

Could you give some concrete examples of certificate chains that exhibit this problem? (Are the certificates in question published publicly?)

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Apr 12, 2019
@sneis
Copy link
Author

sneis commented Apr 13, 2019

I did a quick attempt at creating a set of suitable dummy certfificates (Basically create a dummy Root with openSSL, create a certificate signed by that root, change the string type defined in openssl.cnf from "utf8only" to "default", refresh the root certificate with one with longer validity). If you unzip the attached certs.zip, using OpenSSl, both
openssl verify -verbose -CAfile cacert.pem subcacert.pem
and
openssl verify -verbose -CAfile cacert-new.pem subcacert.pem
successfully verify the certificate in subcacert.pem, while I claim that with the routines from crypto/x509 subcacert.pem can only be validated against cacert.pem while validation against cacert-new.pem is going to fail.
certs.zip

@bcmills bcmills removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Apr 13, 2019
@bcmills bcmills added this to the Go1.13 milestone Apr 13, 2019
@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019
@FiloSottile
Copy link
Contributor

@sleevi Can you help me with some context on how often non-exact comparisons are necessary in chain building? I do believe we are off-spec here, but it's also an extremely sensitive area, and I am uncomfortable relaxing comparison rules. In Go we strive to keep complexity under control by implementing only the important subsets of the specs. Is this something we can get away without?

@sleevi
Copy link

sleevi commented Oct 1, 2019

@FiloSottile You're totally on-spec. Just the old spec ;)

RFC 2459 allows for byte-for-byte comparison of basically everything but PrintableString, and doesn't require implementations to collapse the string types.

RFC 3280 continues this - implementations MAY assume different strings types are different strings.

While RFC 5280 does change this up, I do want to highlight an important part of the profile - namely:

   When encoding attribute values of type DirectoryString, conforming
   CAs MUST use PrintableString or UTF8String encoding, with the
   following exceptions:

      (a)  When the subject of the certificate is a CA, the subject
           field MUST be encoded in the same way as it is encoded in the
           issuer field (Section 4.1.2.4) in all certificates issued by
           the subject CA.  Thus, if the subject CA encodes attributes
           in the issuer fields of certificates that it issues using the
           TeletexString, BMPString, or UniversalString encodings, then
           the subject field of certificates issued to that CA MUST use
           the same encoding.

Now, in practice, I'm only aware of a single mainstream implementation implementing the rules defined in RFC 5280, Section 7.1 - Microsoft Windows. Other implementations among popular OSes and libraries (Mozilla NSS, Apple macOS and iOS, Google Chrome and Android, most of the *SSL variants like wolf/cyrus/matrix/ya/etc, including OpenSSL/BoringSSL), do not implement that, and at best assert conformity with 3280 rather than 5280, because 5280's rules require a full LDAP stringprep table. It's been a while, but I think the Sun Java implementation tried "similar, but not quite" with respect to LDAP StringPrep.

RFC 5280, Section 4.2.1.10 also anticipates this, when in discussing nameConstraints, it states

   When applying restrictions of the form directoryName, an
   implementation MUST compare DN attributes.  At a minimum,
   implementations MUST perform the DN comparison rules specified in
   Section 7.1.  CAs issuing certificates with a restriction of the form
   directoryName SHOULD NOT rely on implementation of the full ISO DN
   name comparison algorithm.  This implies name restrictions MUST be
   stated identically to the encoding used in the subject field or
   subjectAltName extension.

Namely, while RFC 5280 requires you do X, CAs should not expect clients to do X.

Now, while I stated that OpenSSL/BoringSSL/NSS do not implement the RFC 5280 algorithm, I'm aware #31440 (comment) claims to the contrary. The reason for this seeming inconsistency is the fact that OpenSSL/BoringSSL/NSS do something closer to the RFC 3280 rules that implementations MAY do; namely, it folds UTF8String and PrintableString into an internal canonical encoding before comparison. Some versions of macOS do this as well. However, this canoncalization is not the StringPrep profile mentioned by RFC 5280, and merely taking advantage of the liberalness of the MAY in 3280 to treat values in different types as equivalent to some internal string representation. However, that liberalness is, in practice, a violation of RFC 3280 itself, since they internally convert UTF8String to a canonical (whitespace folded, case normalized) form, which 3280 states that attribute values other than PrintableString are case sensitive and only PrintableString is compared in that way.

There are, unfortunately, some (legacy) public CAs that relied on this franken-hybrid approach, and did not follow the profile of using ensuring issuer and subject were consistently encoded. I'm not aware of any meaningful or significant compatibility issues you'll run into publicly. On internal CAs, this most commonly seems to be from internal CAs developed using older versions of OpenSSL (which did not default to UTF8String) and certificates issued with newer versions (which do default to UTF8String), and configuration being done by config files rather than by copying the Issuer->Subject.

I'm not sure if all of this info was helpful, @FiloSottile , but my $.02 would be that unless Golang is shipping with a full StringPrep implementation (... which has its own sharp edges), that the most consistent thing to do with the ecosystem would be the 3280 behaviour and note it as a "Known Issue". The second most consistent thing, which is more of the "make up the rules but others did it first", is to fold UTF8String and PrintableString into the same internal representation, then case fold and whitespace fold. Which is a violation of all the RFCs, but... matches others.

@sleevi
Copy link

sleevi commented Oct 1, 2019

Ah, and I somehow missed that #31440 (comment) mentioned that, yes, it's the "utf8only" v "default" switch in OpenSSL.

@FiloSottile
Copy link
Contributor

Thanks @sleevi for the full overview. Given this context, I'm inclined to keep things as is, in particular because this is the first time it comes up in 10 years. Go applies a very strict complexity budget, and it seems we will not take a compatibility hit sufficient to justify the complexity of any of the solutions.

I will leave this open so we get to know if it affects more people.

@FiloSottile FiloSottile modified the milestones: Go1.14, Unplanned Oct 1, 2019
@sneis
Copy link
Author

sneis commented Oct 9, 2019

I have been reading that paragraph from section 4.1.2.6 of RFC5280 which @sleevi quoted as "you must be using UFT8String or PrintableString and do the 'complicated' comparison for all DirectoryStrings (including subject and issuer field) - but, for historical reasons, it is permissible to use one of the other three string types for subject and issuer fields, provided you follow the rules for such an exception (i.e. if you use neither UTF8String nor PrintableString, then - and only then - you have to use the exact same encoding for the subject field of the issuing CA and all the issuer fields of all the issued sub-CAs )."
So, to me it sounds like it's essentially requiring everyone to do it the way OpenSSL does it.

However, since english is not my native language (and the kind of "legal english" used in RFCs sometimes doesn't make things easier to understand, even though it's "designed" to prevent misunderstandings), that might have been a misunderstanding on my part. If it is, I feel like it happened to the OpenSSL people as well, which is a bit of a consolation. ;-)

Anyway, I have to agree with the argument, that it apparently might not be worth the additional complexity.

@ikapelyukhin
Copy link

I will leave this open so we get to know if it affects more people.

We ran into this issue as well. I've published the scripts I used to diagnose the issue in this repo. We ended up re-generating server certificates to have matching ASN.1 types.

@paulgriffiths
Copy link

paulgriffiths commented Dec 6, 2019

I will leave this open so we get to know if it affects more people.

I have also run into this issue with cross-signed CA certificates, where the original CA certificate encoded the subject name as PrintableString, while the cross-signed CA certificate encoded the subject name as UTF8String. Since the two CA certificates are issued by different entities, this is definitely a real thing that can happen.

@kse-clearhaus
Copy link

I will leave this open so we get to know if it affects more people.

We have also run in to this issue. In our case, the intermediate certificate uses UTF8STRING, while the leaf uses PRINTABLESTRING.
This certificate chain is used for financial services at one of the largest card payment organizations.

There's sadly no hope of getting them to re-issue the certificate.

@ahmet2mir
Copy link

Hello, we face the same issue on Sensu Go, our CA uses PrintableString where our automate for SSL certs set UTF8STRING on some fields, Go refuse the cert due to this.
Go should at least strips off the outer sequence header, see http://openssl.6102.n7.nabble.com/utf8string-vs-printablestring-mismatch-in-certificate-checking-td25810.html
To fix it, we just ensure that our automate use the same format than the CA.

@tardevnull
Copy link

tardevnull commented Oct 21, 2020

I implemented RFC4518 and "comparison of distinguished name" described in RFC 5280 section-7.
https://github.com/tardevnull/ldapstrprep
https://github.com/tardevnull/dn
I experimentally implemented x509 pkg which can solve the following issue.
https://github.com/tardevnull/x509
Only difference with crypto/x509 is overridden "isValid" function.
You can try this:
https://play.golang.org/p/r-mLfrktw8B
This is case that CA's subject "C=JP(PrintableString),O=FOO(UTF8String),CN=CA(UTF8String)" and End Entity's issuer "C=JP(PrintableString),O=foo(UTF8String),CN=ca(PrintableString)".
At least for my case, this implementation is must. So, I implemented it.
I hope this will be imported to golang/go.

@b4ldr
Copy link

b4ldr commented Aug 30, 2023

I will leave this open so we get to know if it affects more people.

This has also been reported as an issue with puppetlabs puppetserver

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Unfortunate
Projects
None yet
Development

No branches or pull requests

12 participants