Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

encoding/asn1: Unmarshal rejects PrintableString containing underscore #14017

Closed
laybay opened this issue Jan 19, 2016 · 11 comments
Closed

encoding/asn1: Unmarshal rejects PrintableString containing underscore #14017

laybay opened this issue Jan 19, 2016 · 11 comments
Labels
FrozenDueToAge
Milestone
Go1.7Early

Comments

@laybay
Copy link

laybay commented Jan 19, 2016

in my program，import a customer X509 digital certificate， But ,one of this get a error, and I found in
KeyBlock,Bytes has a code 130 0x82 is not printable,so get reject, but I use C program and java all can load success! Please help me if thiis is a go mistake?
Cert, err2 := x509.ParseCertificate(KeyBlock.Bytes)
28 if err2 != nil {
29 return nil, e.New(1004, err2, "ReadX509Cert x509.ParseCertificate error;")
30 }
@agl
Copy link
Contributor

agl commented Jan 19, 2016

Can you include the certificate? This sounds like it's just a case where Go is being stricter about the rules.

(Note: you can still run a TLS server with a certificate that doesn't parse if you wish.)

@agl agl self-assigned this Jan 19, 2016
@ianlancetaylor ianlancetaylor changed the title x509.ParseCertificate get faild err:asn1: syntax error: PrintableString contains invalid character crypto/x509: ParseCertificate get failed err:asn1: syntax error: PrintableString contains invalid character Jan 19, 2016
@ianlancetaylor ianlancetaylor added this to the Go1.6Maybe milestone Jan 19, 2016
@laybay
Copy link
Author

laybay commented Jan 21, 2016

-----BEGIN CERTIFICATE-----
MIIELjCCAxagAwIBAgIFIABlIQQwDQYJKoZIhvcNAQEFBQAwWTELMAkGA1UEBhMC
Q04xMDAuBgNVBAoTJ0NoaW5hIEZpbmFuY2lhbCBDZXJ0aWZpY2F0aW9uIEF1dGhv
cml0eTEYMBYGA1UEAxMPQ0ZDQSBURVNUIE9DQTExMB4XDTEzMTIxODA5MjcwN1oX
DTE1MTIxODA5MjcwN1owazELMAkGA1UEBhMCQ04xDjAMBgNVBAoTBXl1emhpMRIw
EAYDVQQLEwlDTUJDX0RDTVMxFDASBgNVBAsTC0luZGl2aWR1YWwxMSIwIAYDVQQD
FBljbj0wMDA0MDAwMDpTSUdOQDAwMDAwMDAyMIIBIjANBgkqhkiG9w0BAQEFAAOC
AQ8AMIIBCgKCAQEAtTBhimpRdnIMC7SBfl3xeSJR8XF0Jqf9XItroLMoffxyNfRB
khKGf87oLDaGfZa6s/+vct2iktHNMcjS8gMmlBj/RjXflAdEVkeQAfSiZFN8Bn9q
14+dEXtVbvrosaFHmfTmHWchMMeXMJlg9K6akM+lVwmPjLfT4bcDk67VVUCy4uLj
xe0YgrBFfqAxcj4/vlHAdKY+3n+keUQ9PrBRuYXQcGgXaBEGlhnCJIrnVnnZdRUS
T3qt6iUUUu7/2rIvvrv+/aLN3li432LuhJc3DWANhwucC7DDpXoFScoqrgpQbofc
vPdbJdsRrLvZ/XYDmmcdYOMsQ7AnZIbVVUws3wIDAQABo4HqMIHnMB8GA1UdIwQY
MBaAFPwLvESaDjGhg6mBhyceBULGv1b4MEgGA1UdIARBMD8wPQYIYIEchu8qAQIw
MTAvBggrBgEFBQcCARYjaHR0cDovL3d3dy5jZmNhLmNvbS5jbi91cy91cy0xNS5o
dG0wOQYDVR0fBDIwMDAuoCygKoYoaHR0cDovLzIxMC43NC40MS44Ny9vY2ExMS9S
U0EvY3JsNjM5LmNybDALBgNVHQ8EBAMCA+gwHQYDVR0OBBYEFLBo5Ut5b4js+bsL
yVe73j4xNcDOMBMGA1UdJQQMMAoGCCsGAQUFBwMCMA0GCSqGSIb3DQEBBQUAA4IB
AQBQbuGIjDNd0wT1dGcjsImtXAmHZaytqUn3mY2VdAEZhR1rLEGjkG+j30zIV7K8
BrUWOf0Pny6tQ+OTevNGCV1hiO2ZQryB50R/eBGSk70dBqIDzpOk4cAj6OBaO+uh
Uqw2jdDOt1rV3UeptMD4Mbs3FhdQy3sDApkZs10px/5PHMohjktnwSuFmXaNwYsX
Sv282c6tjiDWYcQrTHZVfDhhrooAjLSQ6wDWW/QnjhbcNbY2lnf9hReYoDBloK8G
O0AH1nfVkMVY0Ja6OdwnJD+NFIL0yPqz1gH7qzrmcGOZefiUPC+LGU0D+qYU1f22
afnVaYT41MCjuM1nXmMcEmLt
-----END CERTIFICATE-----

@laybay
Copy link
Author

laybay commented Jan 21, 2016

this is the load fail x509 cert please check .thanks

@laybay
Copy link
Author

laybay commented Jan 22, 2016

@agl @ianlancetaylor I have upload the certificate.... please help me thanks

@szank
Copy link

szank commented Jan 22, 2016

@laybay I did some digging, and I think there is a bug in the asn1 package.
Your certificate looks like this :
// EDITED !

$ openssl asn1parse -in githubtestcert.pem 
    0:d=0  hl=4 l=1070 cons: SEQUENCE          
    4:d=1  hl=4 l= 790 cons: SEQUENCE          
    8:d=2  hl=2 l=   3 cons: cont [ 0 ]        
   10:d=3  hl=2 l=   1 prim: INTEGER           :02
   13:d=2  hl=2 l=   5 prim: INTEGER           :2000652104
   20:d=2  hl=2 l=  13 cons: SEQUENCE          
   22:d=3  hl=2 l=   9 prim: OBJECT            :sha1WithRSAEncryption
   33:d=3  hl=2 l=   0 prim: NULL              
   35:d=2  hl=2 l=  89 cons: SEQUENCE          
   37:d=3  hl=2 l=  11 cons: SET               
   39:d=4  hl=2 l=   9 cons: SEQUENCE          
   41:d=5  hl=2 l=   3 prim: OBJECT            :countryName
   46:d=5  hl=2 l=   2 prim: PRINTABLESTRING   :CN
   50:d=3  hl=2 l=  48 cons: SET               
   52:d=4  hl=2 l=  46 cons: SEQUENCE          
   54:d=5  hl=2 l=   3 prim: OBJECT            :organizationName
   59:d=5  hl=2 l=  39 prim: PRINTABLESTRING   :China Financial Certification Authority
  100:d=3  hl=2 l=  24 cons: SET               
  102:d=4  hl=2 l=  22 cons: SEQUENCE          
  104:d=5  hl=2 l=   3 prim: OBJECT            :commonName
  109:d=5  hl=2 l=  15 prim: PRINTABLESTRING   :CFCA TEST OCA11
  126:d=2  hl=2 l=  30 cons: SEQUENCE          
  128:d=3  hl=2 l=  13 prim: UTCTIME           :131218092707Z
  143:d=3  hl=2 l=  13 prim: UTCTIME           :151218092707Z
  158:d=2  hl=2 l= 107 cons: SEQUENCE          
  160:d=3  hl=2 l=  11 cons: SET               
  162:d=4  hl=2 l=   9 cons: SEQUENCE          
  164:d=5  hl=2 l=   3 prim: OBJECT            :countryName
  169:d=5  hl=2 l=   2 prim: PRINTABLESTRING   :CN
  173:d=3  hl=2 l=  14 cons: SET               
  175:d=4  hl=2 l=  12 cons: SEQUENCE          
  177:d=5  hl=2 l=   3 prim: OBJECT            :organizationName
  182:d=5  hl=2 l=   5 prim: PRINTABLESTRING   :yuzhi
  189:d=3  hl=2 l=  18 cons: SET               
  191:d=4  hl=2 l=  16 cons: SEQUENCE          
  193:d=5  hl=2 l=   3 prim: OBJECT            :organizationalUnitName
  198:d=5  hl=2 l=   9 prim: PRINTABLESTRING   :CMBC_DCMS
  209:d=3  hl=2 l=  20 cons: SET               
  211:d=4  hl=2 l=  18 cons: SEQUENCE          
  213:d=5  hl=2 l=   3 prim: OBJECT            :organizationalUnitName
  218:d=5  hl=2 l=  11 prim: PRINTABLESTRING   :Individual1
  231:d=3  hl=2 l=  34 cons: SET               
  233:d=4  hl=2 l=  32 cons: SEQUENCE          
  235:d=5  hl=2 l=   3 prim: OBJECT            :commonName
  240:d=5  hl=2 l=  25 prim: T61STRING         :cn=00040000:SIGN@00000002
  267:d=2  hl=4 l= 290 cons: SEQUENCE          
  271:d=3  hl=2 l=  13 cons: SEQUENCE          
  273:d=4  hl=2 l=   9 prim: OBJECT            :rsaEncryption
  284:d=4  hl=2 l=   0 prim: NULL              
  286:d=3  hl=4 l= 271 prim: BIT STRING        
  561:d=2  hl=3 l= 234 cons: cont [ 3 ]        
  564:d=3  hl=3 l= 231 cons: SEQUENCE          
  567:d=4  hl=2 l=  31 cons: SEQUENCE          
  569:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Authority Key Identifier
  574:d=5  hl=2 l=  24 prim: OCTET STRING      [HEX DUMP]:30168014FC0BBC449A0E31A183A98187271E0542C6BF56F8
  600:d=4  hl=2 l=  72 cons: SEQUENCE          
  602:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Certificate Policies
  607:d=5  hl=2 l=  65 prim: OCTET STRING      [HEX DUMP]:303F303D060860811C86EF2A01023031302F06082B060105050702011623687474703A2F2F7777772E636663612E636F6D2E636E2F75732F75732D31352E68746D
  674:d=4  hl=2 l=  57 cons: SEQUENCE          
  676:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 CRL Distribution Points
  681:d=5  hl=2 l=  50 prim: OCTET STRING      [HEX DUMP]:3030302EA02CA02A8628687474703A2F2F3231302E37342E34312E38372F6F636131312F5253412F63726C3633392E63726C
  733:d=4  hl=2 l=  11 cons: SEQUENCE          
  735:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Key Usage
  740:d=5  hl=2 l=   4 prim: OCTET STRING      [HEX DUMP]:030203E8
  746:d=4  hl=2 l=  29 cons: SEQUENCE          
  748:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Subject Key Identifier
  753:d=5  hl=2 l=  22 prim: OCTET STRING      [HEX DUMP]:0414B068E54B796F88ECF9BB0BC957BBDE3E3135C0CE
  777:d=4  hl=2 l=  19 cons: SEQUENCE          
  779:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Extended Key Usage
  784:d=5  hl=2 l=  12 prim: OCTET STRING      [HEX DUMP]:300A06082B06010505070302
  798:d=1  hl=2 l=  13 cons: SEQUENCE          
  800:d=2  hl=2 l=   9 prim: OBJECT            :sha1WithRSAEncryption
  811:d=2  hl=2 l=   0 prim: NULL              
  813:d=1  hl=4 l= 257 prim: BIT STRING

EDITED :
I have found the problem :

189:d=3  hl=2 l=  18 cons: SET               
  191:d=4  hl=2 l=  16 cons: SEQUENCE          
  193:d=5  hl=2 l=   3 prim: OBJECT            :organizationalUnitName
  198:d=5  hl=2 l=   9 prim: PRINTABLESTRING   :CMBC_DCMS

According to the ASN.1 spec :

The PrintableString type denotes an arbitrary string of printable characters from the following character set:
A, B, ..., Z
a, b, ..., z
0, 1, ..., 9
(space) ' ( ) + , - . / : = ?

The _ is not a valid PrintableString character. Openssl and other libraries does ignore the spec I suppose because otherwise it would break a lot of things.
If go wants to break the spec to avoid this kind of problems should be discussed I think ( if it haven't been discussed before )

@szank
Copy link

szank commented Jan 22, 2016

@laybay I have just checked how openssl handle PRINTABLESTRING. This is a CSR but certificate should be handled the same way :

# openssl asn1parse -in testCSR.csr -i
    0:d=0  hl=4 l= 414 cons: SEQUENCE          
    4:d=1  hl=4 l= 263 cons:  SEQUENCE          
    8:d=2  hl=2 l=   1 prim:   INTEGER           :00
   11:d=2  hl=2 l=  94 cons:   SEQUENCE          
   13:d=3  hl=2 l=  11 cons:    SET               
   15:d=4  hl=2 l=   9 cons:     SEQUENCE          
   17:d=5  hl=2 l=   3 prim:      OBJECT            :countryName
   22:d=5  hl=2 l=   2 prim:      PRINTABLESTRING   :AU
   26:d=3  hl=2 l=  13 cons:    SET               
   28:d=4  hl=2 l=  11 cons:     SEQUENCE          
   30:d=5  hl=2 l=   3 prim:      OBJECT            :stateOrProvinceName
   35:d=5  hl=2 l=   4 prim:      PRINTABLESTRING   :test
   41:d=3  hl=2 l=  13 cons:    SET               
   43:d=4  hl=2 l=  11 cons:     SEQUENCE          
   45:d=5  hl=2 l=   3 prim:      OBJECT            :localityName
   50:d=5  hl=2 l=   4 prim:      PRINTABLESTRING   :city
   56:d=3  hl=2 l=  21 cons:    SET               
   58:d=4  hl=2 l=  19 cons:     SEQUENCE          
   60:d=5  hl=2 l=   3 prim:      OBJECT            :organizationName
   65:d=5  hl=2 l=  12 prim:      T61STRING         :!@#$%^&*(()_
   79:d=3  hl=2 l=  11 cons:    SET               
   81:d=4  hl=2 l=   9 cons:     SEQUENCE          
   83:d=5  hl=2 l=   3 prim:      OBJECT            :organizationalUnitName
   88:d=5  hl=2 l=   2 prim:      PRINTABLESTRING   :OU
   92:d=3  hl=2 l=  13 cons:    SET               
   94:d=4  hl=2 l=  11 cons:     SEQUENCE          
   96:d=5  hl=2 l=   3 prim:      OBJECT            :commonName
  101:d=5  hl=2 l=   4 prim:      PRINTABLESTRING   :AAA 
  107:d=2  hl=3 l= 159 cons:   SEQUENCE          
  110:d=3  hl=2 l=  13 cons:    SEQUENCE          
  112:d=4  hl=2 l=   9 prim:     OBJECT            :rsaEncryption
  123:d=4  hl=2 l=   0 prim:     NULL              
  125:d=3  hl=3 l= 141 prim:    BIT STRING        
  269:d=2  hl=2 l=   0 cons:   cont [ 0 ]        
  271:d=1  hl=2 l=  13 cons:  SEQUENCE          
  273:d=2  hl=2 l=   9 prim:   OBJECT            :sha256WithRSAEncryption
  284:d=2  hl=2 l=   0 prim:   NULL              
  286:d=1  hl=3 l= 129 prim:  BIT STRING

As you can see if the RDN can be represented as a PRINTABLESTRING (localityName) PRINTABLESTRING is used. If the RDN contains characters forbidden in the PRINTABLESTRING, (organizationName) then T61STRING is used instead.

I guess the certificate wasn't generated with openssl.

@rsc rsc changed the title crypto/x509: ParseCertificate get failed err:asn1: syntax error: PrintableString contains invalid character encoding/asn1: Unmarshal rejects PrintableString containing underscore Jan 24, 2016
@rsc
Copy link
Contributor

rsc commented Jan 24, 2016

@agl, should we accept _ in PrintableString? Seems like probably yes? What restrictions does Chrome apply to PrintableString? Thanks.

@rsc
Copy link
Contributor

rsc commented Jan 26, 2016

Per discussion with @agl, this is the first time we've seen a certificate that is malformed in this particular way, so we're not going to rush into relaxing the requirements. If you can give us more context about what software (erroneously) generated that certificate and how common the problem might be, that would help us decide what to do for Go 1.7. Adam is thinking about scanning through the Certificate Transparency logs to see if there are more there. Thanks.

@rsc rsc modified the milestones: Go1.7Early, Go1.6Maybe Jan 26, 2016
@agl
Copy link
Contributor

agl commented Jan 28, 2016

I scanned 4.5M of the 12M certificates in the Certificate Transparency "pilot" log (I ran out of disk space downloading it all) and below are the counts for the invalid characters found in PRINTABLESTRING values:

  1  ´
  1  !
  1  #
  1  å
  1  ^K
  1  ñ
  1  ø
  1  ^Y
  2  "
  2  á
  2  ç
  2  í
  3  ó
  3  (tab)
  4  @
  4  à
  4  Ý
  5  ü
121  &

That only includes public certificates but it doesn't suggest that underscore is sufficiently widely misplaced in these strings to bend the standard for. (Ampersand is used in about one in 40K certificates and might be something that we have to make an exception for in the future.)

@agl agl closed this as completed Mar 10, 2016
@jskrepnek
Copy link

Also dealing with this issue: a certificate with an underscore, in my case a client certificate. Looking for advice about how I might still use this certificate without using tls.LoadX509KeyPair.

@szank
Copy link

szank commented Jun 22, 2016

Decode it manually, verify manually (if you wish), extract the public key and there you go. What exactly do you wan to do with it ?

@golang golang locked and limited conversation to collaborators Jun 22, 2017
@rsc rsc unassigned agl Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
Go1.7Early
Development

No branches or pull requests
7 participants
@agl @rsc @szank @jskrepnek @ianlancetaylor @laybay @gopherbot