Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/crypto/openpgp: strips \r decrypting binary data #13409

Closed
tobstarr opened this issue Nov 26, 2015 · 12 comments
Closed

x/crypto/openpgp: strips \r decrypting binary data #13409

tobstarr opened this issue Nov 26, 2015 · 12 comments

Comments

@tobstarr
Copy link

When encrypting specific (in my case gzip compressed) plaintexts it seems that golang.org/x/crypto/openpgp generates ciphertexts which can then no longer be decrypted with the gpg cli tool. Those broken ciphertexts can be decrypted with the openpgp package itself just fine though.

I created a gist with some example plaintexts (good and bad) and some test code to reproduce the issue at https://gist.github.com/tobstarr/a4afa1ec7abf06fbe9e5

As described in the gist I tested it both with gpg versions 1.4.16 on ubuntu 14.04 and gpg version 1.4.18 on ubuntu 15.10 with my own key (4096 RSA) and a newly generated one (2048 RSA).

@tobstarr
Copy link
Author

/cc @nightlyone

@tobstarr
Copy link
Author

Regarding go versions: I tested it on ubuntu 15.10 with go versions 1.4.3 and 1.5.1 and on ubuntu 14.04 with version 1.2.1

@rsc
Copy link
Contributor

rsc commented Dec 28, 2015

Looking at the gist, it shouldn't be necessary to drag tar into this. Either gpg spits out the original file or it does not. When it does not, how does the gpg output differ from the original?

@rsc rsc added this to the Unreleased milestone Dec 28, 2015
@tobstarr
Copy link
Author

tobstarr commented Jan 5, 2016

I did some brute forcing and it seems all issues are related to different treating of the "\r" character.

When decrypting with gpg all "\r" characters seem to be stripped, when decrypting with the openpgp package they are still present (so that data must still be somewhere in the ciphertext).

@rsc
Copy link
Contributor

rsc commented Jan 5, 2016

/cc @bradfitz openpgp seems to be stripping \r in Decrypt somewhere.

@rsc rsc changed the title x/crypto/openpgp: encryption of specific gzip compressed plaintext leads to incompatibility with the gpg cli tool x/crypto/openpgp: strips \r decrypting binary data Jan 5, 2016
@tobstarr
Copy link
Author

tobstarr commented Jan 5, 2016

@bradfitz actually it does something to "\r" when encrypting which then leads to the gpg cli tool to strip those, when decrypting with openpgp they are still present.

@tobstarr
Copy link
Author

tobstarr commented Jan 5, 2016

See the example in the (now updated) gist at https://gist.github.com/tobstarr/a4afa1ec7abf06fbe9e5

@bradfitz
Copy link
Contributor

bradfitz commented Jan 5, 2016

/cc @agl as FYI

But I can look at this too.

@rsc
Copy link
Contributor

rsc commented Jan 6, 2016

You said the "now updated gist" but I still see the old one with output not matching the program.

l.Printf("input=%q output=%q otuput_openpgp=%q", pt, dec, decOpenpgp)

does not match:

input="hello world" output="hello world"
input="hello\n\rworld\n\r" output="hello\nworld\n"
input="\n" output="\n"
input="\t" output="\t"
input="\r" output=""

There is no otuput_openpgp= shown.

@tobstarr
Copy link
Author

tobstarr commented Jan 6, 2016

@rsc sorry, by updated I was referring to original version (with the random tar input). I now also updated the output in the README though.

@rsc
Copy link
Contributor

rsc commented Jan 6, 2016

OK, thanks. With the corrected output I now see that:

  1. Using Go's openpgp library you encrypt a file with \r characters.
  2. Using Go's openpgp library to decrypt produces the original bytes verbatim.
  3. Using the gpg command line tool to decrypt strips the \r characters.

That sounds like a problem in gpg, not Go's openpgp library.

Can you check whether a round trip using gpg alone (leave Go openpgp completely out of it) preserves the \r characters?

If gpg cannot round trip a binary file containing \r characters, then I think it's just gpg's issue, although it may be intended that gpg is only used for text, and binary uses may be expected to do base64 or uuencoding or something like that first. But if gpg can round trip a binary file containing \r's, then maybe Go's openpgp library encryption in step 1 is failing to set some kind of "binary-safe" bit in the encrypted output.

Thanks.

@tobstarr
Copy link
Author

tobstarr commented Jan 6, 2016

Okay, my bad. I looked through the code and saw that there is "IsBinary" flag in the FileHints options which is false by default. My problem was that gpg treats everything as binary by default and in openpgp it is the other way round.

When I set that flag all my tests pass. Sorry for the inconvenience,

@tobstarr tobstarr closed this as completed Jan 6, 2016
@golang golang locked and limited conversation to collaborators Jan 7, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants