Re: RSA sign/verify and hash generation functions

[Nikos Mavrogiannopoulos]

On 12/08/2010 12:30 AM, Murray S. Kucherawy wrote:
> I got a fair bit further, but I'm stuck now with GnuTLS not generating the 
> same signature as OpenSSL under the same circumstances.  Anyone that can spot 
> what I've missed, please do let me know.
> 
> Here's what I'm doing:
> 
> gnutls_datum_t rsa_out; /* output signature */
> gnutls_datum_t dd; /* SHA256 digest; size = 20 */
> gnutls_datum_t key; /* private key in PEM form */
> gnutls_x509_privkey_t privkey;
> gnutls_privkey_t rsa_key;
> 
> key.data = <buffer containing PEM formatted private key>
> key.len = strlen(key.data);
> 
> dd.data = (SHA256 of object to sign);
> dd.size = 20 (size of a SHA256 digest);

Why 20? SHA-1 is 20 bytes. SHA256 is 32 bytes.

> assert(gnutls_privkey_init(&rsa_key) == GNUTLS_E_SUCCESS);
> assert(gnutls_x509_privkey_init(&privkey) == GNUTLS_E_SUCCESS);
> assert(gnutls_x509_privkey_import(privkey, &key, GNUTLS_X509_FMT_PEM) == 
> GNUTLS_E_SUCCESS);
> assert(gnutls_privkey_import_x509(rsa_key, privkey, 0) == GNUTLS_E_SUCCESS);
> assert(gnutls_privkey_sign_hash(rsa_key, &dd, &rsa_out == GNUTLS_E_SUCCESS);
> At this point, comparing "rsa_out" in the GnuTLS case to the OpenSSL case 
> reveals that rsa_out.size is the same, but the data in rsa_out.data is not.
> I haven't tried the public key verification code nor the public key 
> extraction (In fact I haven't even found that yet).

Do you mean the gnutls_x509_crt_verify_hash()?
Which signing method do you use with openssl? In gnutls we support only
PKCS #1 1.5 signatures (that one required by TLS).

regards,
Nikos