Joseph Birr-Pixton‘s entry to the 2017 Underhanded Crypto Contest is the EVP_VerifyFinal API call that actually exists in OpenSSL. This isn’t to suggest someone intentionally backdoored OpenSSL, but this API call has really poor usability, as Joesph explains:
The design of EVP_VerifyFinal
OpenSSL’s EVP_VerifyFinal function has a poor choice of return value semantics, which means naive callers can accidentally treat invalid signatures as valid.
There is indeed such vulnerable code scattered around the internet.
The Semantics
The relevent part of EVP_VerifyFinal inherits the return values of EVP_PKEY_verify. The documentation says:
EVP_PKEY_verify_init() and EVP_PKEY_verify() return 1 if the verification was
successful and 0 if it failed. Unlike other functions the return value 0 from
EVP_PKEY_verify() only indicates that the signature did not not verify
successfully (that is tbs did not match the original data or the signature was
of invalid form) it is not an indication of a more serious error.
A negative value indicates an error other that signature verification failure.
In particular a return value of -2 indicates the operation is not supported by
the public key algorithm.In C, any non-zero integer is ‘truthy’ while only zero integers are ‘falsy’. This means a naïve caller can achieve a working implementation along the following lines:
if (EVP_VerifyFinal(ctx, sig, siglen, pubkey)) {
/* signature valid */
} else {
/* signature invalid */
}
However, this code is incorrect if EVP_VerifyFinal fails with an ‘error other that signature verification failure’ (sic). Such errors include things like memory allocation failures; so an attacker able to cause memory pressure can bypass signature checks in such callers.
Example Outcome: Bitcoin Core
Bitcoin core is a fairly typical example of OpenSSL callers. It contains this code:
EVP_PKEY *pubkey = X509_get_pubkey(signing_cert);
EVP_MD_CTX_init(ctx);
if (!EVP_VerifyInit_ex(ctx, digestAlgorithm, NULL) ||
!EVP_VerifyUpdate(ctx, data_to_verify.data(), data_to_verify.size()) ||
!EVP_VerifyFinal(ctx, (const unsigned char*)paymentRequest.signature().data(), (unsigned int)paymentRequest.signature().size(), pubkey)) {
throw SSLVerifyError("Bad signature, invalid payment request.");
}BoringSSL
BoringSSL unified its return code semantics, and in doing so addressed this problem.
An amusing side effect of this change is BoringSSL itself contains code which calls this function in a truthy way. This is obviously safe, but might lead astray OpenSSL users looking for code to copy.
LibreSSL
LibreSSL has not addressed this problem.
Related Work
CVE-2008-5077 is a collection of related vulnerabilities within OpenSSL itself resulting from this design error.
The lesson here is that there needn’t be anything technically wrong with the code for things to go wrong. Confusing APIs and usability problems are an equally powerful source of bugs (and a great place to hide backdoors!).
You can find Joseph’s entire submission in the archive.