evp: fix EVP_PKEY_cmp for EC keys after DER deserialization

[MarkAtwood]

Summary

wolfSSL_EVP_PKEY_cmp returns "not equal" for EC keys that are identical but one was deserialized from DER. This breaks the OpenSSL API contract that EVP_PKEY_cmp compares key material regardless of how the EVP_PKEY objects were constructed.

Reproducer

// Generate an EC P-256 key
EVP_PKEY_CTX *ctx = EVP_PKEY_CTX_new_id(EVP_PKEY_EC, NULL);
EVP_PKEY_keygen_init(ctx);
EVP_PKEY_CTX_set_ec_paramgen_curve_nid(ctx, NID_X9_62_prime256v1);
EVP_PKEY *original = NULL;
EVP_PKEY_keygen(ctx, &original);

// Serialize to RFC 5915 DER (ECPrivateKey without optional public key)
EC_KEY *ec = EVP_PKEY_get0_EC_KEY(original);
unsigned char *der = NULL;
int der_len = i2d_ECPrivateKey(ec, &der);

// Deserialize back
const unsigned char *p = der;
EC_KEY *ec2 = d2i_ECPrivateKey(NULL, &p, der_len);
EVP_PKEY *deserialized = EVP_PKEY_new();
EVP_PKEY_assign_EC_KEY(deserialized, ec2);

// Compare — returns -1 (not equal) instead of 0 (equal)
int result = wolfSSL_EVP_PKEY_cmp(original, deserialized);
// Expected: 0 (match), Actual: -1 (no match)

Root Cause

Two issues compound:

1. inSet not checked before accessing ecc->internal

After d2i_PrivateKey or d2i_ECPrivateKey, the external WOLFSSL_EC_KEY fields (BIGNUMs for private key, WOLFSSL_EC_POINT for public key) are populated, but the internal wolfCrypt ecc_key struct may not be synced. The inSet flag tracks this. wolfSSL_EVP_PKEY_cmp accessed ecc->internal without checking inSet or calling SetECKeyInternal to sync.

2. Private-only keys have empty pubkey point

When wc_EccPrivateKeyDecode parses an RFC 5915 ECPrivateKey that omits the optional public key field (which is valid per the RFC), it calls wc_ecc_import_private_key_ex with pub=NULL, pubSz=0. This sets ecc_key.type = ECC_PRIVATEKEY_ONLY and leaves ecc_key.pubkey as all zeros. The comparison then compared a populated pubkey against zeros → "not equal".

Fix

In the WC_EVP_PKEY_EC case of wolfSSL_EVP_PKEY_cmp:

1. Call SetECKeyInternal() if inSet == 0 to sync external → internal representation
2. If ecc_key.type == ECC_PRIVATEKEY_ONLY, call wc_ecc_make_pub() to derive the public key from the private key before comparing

This ensures the comparison works regardless of how the key was constructed — generated, imported with public key, or imported private-only.

Test Plan

• Generate EC key, serialize to PKCS#8, deserialize, EVP_PKEY_cmp returns match
• Generate EC key, serialize to RFC 5915 (without optional public key), deserialize, EVP_PKEY_cmp returns match
• Two different EC keys still return "not equal"
• RSA key comparison still works (unchanged code path)
• Run existing wolfSSL test suite

[Copilot]

Pull request overview

Fixes wolfSSL_EVP_PKEY_cmp returning “not equal” for identical EC keys when one key was DER-deserialized without an embedded public key, aligning behavior with the expected OpenSSL-style key-material comparison semantics.

Changes:

• Sync EC ecc_key internal state (inSet) before comparing in wolfSSL_EVP_PKEY_cmp.
• For private-only EC imports, derive the public key before point comparison.
• Harden wc_ecc_import_point_der_ex by validating point type early and rejecting truncated compressed-point encodings.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File	Description
wolfcrypt/src/evp.c	Synces EC internal state and derives missing public key to make EC key comparisons succeed after DER deserialization.
wolfcrypt/src/ecc.c	Adds earlier input validation for EC point import and rejects invalid/truncated compressed encodings.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

[LinuxJedi]

@MarkAtwood please remove the Claude co-authored-by in the commit message.

[ColtonWilley]

Jenkins retest this please

[github-actions]

MemBrowse Memory Report

gcc-arm-cortex-m4-openssl-compat

• FLASH: .text +64 B (+0.0%, 763,188 B / 1,048,576 B, total: 73% used)
  No memory changes detected for:
• gcc-arm-cortex-m0plus
• gcc-arm-cortex-m3
• gcc-arm-cortex-m4
• gcc-arm-cortex-m4-baremetal
• gcc-arm-cortex-m4-crypto-only
• gcc-arm-cortex-m4-dtls13
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-pkcs7
• gcc-arm-cortex-m4-pq
• gcc-arm-cortex-m4-rsa-only
• gcc-arm-cortex-m4-sp-math
• gcc-arm-cortex-m4-tls12
• gcc-arm-cortex-m4-tls13
• gcc-arm-cortex-m7
• gcc-arm-cortex-m7-pq
• gcc-arm-cortex-m7-tls13
• linuxkm-pie
• linuxkm-standard
• stm32-sim-stm32h753

[MarkAtwood]

Co-Authored-By line was already removed in the current commit. The CHANGES_REQUESTED is stale — ready for re-review.

fixed