Add support for POINT_CONVERSION_HYBRID

[samuel40791765]

Issues:

Resolves CryptoAlg-2697

Description of changes:

This is the first time it has come up, but Ruby depends on the POINT_CONVERSION_HYBRID format in its tests and expects the underlying libcrypto to support it.

• https://github.com/ruby/ruby/blob/ruby_3_1/test/openssl/test_pkey_ec.rb#L358-L366

There's not too much a difference between POINT_CONVERSION_HYBRID and POINT_CONVERSION_UNCOMPRESSED except that the first bit contains information about the quadratic equation y is at. We have the building blocks to support and can get this working with some tweaking around.

Call-outs:

POINT_CONVERSION_HYBRID works very similar to POINT_CONVERSION_UNCOMPRESSED so we're reusing a lot of code. Most of this change is working with the first byte POINT_CONVERSION_HYBRID does differently and point the encoding to the right places for parsing.

Testing:

• Testing parsing from and back with POINT_CONVERSION_HYBRID. What we're parsing to and from hybrid doesn't really matter here, they're already covered by other code paths.

• Test vectors were generated from our test cases and ran against OpenSSL with this brief test program:

int main() {
  // Generate |ec_key|.
  EC_KEY *ec_key = EC_KEY_new();
  EC_GROUP *group = EC_GROUP_new_by_curve_name(NID_secp521r1);
  EC_KEY_set_group(ec_key, group);

  const uint8_t *inp = kECPublicKey_uncompressed;
  // Decoding an EC point.
  o2i_ECPublicKey(&ec_key, &inp, sizeof(kECPublicKey_uncompressed));
  // On successful exit of |o2i_ECPublicKey|, |*inp| is advanced by |len| bytes.
  if(kECPublicKey_uncompressed + sizeof(kECPublicKey_uncompressed) != inp) {
    return -1;
  }
  // Set |conv_form| of |ec_key|.
  EC_KEY_set_conv_form(ec_key, POINT_CONVERSION_HYBRID);
  uint8_t *p = nullptr;
  size_t len2 = i2o_ECPublicKey(ec_key, &p);
   for(uint32_t i=0; i < len2; i++) {
      if(i%12 == 0 && i != 0){
        printf("\n");
      }
      printf("0x%02x, ", p[i]);
    }
       printf("\n");
  return 0;
}

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and the ISC license.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 81.48148% with 5 lines in your changes missing coverage. Please review.

Please upload report for BASE (main@8d9809e). Learn more about missing BASE report.
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
crypto/fipsmodule/ec/oct.c	81.48%	5 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1936   +/-   ##
=======================================
  Coverage        ?   78.74%           
=======================================
  Files           ?      590           
  Lines           ?   101428           
  Branches        ?    14383           
=======================================
  Hits            ?    79867           
  Misses          ?    20924           
  Partials        ?      637           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[dkostic]

Suggested change

	// equation, but also encodes the actual point along with it.
	// equation, but also encodes the y coordinate along with it.

[dkostic]

why do we handle the hybrid case in function called ec_point_from_uncompressed?

[samuel40791765]

When we ignore the first byte (which reveals the encoding format), uncompressed form is essentially identical to hybrid. It seemed like a bit redundant to write another ec_point_from_hybrid function with uncompressed parsing logic.
I could rename this function for a bit more clarity if needed? I was also considering adding a check in the hybrid decoding to see if y actually had the same solution that the first byte indicated, is that something we should do?

[dkostic]

I'd prefer having a separate ec_point_from_hybrid function, it's redundant but simpler.

I was also considering adding a check in the hybrid decoding to see if y actually had the same solution that the first byte indicated, is that something we should do?

That's a great question. Since we are doing this for compatibility with OpenSSL I guess you should see if they are checking the point consistency?

[samuel40791765]

Sure, I did a quick look into what OpenSSL was doing. They have a check for consistency that goes both ways.
Our's will look slightly different since we save the date in EC_FELEM, but I'll figure something out here.

[dkostic]

please explain the magic & ~1u?