RSA/sigGen expected results contain the OID of the SHA2 algorithm instead of SHA3

[szendros]

environment
Demo

testSessionId
384995

vsId
1563841

Algorithm registration

 {
        "revision":"FIPS186-5",
        "algorithm":"RSA",
        "mode":"sigGen",
        "capabilities":[
          {
            "sigType":"pkcs1v1.5",
            "properties":[
              {
                "modulo":2048,
                "hashPair":[
                  {
                    "hashAlg":"SHA2-224"
                  },
                  {
                    "hashAlg":"SHA2-256"
                  },
                  {
                    "hashAlg":"SHA2-384"
                  },
                  {
                    "hashAlg":"SHA2-512"
                  },
                  {
                    "hashAlg":"SHA3-224"
                  },
                  {
                    "hashAlg":"SHA3-256"
                  },
                  {
                    "hashAlg":"SHA3-384"
                  },
                  {
                    "hashAlg":"SHA3-512"
                  },
                  {
                    "hashAlg":"SHAKE-256"
                  }
                ]
              },
              {
                "modulo":3072,
                "hashPair":[
                  {
                    "hashAlg":"SHA2-256"
                  },
                  {
                    "hashAlg":"SHA2-384"
                  },
                  {
                    "hashAlg":"SHA2-512"
                  },
                  {
                    "hashAlg":"SHA3-256"
                  },
                  {
                    "hashAlg":"SHA3-384"
                  },
                  {
                    "hashAlg":"SHA3-512"
                  },
                  {
                    "hashAlg":"SHAKE-256"
                  }
                ]
              },
              {
                "modulo":4096,
                "hashPair":[
                  {
                    "hashAlg":"SHA2-384"
                  },
                  {
                    "hashAlg":"SHA2-512"
                  },
                  {
                    "hashAlg":"SHA3-384"
                  },
                  {
                    "hashAlg":"SHA3-512"
                  }
                ]
              }
            ]
          },
          {
            "sigType":"pss",
            "properties":[
              {
                "modulo":2048,
                "maskFunction":[
                  "mgf1"
                ],
                "hashPair":[
                  {
                    "hashAlg":"SHA2-224",
                    "saltLen":28
                  },
                  {
                    "hashAlg":"SHA2-256",
                    "saltLen":32
                  },
                  {
                    "hashAlg":"SHA2-384",
                    "saltLen":48
                  },
                  {
                    "hashAlg":"SHA2-512",
                    "saltLen":64
                  },
                  {
                    "hashAlg":"SHA3-224",
                    "saltLen":28
                  },
                  {
                    "hashAlg":"SHA3-256",
                    "saltLen":32
                  },
                  {
                    "hashAlg":"SHA3-384",
                    "saltLen":48
                  },
                  {
                    "hashAlg":"SHA3-512",
                    "saltLen":64
                  },
                  {
                    "hashAlg":"SHAKE-256",
                    "saltLen":32
                  }
                ]
              },
              {
                "modulo":3072,
                "maskFunction":[
                  "mgf1"
                ],
                "hashPair":[
                  {
                    "hashAlg":"SHA2-256",
                    "saltLen":32
                  },
                  {
                    "hashAlg":"SHA2-384",
                    "saltLen":48
                  },
                  {
                    "hashAlg":"SHA2-512",
                    "saltLen":64
                  },
                  {
                    "hashAlg":"SHA3-256",
                    "saltLen":32
                  },
                  {
                    "hashAlg":"SHA3-384",
                    "saltLen":48
                  },
                  {
                    "hashAlg":"SHA3-512",
                    "saltLen":64
                  },
                  {
                    "hashAlg":"SHAKE-256",
                    "saltLen":32
                  }
                ]
              },
              {
                "modulo":4096,
                "maskFunction":[
                  "mgf1"
                ],
                "hashPair":[
                  {
                    "hashAlg":"SHA2-384",
                    "saltLen":48
                  },
                  {
                    "hashAlg":"SHA2-512",
                    "saltLen":64
                  },
                  {
                    "hashAlg":"SHA3-384",
                    "saltLen":48
                  },
                  {
                    "hashAlg":"SHA3-512",
                    "saltLen":64
                  }
                ]
              }
            ]
          }

Endpoint in which the error is experienced
https://demo.acvts.nist.gov/acvp/v1/testSessions GET

Expected behavior
For RSA/sigGen where the digest was SHA3 and sigType: pkcs1v1.5
the expected results returned by ACVP server contained a DigestInfo structure with the OID of the SHA2 algorithm (not the SHA3 algorithm as expected)

Additional context
For example:
Expected result:
"tcId": 37, "signature": "710E8AE0047954FA798E74F020F952142F8291D355E35DB8350946C6DE556046659074A54E25F79BC34B5E8B8B612F0C060F8D0A3780541BDF3BF50313B3CEABFA5B1D017DB5B9066FD2FABEC50B7AB643049EDD09C590B61CBBAE261ACBE64BB6EC13C40545D44C68520B5BFFFC4C9558DBC4E2F5EB89AFCB29077E24C812CA77EE4B8FC39654B4EA4E7B0886CF51243C3E082F038EC3C6375BF37BB7B4C9EE6BC9A11B5AB74741DEACCBC32F1739069CC58F0C0A114637579934E77148A2B1111330A443B4B950BC9290E338AB80A0832071FB3DB52049707900A89ABAC151A8284632D9FC6038054C08974D16E3B18D5B83923BDE122855AA971C10E5D6F80A9C2AF052B9C19A844D0C516287FE70F4F88B0D76B81A0A3EAE6118D99A67E215A153ECC91CFD8F56FDD64E33003BE5EF9898E2A8BE79207F4710E22BB7F76C9016E8297993192DAC879DBCFA7EF724147E91A021298C2C7A11620D2A0D204448B967F00E861C077E1326849B2481BC1D39C92B5D555811058EB80C9A68E27C" },
The request testvector for tcId: 37 has "hashAlg": "SHA3-256",
In the decoded DigestInfo structure the ASN.1 structure:

 algor: 
    algorithm: sha256 (2.16.840.1.101.3.4.2.1)
    parameter: NULL
  digest: 
    0000 - c9 b8 85 23 ff 04 9d ed-81 81 b8 28 5f ba fc 86   ...#.......(_...
    0010 - da 14 e4 02 fc d0 af 4d-d8 17 83 f7 9f 51 b2 74   .......M.....Q.t

OID 2.16.840.1.101.3.4.2.1 is the OID of SHA2-256, which is expected to be 2.16.840.1.101.3.4.2.8 (SHA3-256)

[livebe01]

Thanks @szendros. Btw, same issue with RSA sigVer. We'll get this fixed

[livebe01]

The fix for this is now on Demo in release v1.1.0.29.

[sandor-szendro-i4p]

I tried RSA/sigGen after the fix, and we stil have a problem for SHA3 signatures.
We retrieved expected values using the "sample" switch and we found the following issue:
in the signature the following ASN.1 can be found:
0:d=0 hl=2 l= 43 cons: SEQUENCE
2:d=1 hl=2 l= 11 cons: SEQUENCE
4:d=2 hl=2 l= 9 prim: OBJECT :2.16.840.1.101.3.4.2.7
15:d=1 hl=2 l= 0 prim: NULL

the length of the sequence that contains the OID seems to be off by 2 bytes

we think it should be:
0:d=0 hl=2 l= 45 cons: SEQUENCE
2:d=1 hl=2 l= 13 cons: SEQUENCE
4:d=2 hl=2 l= 9 prim: OBJECT :2.16.840.1.101.3.4.2.7

OpenSSL parser also throws an error:
Error in encoding
23340:error:0D07209B:asn1 encoding routines:ASN1_get_object:too long:.\crypto\asn1\asn1_lib.c:147:

This problem can be encountered only for SHA3 OIDs it is all right for SHA2 OIDs.

Can you please confirm if this error is on the ACVP side?

[jbrock24]

@sandor-szendro-i4p I'm looking into this.

[jbrock24]

OK, I see the error and have fixed it, sorry about that. It will be out with the next HOTFIX ASAP.

[sandor-szendro-i4p]

@jbrock24 Thank you

[livebe01]

The fix for this is on Demo with release v1.1.0.29-hotfix-1

[sandor-szendro-i4p]

Thanks, it's working.

[livebe01]

The fix for this is on Prod in release v1.1.0.29-hotfix-1.