Is it possible that SPHINCS+ algorithm is faster than Dilithium for larger files?

[ddziechciarz]

Hello, I've been performing some benchmarks of liboqs signature algorithms and noticed that for larger files (around 1GB) SPHINCS+-SHA2-256f-simple algorithm seems to be performing signing operation faster than Dilithium5. I was expecting it to be other way around, especially taking into consideration benchmarks on official liboqs website - clearly showing better performance of Dilithium5. Following image showcases signing and verification times for claimed NIST level 5 algorithms. The script has been ran 100 times, and later results collected. To remove any errors, I've also performed 10 "warmup" iterations before each run.

I've been testing liboqs and its different wrappers, and I achieved the same result in C, C++, golang and Python.

To further test it, I've used speed_sig.c script with message_len modified to 1000000000 and duration of each test set to 10 seconds, once again yielding similar results.

I was wondering whether it's possible that SPHINCS+-SHA2-256f-simple actually scales better with input size or it might be caused by some environmental factors. I'm interested if someone performed similar measurements for larger (1GB) file sizes?

[SWilson4]

The difference in performance between SPHINCS+-SHA2-* and SPHINCS+-SHAKE-* leads me to believe that SHA2 runs significantly faster than SHA3 on your system. This could explain why Dilithium lags behind—it makes heavy use of SHA3.

May I ask what system you're running on and how you've configured liboqs? (i.e., the build command you used)

[ddziechciarz]

Hello, I'm running these tests on docker container with Ubuntu 24.04 LTS, I've been following build instruction in README file. I'm also having same results with python wrapper running on my Windows 10 PC. If it's helpful, I'm using AMD Ryzen 5 5600X processor

[baentsch]

Two thoughts/questions: 1) Why not use a profiler to take a look where most time is spent? 2) What's the reason for not hashing files that large first?

[ddziechciarz]

Hello, I must say I'm not very proficient with a profiler and I wanted to have a large set of data to also calculate average and errors (hence 100 iterations). As for hashing, this code will not be used in any application and it would probably be a better option to hash the file first, I just wanted to know how each algorithm would scale with different input sizes