-
Notifications
You must be signed in to change notification settings - Fork 180
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implementation: Test vectors #62
Comments
Could you maybe also add a few test vectors for the SK -> AES-CTR key hmac ? |
Sure! They are now included above below the line |
I am still having problems recreating your ephemeral. From issue #57 it was pointed out that the IV starts at 0, but I suspect that you have started from 1 - still this is not the only discrepancy that I am dealing with. |
I have included some of these test vectors in the unit tests for the Rust library here : https://github.com/snakehand/dp-3t-client - but the epehemeral IV is is still off by 1 compared to Zenroom. |
@jaromil , @snakehand - having trouble generating exactly the same with Apple's crypto. Could either of you be so kind to describe how this "n * 10000" string is serialised to uint8_t's ? I.e some slightly lower level pseudo code ? |
I think you are looking at older code, the IV is a simple counter now, and nonce=0. In Rust I do this: let mut serial = [0u8; 16];
serial[12..16].copy_from_slice(&i.to_be_bytes());
let mut block = GenericArray::clone_from_slice(&serial);
cipher.encrypt_block(&mut block); I.e just copy the counter to big endian bytes at the end of the 16 byte AES input block. |
So I see you do Aes256::new( ... with the key as HMAC of Sk and a password = "Decen.." You then do serial = 16 bytes; top 4 bytes contain a 32 bit/4byte unsigned integer in Big endian order. And AES encrypt this rolling. And the result is again 16 bytes. Correct ? |
(This comment has been edited to sync to the reference implementation released recently.) The following Kotlin results in the same output as the reference implementation (see DP-3T/reference_implementation#9). fun ephIDs(n: Int): List<EphID> {
val sha256_HMAC: Mac = Mac.getInstance("HmacSHA256")
sha256_HMAC.init(SecretKeySpec(sk, "HmacSHA256"))
val prf = sha256_HMAC.doFinal("Broadcast key".toByteArray())
val cipher = Cipher.getInstance("AES/CTR/NoPadding")
val b = ByteArray(16 * n)
cipher.init(Cipher.ENCRYPT_MODE, SecretKeySpec(prf, "AES"), IvParameterSpec(ByteArray(16)))
cipher.update(b, 0, b.size, b, 0)
var out = mutableListOf<EphID>();
for (i in 0 until n) {
out.add(EphID(b.sliceArray(i*16 until i*16+16)));
}
return out
} This was useful to me in confirming that the performance would be manageable on low-spec Android phones. |
Ok - so in C this would be:
and I take it that IvParameterSpec(ByteArray(16) is simply an IV of 16 bytes that are 0x0 each ? And we then create 16 byte of zero's plaintext blocks with cipher.update(). Correct. What is your second EphiID ? |
I am no Java / Kotlin (?) expert but I think MessageDigest.getInstance("SHA-256") yields the wrong digest. You want to somehow use HMAC-SHA256 with SK as the key, and not just feed it into the digest algorithm |
To avoid further ambiguity I have updated the first post in this issue with the actual code used and with two columns for the EphID vectors: left shows the counter (used as |
@jaromil I think you have the IVs backwards, your loop is counting down in : https://github.com/DECODEproject/Zenroom/blob/master/src/lua/zencode_dp3t.lua |
I updated https://github.com/snakehand/dp-3t-client with a small example that prints some test vectors, including the 0 IV:
|
Happy to see we aligned! |
Lovely - I'll update the C/Swift/ObjC to match. Meanwhile I've put up a small fork of this repo with a branch with temporary editable version of the 3 documents - so it is easier to make notes about details like this in those documents. Reverse engineered/copy-paste job (https://github.com/dirkx/DP-3T-Documents/tree/editable-version -- directory 'src'). |
Hmm - not having much luck; the plaintext is 16 x 0 bytes for each Eph_id ? Below is the code I am using (runnable version at https://gist.github.com/dirkx/53143596fa935b6de96e6521d82797b6) And correct that the sk is used as the key in the HMAC; and that it sighs the "Decentralized.. " string ? And internally rust/Zenroom does the 14 round cycle on the key to expand it, etc ?
|
I have now added a C library wrapper to my Rust code (check out clib) https://github.com/snakehand/dp-3t-client - This should make the code that I have very portable, and could possibly be linked straight into a mobile app on Android / iOS. I have tested performance on a Raspberry Pi, and it seems OK. ( This library is self contained and has no dependencies on ssl libraries. ) |
Also Zenroom is ported to react-native android/iOS and we ran some benchmarks on rpi3. I am curious about performance, would you share benchmarks? my test manages to find a single matching SK among 20.000 others in 11 seconds. |
I ran this test on a single core on a Raspberry Pi 4 in around 1 second: fn speed_test() {
let mut key: [u8; 32] = [0; 32];
let mut total = 0;
for i in 0..1000_u32 {
key[28..32].copy_from_slice(&i.to_be_bytes());
let rp = ReplayKey::new(0, 14, 8, &key);
total += rp.fold(0_u64, |s, _e| s + 1);
}
println!("{} ephemerals", total);
} This test generates 1000 * 14 * 8 ephemeral IDs , but without any lookup. ( Lookup should be a O( log n ) operation, and not add a whole lot time ) I should add that the crypto implementations are constant time to prevent leaking secrets. I am not sure how this affects performance. |
@snakehand @jaromil @jeffallen @cascremers @carmelatroncoso - tried to capture all what was said here and on slack in PR #117 in terms of test vectors and exact protocol interpretations. I.e. one that takes away all 'e.g.' in the documents. |
With the arrival of the official reference implementation, this issue needs to be updated with test vectors from it, and other implementations should sync to them. See DP-3T/reference_implementation#9 for proposed test vectors to use. |
Hi Jeff, yes I see the reference implementation in python now, it generates EphIDs segmenting the output of a bigger chunk of data out of AES-CTR. I will do the same and update my vectors ASAP. |
@jaromil, @snakehand - did you manage ? We cannot get it to work and 3 different people now all independently got the same values, in 3 languages, but not those of the team python output. So either we are all making the exact same mistake/misreading - or python/Crypthome is special. See https://lists.dlitz.net/pipermail/pycrypto/2020/000909.html for my summary. |
Hey @dirkx, |
Goodmorning everyone! I have updated the vectors here using both the broadcast key with and without BOM prefix. |
@jaromil - where is the code ? |
It is in the first post of the issue: Zenroom is a virtual machine without external dependencies using Lua as direct-syntax parser, Milagro as crypto backend and its own Zencode language as DSL. Here you find the Zencode implementation inside Zenroom I paste the Lua code from: https://github.com/DECODEproject/Zenroom/blob/master/src/lua/zencode_dp3t.lua More development information about Zenroom is at https://dev.zenroom.org |
And with BOM you mean a UTF-8 like apple has in front of the broadcast key ? |
Yes, I provided that too since I noticed their design has changed that detail so useful to have both. |
Why is a BOM marker even required ? Should the implementation have to handle different byte orderings ? UTF-8 does not require a BOM, since the byte ordering is unambiguous. UTF-16 and UTF-32 does though, but you also need to specify a word size for a BOM to make sense. What is the word size that this BOM refers to ? 16,32,64,128,256 bits ? - Having to use this type of "solution" at all seems a bit alien to me. It is better to have a solid spec that does not leave any openings for byte ordering ambiguities. |
To me too, but then please talk some sense to them, I'm just sticking to my plan to provide test vectors ;^) I have added a note about this weirdness. |
The broadcast key used by Zenroom and that used by the DP-3T reference implementation, and the DP-3T app all differ: I just filed an issue about straightening out the latter two, here: DP-3T/reference_implementation#11 (I do not see any indication of BOM in either the spec or the two DP-3T implementations. Where are you guys seeing that?) |
I didn't push the dp-3t zenroom scenario update yet, did that in dyne/Zenroom@fc72b32 the broadcast key is set in zencode (as an external variable lets say). Regarding: I confirm running both scripts we have matching vectors! 🤠 |
@snakehand w.r.t. to the BOM. Completely agreed. There is no need for that key to be anything but a well defined, easy to not mis-implement, byte sequence (with no entropy requirements). So having it a pure US-ASCII, 7 bit safe, printable (32 .. 126) thing is goodness. With no \0 or any other termination. So if it is 'Broadcast key' it is exactly those 13 chars from B to and including y. And that is it. And yes - that does mean that modern fancy languages need to be a bit careful or define it as something like seed = [ 0x42, 0x72, 0x6f, 0x61, 0x64, 0x63, 0x61, 0x73, 0x74, 0x20, 0x6b, 0x65, 0x79 ] if their internal string representation is fancy with BOMs and full unicode normalisation/ligatures/etc. |
|
I've double checked my results using an online tool (cryptii.com) and everything looks correct from my side. My test vector:
Still a mistery to me how we are getting these other vectors from @dirkx's python branch, note that PRF is identical, but I am not sure about how Cryptodome's AES parameters work:
|
From this website |
I've found the issue. Everyone here is using AES256 with a 128bit key, and truncating all the blocks in half. Can you clarify this is the intended behavior? I was simply assuming that AES128 would produce the intended results. In other words, this is what you are doing at the moment:
First 256-bit EPHID as result of the above: Discard the lower 128 bits: Can you please indicate whether this is intended or not, and what is the rationale of using a 256-bit algorithms with all parameters cut down to 128 bits? Thanks, /d |
Isn't AES block size always 128 bits regardless of key size ? In my Rust code ( https://github.com/snakehand/dp-3t-client ) I can get both variants by changing between Aes128 and Aes256, but the truncation happens when the SHA256 output is is fed as a 128 bits AES key. ( The Rust AES code will actually cause a panic or fail to compile of I feed it a key of incorrect size, whereas C and possibly python might just silently truncate the key ) |
The mistake, I think is, I and some others, took the 128 bit as meaning AES128_CTR; whereas in fact it is AES256_CTR with the last 128 bit chopped off. Will verify and update / describe in the implementation notes. |
Thanks for confirming the exact algorithms used. I've fixed it on decode-proximity-hw, indeed I was assuming AES128 as I was confused by the TRUNCATE_128 method in the reference. Now the nRF52 device gives the same EphIds:
The code to generate EphIds on embedded is here |
I''ve brought the C and Python code in line with DP-3T/reference_implementation#10 -- and swift/java are fine too. And dirkx@9c08238 updated too. |
To help align with other implementations of this protocol I’m sharing my test vectors for design 1, which are the expected results of two cryptographic transformations in DP3T given known inputs.
Zenroom passes SHA256 FIPS140–2 and AES-CTR NIST compliancy and the vectors below match the reference implementation. ✔️
The Zenroom code used to derive a new SK is:
The Zenroom code used to derive EphIDs (see the dp3t scenario implementation) is:
Zencode used for ephids derivation:
More info about Zenroom is available at https://dev.zenroom.org
Vectors using the public broacast key: "Broadcast key" with BOM, in hexadecimals:
EFBBBF42726f616463617374206b6579
(:warning: its weird to have a BOM)The text was updated successfully, but these errors were encountered: