-
Notifications
You must be signed in to change notification settings - Fork 98
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
## Contents of this release [FEAT]: support for multi-device execution: #356 [FEAT]: full support for new mixed-radix NTT: #367, #368 and #371 [FEAT]: examples for Poseidon hash and tree builder based on it (currently only on C++ side): #375 [PERF]: MSM performance upgrades & zero point handling: #372
- Loading branch information
Showing
57 changed files
with
1,661 additions
and
883 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,5 @@ | ||
inout | ||
crate | ||
lmit | ||
mut | ||
uint |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
cmake_minimum_required(VERSION 3.18) | ||
set(CMAKE_CXX_STANDARD 17) | ||
set(CMAKE_CUDA_STANDARD 17) | ||
set(CMAKE_CUDA_STANDARD_REQUIRED TRUE) | ||
set(CMAKE_CXX_STANDARD_REQUIRED TRUE) | ||
if (${CMAKE_VERSION} VERSION_LESS "3.24.0") | ||
set(CMAKE_CUDA_ARCHITECTURES ${CUDA_ARCH}) | ||
else() | ||
set(CMAKE_CUDA_ARCHITECTURES native) # on 3.24+, on earlier it is ignored, and the target is not passed | ||
endif () | ||
project(icicle LANGUAGES CUDA CXX) | ||
|
||
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} --expt-relaxed-constexpr") | ||
set(CMAKE_CUDA_FLAGS_RELEASE "") | ||
set(CMAKE_CUDA_FLAGS_DEBUG "${CMAKE_CUDA_FLAGS_DEBUG} -g -G -O0") | ||
# change the path to your Icicle location | ||
include_directories("../../../icicle") | ||
add_executable( | ||
example | ||
example.cu | ||
) | ||
|
||
find_library(NVML_LIBRARY nvidia-ml PATHS /usr/local/cuda-12.0/targets/x86_64-linux/lib/stubs/ ) | ||
target_link_libraries(example ${NVML_LIBRARY}) | ||
set_target_properties(example PROPERTIES CUDA_SEPARABLE_COMPILATION ON) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
# Icicle example: build a Merkle tree using Poseidon hash | ||
|
||
## Best-Practices | ||
|
||
We recommend to run our examples in [ZK-containers](../../ZK-containers.md) to save your time and mental energy. | ||
|
||
## Key-Takeaway | ||
|
||
`Icicle` provides CUDA C++ template `poseidon_hash` to accelerate the popular [Poseidon hash function](https://www.poseidon-hash.info/). | ||
|
||
## Concise Usage Explanation | ||
|
||
```c++ | ||
#include "appUtils/poseidon/poseidon.cu" | ||
... | ||
poseidon_hash<scalar_t, arity+1>(input, output, n, constants, config); | ||
``` | ||
|
||
**Parameters:** | ||
|
||
- **`scalar_t`:** a scalar field of the selected curve. | ||
You can think of field's elements as 32-byte integers modulo `p`, where `p` is a prime number, specific to this field. | ||
|
||
- **arity:** number of elements in a hashed block. | ||
|
||
- **n:** number of blocks we hash in parallel. | ||
|
||
- **input, output:** `scalar_t` arrays of size $arity*n$ and $n$ respectively. | ||
|
||
- **constants:** are defined as below | ||
|
||
```c++ | ||
device_context::DeviceContext ctx= device_context::get_default_device_context(); | ||
PoseidonConstants<scalar_t> constants; | ||
init_optimized_poseidon_constants<scalar_t>(ctx, &constants); | ||
``` | ||
|
||
## What's in the example | ||
|
||
1. Define the size of the example: the height of the full binary Merkle tree. | ||
2. Hash blocks in parallel. The tree width determines the number of blocks to hash. | ||
3. Build a Merkle tree from the hashes. | ||
4. Use the tree to generate a membership proof for one of computed hashes. | ||
5. Validate the hash membership. | ||
6. Tamper the hash. | ||
7. Invalidate the membership of the tempered hash. | ||
|
||
## Details | ||
|
||
### Merkle tree structure | ||
|
||
Our Merkle tree is a **full binary tree** stored in a 1D array. | ||
The tree nodes are stored following a level-first traversal of the binary tree. | ||
For a given level, we use offset to number elements from left to right. The node numbers on the figure below correspond to their locations in the array. | ||
|
||
```text | ||
Tree Level | ||
0 0 | ||
/ \ | ||
1 2 1 | ||
/ \ / \ | ||
3 4 5 6 2 | ||
1D array representation: {0, 1, 2, 3, 4, 5, 6} | ||
``` | ||
|
||
### Membership proof structure | ||
|
||
We use two arrays: | ||
|
||
- position (left/right) of the node along the path toward the root | ||
- hash of a second node with the same parent |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
#!/bin/bash | ||
|
||
# Exit immediately on error | ||
set -e | ||
|
||
rm -rf build | ||
mkdir -p build | ||
cmake -S . -B build | ||
cmake --build build |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,152 @@ | ||
#include <chrono> | ||
#include <fstream> | ||
#include <iostream> | ||
|
||
// select the curve | ||
#define CURVE_ID 2 | ||
// include Poseidon template | ||
#include "appUtils/poseidon/poseidon.cu" | ||
using namespace poseidon; | ||
using namespace curve_config; | ||
|
||
device_context::DeviceContext ctx= device_context::get_default_device_context(); | ||
|
||
// location of a tree node in the array for a given level and offset | ||
inline uint32_t tree_index(uint32_t level, uint32_t offset) { return (1 << level) - 1 + offset; } | ||
|
||
// We assume the tree has leaves already set, compute all other levels | ||
void build_tree( | ||
const uint32_t tree_height, scalar_t* tree, PoseidonConstants<scalar_t> * constants, PoseidonConfig config) | ||
{ | ||
for (uint32_t level = tree_height - 1; level > 0; level--) { | ||
const uint32_t next_level = level - 1; | ||
const uint32_t next_level_width = 1 << next_level; | ||
poseidon_hash<scalar_t, 2+1>( | ||
&tree[tree_index(level, 0)], &tree[tree_index(next_level, 0)], next_level_width, *constants, config); | ||
} | ||
} | ||
|
||
// linear search leaves for a given hash, return offset | ||
uint32_t query_membership(scalar_t query, scalar_t* tree, const uint32_t tree_height) | ||
{ | ||
const uint32_t tree_width = (1 << (tree_height - 1)); | ||
for (uint32_t i = 0; i < tree_width; i++) { | ||
const scalar_t leaf = tree[tree_index(tree_height - 1, i)]; | ||
if (leaf == query) { | ||
return i; // found the hash | ||
} | ||
} | ||
return tree_height; // hash not found | ||
} | ||
|
||
void generate_proof( | ||
uint32_t position, | ||
scalar_t* tree, | ||
const uint32_t tree_height, | ||
uint32_t* proof_lr, | ||
scalar_t* proof_hash) | ||
{ | ||
uint32_t level_index = position; | ||
for (uint32_t level = tree_height - 1; level > 0; level--) { | ||
uint32_t lr; | ||
uint32_t neighbour_index; | ||
lr = level_index % 2; | ||
if (lr == 0) { | ||
// left | ||
neighbour_index = level_index + 1; | ||
} else { | ||
// right | ||
neighbour_index = level_index - 1; | ||
} | ||
proof_lr[level] = lr; | ||
proof_hash[level] = tree[tree_index(level, neighbour_index)]; | ||
level_index /= 2; | ||
} | ||
// the proof must match this: | ||
proof_hash[0] = tree[tree_index(0, 0)]; | ||
} | ||
|
||
uint32_t validate_proof( | ||
const scalar_t hash, | ||
const uint32_t tree_height, | ||
const uint32_t* proof_lr, | ||
const scalar_t* proof_hash, | ||
PoseidonConstants<scalar_t> * constants, | ||
PoseidonConfig config) | ||
{ | ||
scalar_t hashes_in[2], hash_out[1], level_hash; | ||
level_hash = hash; | ||
for (uint32_t level = tree_height - 1; level > 0; level--) { | ||
if (proof_lr[level] == 0) { | ||
hashes_in[0] = level_hash; | ||
hashes_in[1] = proof_hash[level]; | ||
} else { | ||
hashes_in[0] = proof_hash[level]; | ||
hashes_in[1] = level_hash; | ||
} | ||
// next level hash | ||
poseidon_hash<scalar_t, 2+1>(hashes_in, hash_out, 1, *constants, config); | ||
level_hash = hash_out[0]; | ||
} | ||
return proof_hash[0] == level_hash; | ||
} | ||
|
||
int main(int argc, char* argv[]) | ||
{ | ||
std::cout << "1. Defining the size of the example: height of the full binary Merkle tree" << std::endl; | ||
const uint32_t tree_height = 21; | ||
std::cout << "Tree height: " << tree_height << std::endl; | ||
const uint32_t tree_arity = 2; | ||
const uint32_t leaf_level = tree_height - 1; | ||
const uint32_t tree_width = 1 << leaf_level; | ||
std::cout << "Tree width: " << tree_width << std::endl; | ||
const uint32_t tree_size = (1 << tree_height) - 1; | ||
std::cout << "Tree size: " << tree_size << std::endl; | ||
scalar_t* tree = static_cast<scalar_t*>(malloc(tree_size * sizeof(scalar_t))); | ||
|
||
std::cout << "2. Hashing blocks in parallel" << std::endl; | ||
const uint32_t data_arity = 4; | ||
std::cout << "Block size (arity): " << data_arity << std::endl; | ||
std::cout << "Initializing blocks..." << std::endl; | ||
scalar_t d = scalar_t::zero(); | ||
scalar_t* data = static_cast<scalar_t*>(malloc(tree_width * data_arity * sizeof(scalar_t))); | ||
for (uint32_t i = 0; i < tree_width * data_arity; i++) { | ||
data[i] = d; | ||
d = d + scalar_t::one(); | ||
} | ||
std::cout << "Hashing blocks into tree leaves..." << std::endl; | ||
PoseidonConstants<scalar_t> constants; | ||
init_optimized_poseidon_constants<scalar_t>(data_arity, ctx, &constants); | ||
PoseidonConfig config = default_poseidon_config<scalar_t>(data_arity+1); | ||
poseidon_hash<curve_config::scalar_t, data_arity+1>(data, &tree[tree_index(leaf_level, 0)], tree_width, constants, config); | ||
|
||
std::cout << "3. Building Merkle tree" << std::endl; | ||
PoseidonConstants<scalar_t> tree_constants; | ||
init_optimized_poseidon_constants<scalar_t>(tree_arity, ctx, &tree_constants); | ||
PoseidonConfig tree_config = default_poseidon_config<scalar_t>(tree_arity+1); | ||
build_tree(tree_height, tree, &tree_constants, tree_config); | ||
|
||
std::cout << "4. Generate membership proof" << std::endl; | ||
uint32_t position = tree_width - 1; | ||
std::cout << "Using the hash for block: " << position << std::endl; | ||
scalar_t query = tree[tree_index(leaf_level, position)]; | ||
uint32_t query_position = query_membership(query, tree, tree_height); | ||
// allocate arrays for the proof | ||
uint32_t* proof_lr = static_cast<uint32_t*>(malloc(tree_height * sizeof(uint32_t))); | ||
scalar_t* proof_hash = static_cast<scalar_t*>(malloc(tree_height * sizeof(scalar_t))); | ||
generate_proof(query_position, tree, tree_height, proof_lr, proof_hash); | ||
|
||
std::cout << "5. Validate the hash membership" << std::endl; | ||
uint32_t validated; | ||
const scalar_t hash = tree[tree_index(leaf_level, query_position)]; | ||
validated = validate_proof(hash, tree_height, proof_lr, proof_hash, &tree_constants, tree_config); | ||
std::cout << "Validated: " << validated << std::endl; | ||
|
||
std::cout << "6. Tamper the hash" << std::endl; | ||
const scalar_t tampered_hash = hash + scalar_t::one(); | ||
validated = validate_proof(tampered_hash, tree_height, proof_lr, proof_hash, &tree_constants, tree_config); | ||
|
||
std::cout << "7. Invalidate tamper hash membership" << std::endl; | ||
std::cout << "Validated: " << validated << std::endl; | ||
return 0; | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
#!/bin/bash | ||
./build/example |
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.