SecretFlow PSI Library

The repo of Private Set Intersection(PSI) and Private Information Retrieval(PIR) from SecretFlow.

This repo is formerly psi/pir part from secretflow/spu repo.

PSI Quick Start with v2 API

For PSI v1 API and PIR, please check documentation.

Release Docker

In the following example, we are going to run PSI at a single host.

1. Check official release docker image at dockerhub. We also have mirrors at Alibaba Cloud: secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/psi-anolis8.

2. Prepare data and config.

receiver.config:

{
    "protocol_config": {
        "protocol": "PROTOCOL_KKRT",
        "role": "ROLE_RECEIVER",
        "broadcast_result": true
    },
    "input_config": {
        "type": "IO_TYPE_FILE_CSV",
        "path": "/root/receiver/receiver_input.csv"
    },
    "output_config": {
        "type": "IO_TYPE_FILE_CSV",
        "path": "/root/receiver/receiver_output.csv"
    },
    "link_config": {
        "parties": [
            {
                "id": "receiver",
                "host": "127.0.0.1:5300"
            },
            {
                "id": "sender",
                "host": "127.0.0.1:5400"
            }
        ]
    },
    "self_link_party": "receiver",
    "keys": [
        "id0",
        "id1"
    ],
    "debug_options": {
        "trace_path": "/root/receiver/receiver.trace"
    },
    "check_duplicates": false,
    "sort_output": false,
    "recovery_config": {
        "enabled": false,
    }
}

sender.config:

{
    "protocol_config": {
        "protocol": "PROTOCOL_KKRT",
        "role": "ROLE_SENDER",
        "broadcast_result": true
    },
    "input_config": {
        "type": "IO_TYPE_FILE_CSV",
        "path": "/root/sender/sender_input.csv"
    },
    "output_config": {
        "type": "IO_TYPE_FILE_CSV",
        "path": "/root/sender/sender_output.csv"
    },
    "link_config": {
        "parties": [
            {
                "id": "receiver",
                "host": "127.0.0.1:5300"
            },
            {
                "id": "sender",
                "host": "127.0.0.1:5400"
            }
        ]
    },
    "self_link_party": "sender",
    "keys": [
        "id0",
        "id1"
    ],
    "debug_options": {
        "trace_path": "/root/sender/sender.trace"
    },
    "check_duplicates": false,
    "sort_output": false,
    "recovery_config": {
        "enabled": false,
    }
}

File Name	Location	Description
receiver.config	/tmp/receiver/receiver.config	Config for receiver.
sender.config	/tmp/sender/sender.config	Config for sender.
receiver_input.csv	/tmp/receiver/receiver_input.config	Input for receiver. Make sure the file contains two id keys - id0 and id1.
sender_input.csv	/tmp/sender/sender_input.config	Input for sender. Make sure the file contains two id keys - id0 and id1.

3. Run PSI

In the first terminal, run the following command

docker run -it  --rm  --network host --mount type=bind,source=/tmp/receiver,target=/root/receiver -w /root  --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --cap-add=NET_ADMIN --privileged=true secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/psi-anolis8:0.1.0beta bash -c "./main --config receiver/receiver.config"

In the other terminal, run the following command simultaneously.

docker run -it  --rm  --network host --mount type=bind,source=/tmp/sender,target=/root/sender -w /root  --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --cap-add=NET_ADMIN --privileged=true secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/psi-anolis8:0.1.0beta bash -c "./main --config sender/sender.config"

Building SecretFlow PSI Library

System Setup

Dev Docker

We use the same dev docker from secretflow/ubuntu-base-ci.

# start container
docker run -d -it --name psi-dev-$(whoami) \
         --mount type=bind,source="$(pwd)",target=/home/admin/dev/ \
         -w /home/admin/dev \
         --cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
         --cap-add=NET_ADMIN \
         --privileged=true \
         --entrypoint="bash" \
         secretflow/ubuntu-base-ci:latest

# attach to build container
docker exec -it psi-dev-$(whoami) bash

Linux

Install gcc>=11.2, cmake>=3.26, ninja, nasm>=2.15, python>=3.8, bazel==6.2.1, golang, xxd, lld

Build & UnitTest

# build as debug
bazel build //... -c dbg

# build as release
bazel build //... -c opt

# test
bazel test //...

Trace

We use Perfetto from Google for tracing.

Please use debug_options.trace_path field in PsiConfig to modify trace file path. The default path is /tmp/psi.trace.

After running psi binaries, please check trace by using Trace Viewer. If this is not applicable, please check this link to deploy your own website.

The alternate way to visualize trace is to use chrome://tracing:

1. Download perfetto assets from https://github.com/google/perfetto/releases/tag/v37.0
2. You should find traceconv binary in assets folder.
3. Transfer trace file to JSON format:

chmod +x traceconv

./traceconv json [trace file path] [json file path]

4. Open chrome://tracing in your chrome and load JSON file.