Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KubeFATE support FATE v2.0.0-beta #921

Merged
merged 4 commits into from
Nov 27, 2023
Merged

KubeFATE support FATE v2.0.0-beta #921

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9a9b93f
Support FATE v2.0.0-beta
owlet42 Nov 13, 2023
10f8456
Update KubeFATE version
owlet42 Nov 24, 2023
86c1b73
fix docker compose healthcheck
owlet42 Nov 24, 2023
71bd53f
fix pulsar bug
owlet42 Nov 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions build/ci/docker-deploy/docker_deploy.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ tar -xzf confs-${target_party_id}.tar
cd confs-${target_party_id}
docker compose down
docker volume rm -f confs-${target_party_id}_shared_dir_examples
docker volume rm -f confs-${target_party_id}_shared_dir_federatedml
docker volume rm -f confs-${target_party_id}_shared_dir_fate
# exclude client service to save time !
docker compose up -d

Expand All @@ -47,8 +47,8 @@ for ((i = 1; i <= MAX_TRY; i++)); do
result=$(docker ps | wc -l)
if [ "${result}" -eq ${CONTAINER_NUM} ]; then
echo "# containers are ok"
FATE_FLOW_STATUS=$(curl -s -X POST localhost:9380/v1/version/get)
success='"retmsg":"success"'
FATE_FLOW_STATUS=$(curl -s -X GET localhost:9380/v2/server/fateflow)
success='"message":"success"'
result=$(echo $FATE_FLOW_STATUS | grep "${success}")
if [[ "$result" != "" ]]
then
Expand Down
33 changes: 32 additions & 1 deletion docker-deploy/.env
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,40 @@
RegistryURI=
TAG=1.11.2-release
SERVING_TAG=2.1.6-release
SSH_PORT=22

# PREFIX: namespace on the registry's server.
# RegistryURI: address of the local registry
# TAG: tag of module images.
# SSH_PORT: port of SSH, default 22


KubeFATE_Version=v2.0.0-beta

# components version

FATEFlow_IMAGE="federatedai/fateflow"
FATEFlow_IMAGE_TAG="v2.0.0-beta"
FATEBoard_IMAGE="federatedai/fateboard"
FATEBoard_IMAGE_TAG="v2.0.0-beta"
MySQL_IMAGE="mysql"
MySQL_IMAGE_TAG="8.0.28"
Client_IMAGE="federatedai/client"
Client_IMAGE_TAG="v2.0.0-beta"

EGGRoll_IMAGE="federatedai/eggroll"
EGGRoll_IMAGE_TAG="v2.0.0-beta"

Nginx_IMAGE="federatedai/nginx"
Nginx_IMAGE_TAG="v2.0.0-beta"
RabbitMQ_IMAGE="federatedai/rabbitmq"
RabbitMQ_IMAGE_TAG="3.8.3-management"
Pulsar_IMAGE="federatedai/pulsar"
Pulsar_IMAGE_TAG="2.10.2"
Hadoop_NameNode_IMAGE="federatedai/hadoop-namenode"
Hadoop_NameNode_IMAGE_TAG="2.0.0-hadoop3.2.1-java8"
Hadoop_DataNode_IMAGE="federatedai/hadoop-datanode"
Hadoop_DataNode_IMAGE_TAG="2.0.0-hadoop3.2.1-java8"
Spark_Master_IMAGE="federatedai/spark-master"
Spark_Master_IMAGE_TAG="v2.0.0-beta"
Spark_Worker_IMAGE="federatedai/spark-worker"
Spark_Worker_IMAGE_TAG="v2.0.0-beta"
12 changes: 6 additions & 6 deletions docker-deploy/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,13 +192,13 @@ The output is shown as follows. If the status of each component is `Up`, and the

```bash
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
confs-10000-client-1 federatedai/client:1.11.2-release "bash -c 'pipeline i…" client About a minute ago Up About a minute 0.0.0.0:20000->20000/tcp, :::20000->20000/tcp
confs-10000-clustermanager-1 federatedai/eggroll:1.11.2-release "/tini -- bash -c 'j…" clustermanager About a minute ago Up About a minute 4670/tcp
confs-10000-fateboard-1 federatedai/fateboard:1.11.2-release "/bin/sh -c 'java -D…" fateboard About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp
confs-10000-fateflow-1 federatedai/fateflow:1.11.2-release "/bin/bash -c 'set -…" fateflow About a minute ago Up About a minute (healthy) 0.0.0.0:9360->9360/tcp, :::9360->9360/tcp, 0.0.0.0:9380->9380/tcp, :::9380->9380/tcp
confs-10000-client-1 federatedai/client:v2.0.0-beta "bash -c 'pipeline i…" client About a minute ago Up About a minute 0.0.0.0:20000->20000/tcp, :::20000->20000/tcp
confs-10000-clustermanager-1 federatedai/eggroll:v2.0.0-beta "/tini -- bash -c 'j…" clustermanager About a minute ago Up About a minute 4670/tcp
confs-10000-fateboard-1 federatedai/fateboard:v2.0.0-beta "/bin/sh -c 'java -D…" fateboard About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp
confs-10000-fateflow-1 federatedai/fateflow:v2.0.0-beta "/bin/bash -c 'set -…" fateflow About a minute ago Up About a minute (healthy) 0.0.0.0:9360->9360/tcp, :::9360->9360/tcp, 0.0.0.0:9380->9380/tcp, :::9380->9380/tcp
confs-10000-mysql-1 mysql:8.0.28 "docker-entrypoint.s…" mysql About a minute ago Up About a minute 3306/tcp, 33060/tcp
confs-10000-nodemanager-1 federatedai/eggroll:1.11.2-release "/tini -- bash -c 'j…" nodemanager About a minute ago Up About a minute 4671/tcp
confs-10000-rollsite-1 federatedai/eggroll:1.11.2-release "/tini -- bash -c 'j…" rollsite About a minute ago Up About a minute 0.0.0.0:9370->9370/tcp, :::9370->9370/tcp
confs-10000-nodemanager-1 federatedai/eggroll:v2.0.0-beta "/tini -- bash -c 'j…" nodemanager About a minute ago Up About a minute 4671/tcp
confs-10000-rollsite-1 federatedai/eggroll:v2.0.0-beta "/tini -- bash -c 'j…" rollsite About a minute ago Up About a minute 0.0.0.0:9370->9370/tcp, :::9370->9370/tcp
```

### Verifying the deployment
Expand Down
12 changes: 6 additions & 6 deletions docker-deploy/README_zh.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -231,13 +231,13 @@ docker compose ps

```bash
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
confs-10000-client-1 federatedai/client:1.11.2-release "bash -c 'pipeline i…" client About a minute ago Up About a minute 0.0.0.0:20000->20000/tcp, :::20000->20000/tcp
confs-10000-clustermanager-1 federatedai/eggroll:1.11.2-release "/tini -- bash -c 'j…" clustermanager About a minute ago Up About a minute 4670/tcp
confs-10000-fateboard-1 federatedai/fateboard:1.11.2-release "/bin/sh -c 'java -D…" fateboard About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp
confs-10000-fateflow-1 federatedai/fateflow:1.11.2-release "/bin/bash -c 'set -…" fateflow About a minute ago Up About a minute (healthy) 0.0.0.0:9360->9360/tcp, :::9360->9360/tcp, 0.0.0.0:9380->9380/tcp, :::9380->9380/tcp
confs-10000-client-1 federatedai/client:v2.0.0-beta "bash -c 'pipeline i…" client About a minute ago Up About a minute 0.0.0.0:20000->20000/tcp, :::20000->20000/tcp
confs-10000-clustermanager-1 federatedai/eggroll:v2.0.0-beta "/tini -- bash -c 'j…" clustermanager About a minute ago Up About a minute 4670/tcp
confs-10000-fateboard-1 federatedai/fateboard:v2.0.0-beta "/bin/sh -c 'java -D…" fateboard About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp
confs-10000-fateflow-1 federatedai/fateflow:v2.0.0-beta "/bin/bash -c 'set -…" fateflow About a minute ago Up About a minute (healthy) 0.0.0.0:9360->9360/tcp, :::9360->9360/tcp, 0.0.0.0:9380->9380/tcp, :::9380->9380/tcp
confs-10000-mysql-1 mysql:8.0.28 "docker-entrypoint.s…" mysql About a minute ago Up About a minute 3306/tcp, 33060/tcp
confs-10000-nodemanager-1 federatedai/eggroll:1.11.2-release "/tini -- bash -c 'j…" nodemanager About a minute ago Up About a minute 4671/tcp
confs-10000-rollsite-1 federatedai/eggroll:1.11.2-release "/tini -- bash -c 'j…" rollsite About a minute ago Up About a minute 0.0.0.0:9370->9370/tcp, :::9370->9370/tcp
confs-10000-nodemanager-1 federatedai/eggroll:v2.0.0-beta "/tini -- bash -c 'j…" nodemanager About a minute ago Up About a minute 4671/tcp
confs-10000-rollsite-1 federatedai/eggroll:v2.0.0-beta "/tini -- bash -c 'j…" rollsite About a minute ago Up About a minute 0.0.0.0:9370->9370/tcp, :::9370->9370/tcp
```

### 验证部署
Expand Down
2 changes: 1 addition & 1 deletion docker-deploy/docker_deploy.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ tar -xzf confs-$target_party_id.tar
cd confs-$target_party_id
docker compose down
docker volume rm -f confs-${target_party_id}_shared_dir_examples
docker volume rm -f confs-${target_party_id}_shared_dir_federatedml
docker volume rm -f confs-${target_party_id}_shared_dir_fate
docker volume rm -f confs-${target_party_id}_sdownload_dir
docker volume rm -f confs-${target_party_id}_fate_flow_logs

Expand Down
Loading
Loading