Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[wip] dm: fix integration test in rocky8.9 #11751

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
+14 −3
Open

[wip] dm: fix integration test in rocky8.9 #11751

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
f0ec446
add log
River2000i Nov 14, 2024
204b8c1
add dependency for validate IP addresses with SANs
River2000i Nov 15, 2024
81dfe81
retry when curl command get not zero
River2000i Nov 18, 2024
6c45472
add log
River2000i Nov 18, 2024
69b422c
add log
River2000i Nov 18, 2024
d40822f
debug
River2000i Nov 18, 2024
3f27bce
Fix TLS generation script to use RSA keys instead of EC keys
River2000i Nov 19, 2024
768db29
revet debug
River2000i Nov 19, 2024
6d18136
fix
River2000i Nov 19, 2024
df4ec61
revert
River2000i Nov 19, 2024
3966225
Revert "Fix TLS generation script to use RSA keys instead of EC keys"
River2000i Nov 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions dm/tests/_utils/run_downstream_cluster
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,15 @@ EOF
--data-dir "$WORK_DIR/pd" &
# wait until PD is online...
i=0
while ! curl "http://$PD_ADDR/pd/api/v1/version"; do
while curl "http://$PD_ADDR/pd/api/v1/version"; do
Copy link
Contributor Author

@River2000i River2000i Nov 19, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If curl return exit code is not 0, it will not retry such as:

Starting PD...

Release Version: v8.5.0-alpha-23-gc8c75510a

Edition: Community

Git Commit Hash: c8c75510a3583c669a962c41e6117157f3d19dca

Git Branch: master

UTC Build Time:  2024-11-14 10:14:13

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed


  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to 127.0.0.1 port 2379: Connection refused

script returned exit code 2

Seems check curl return without error(exit code is 0) is enough🤔
ref tiup: https://github.com/pingcap/tiup/blob/master/pkg/cluster/api/pdapi.go#L188-L195

i=$((i + 1))
if [ "$i" -gt 20 ]; then
echo 'Failed to start PD'
return 1
fi
sleep 3
done
echo 'Start PD success'
}

start_tikv() {
Expand All @@ -54,14 +55,15 @@ start_tikv() {
--log-file "$WORK_DIR/tikv.log" \
--log-level info \
-s "$WORK_DIR/tikv" &
while ! curl "http://$PD_ADDR/pd/api/v1/cluster/status" | tee /dev/stderr | grep '"is_initialized": true'; do
while curl "http://$PD_ADDR/pd/api/v1/cluster/status" | tee /dev/stderr | grep '"is_initialized": true'; do
i=$((i + 1))
if [ "$i" -gt 20 ]; then
echo 'Failed to initialize TiKV cluster'
return 1
fi
sleep 5
done
echo 'Start TiKV success'
}

start_tidb() {
Expand All @@ -75,14 +77,15 @@ start_tidb() {
--log-file "$WORK_DIR/tidb.log" &

i=0
while ! curl "http://$TIDB_IP:10080/status"; do
while curl "http://$TIDB_IP:10080/status"; do
i=$((i + 1))
if [ "$i" -gt 50 ]; then
echo 'Failed to start TiDB'
return 1
fi
sleep 3
done
echo 'Start TiDB success'
}

start_pd
Expand Down
2 changes: 2 additions & 0 deletions dm/tests/many_tables/run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,9 @@ function run() {
pkill -hup tidb-server 2>/dev/null || true
wait_process_exit tidb-server
# now worker will process some binlog events, save table checkpoint and meet downstream error
echo "start incremental_data_2"
incremental_data_2
echo "finish incremental_data_2"
sleep 30

resume_num=$(grep 'unit process error' $WORK_DIR/worker1/log/dm-worker.log | wc -l)
Expand Down
1 change: 1 addition & 0 deletions dm/tests/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
requests
ipaddress
5 changes: 5 additions & 0 deletions dm/tests/tls/run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,13 @@ EOF
--log-file "$WORK_DIR/tidb.log" 2>&1 &

sleep 5

echo "show databases without TLS"
mysql -uroot -h127.0.0.1 -P4400 --default-character-set utf8 -E -e "SHOW DATABASES;"
echo "drop database tls with TLS"
# if execute failed, print tidb's log for debug
mysql -uroot -h127.0.0.1 -P4400 --default-character-set utf8 --ssl-ca $cur/conf/ca.pem --ssl-cert $cur/conf/dm.pem --ssl-key $cur/conf/dm.key -E -e "drop database if exists tls" || (cat $WORK_DIR/tidb.log && exit 1)
echo "drop database dm_meta with TLS"
mysql -uroot -h127.0.0.1 -P4400 --default-character-set utf8 --ssl-ca $cur/conf/ca.pem --ssl-cert $cur/conf/dm.pem --ssl-key $cur/conf/dm.key -E -e "drop database if exists dm_meta"
}

Expand Down
Loading