This repo implements a new prototype of distributed task scheduler.
Go version no less than 1.18
Please run make tools_setup to build tools
Simply run make to compile.
protoc version no less than 3.8.0
Use make unit_test to run unit test and integrated test.
Run make dev before submitting pr.
Master is set to process the requests from outside and to schedule and dispatch tasks.
Executor is the worker process to run the tasks.
Master-Client is set to interact with master.
TODO
./bin/master --config=./sample/config/master.toml --master-addr 0.0.0.0:10240 --advertise-addr ${ip0}:10240
Replace ip0 with your advertising ip.
./bin/executor --config=./sample/config/executor.toml --join ${ip0}:10240 --worker-addr 0.0.0.0:10241 --advertise-addr ${ip1}:10241
Replace ip1 with your advertising executor ip.
Scaling out executor is quite simple. Right now we only support static config of masters, without scaling in and out dynamically.
In this case, we assume three node ips are ip0, ip1 and ip2. Replace them with real ip or hostname when operating.
./bin/master --name=ip0 --config=./sample/config/master.toml --master-addr 0.0.0.0:10240 --advertise-addr http://${ip0}:10240 --peer-urls 0.0.0.0:8291 --advertise-peer-urls http://${ip0}:8291 --initial-cluster ip0=http://${ip0}:8291,ip1=http://${ip1}:8291,ip2=http://${ip2}:8291
Note that --name can be any names, but has to be different from other two masters.
Deploying masters for ip1 and ip2 are similar.
./bin/executor --config=./sample/config/executor.toml --join ${ip0}:10240,${ip1}:10240,${ip2}:10240 --worker-addr 0.0.0.0:10241 --advertise-addr ${ip0}:10241