Skip to content

A lightweight distributed task scheduling framework based on SpringBoot framework. It not only has high availability and scalability but also is easy to customize your business application by API provided.

License

Notifications You must be signed in to change notification settings

paganini2008/chaconne

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

87 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Chaconne Framework

a lightweight distributed task scheduling framework

Chaconne is a lightweight distributed task scheduling framework based on SpringBoot framework. It not only has high availability and scalability but also is easy to customize your business application by API provided.

Features

  • Perfectly supporting for SpringBoot framework (2.2.x or later)
  • Supporting master-slave cluster mode and load-balance cluster mode for scheduling tasks
  • Providing rich APIs for defining a task
  • Exposing plenty of external user interfaces to get information about cluster, task and task state
  • Supporting dynamically saving, pausing and deleting tasks
  • Supporting retry after task failure and failure transferring
  • Supporting task logs saving and tracking
  • Supporting task segmentation and recombination of execution results
  • Providing APIs to build DAG to describe dependency relationship between tasks
  • Supporting to customize task termination policy
  • Supporting to freeze and reset a task if has run a long time
  • Supporting email alarm if task running encounters exceptions

Modules

  • chaconne-spring-boot-starter The core class library of chaconne framework, providing implementations of most of core function and a series of external user API, including cluster management, task management and task runtime management.

  • chaconne-manager If being deployed with decentralized mode, chaconne framework provides an example application to act scheduler role to call task executors.

  • chaconne-console Chaconne framework provides a web application to display task info, running data and statistical data.

Deploy

  • Decentralized Deployment Mode No fixed and specific scheduler role, any application in the chaconne cluster can play the scheduler or executor role during the period of task scheduling.

  • Centralized Deployment Mode Differentiating the applications in the chaconne cluster as fixed scheduler and executor role during the period of task scheduling.

Install

<dependency>
    <artifactId>chaconne-spring-boot-starter</artifactId>
    <groupId>indi.atlantis.framework</groupId>
    <version>1.0-RC1</version>
</dependency>

Compatibility

  • Jdk1.8 (or later)
  • Spring Boot Framework 2.2.x (or later)
  • Redis 4.x (or later)
  • MySQL 5.x (or later)

Quick Start

How to define a task ?

Example 1

@ChacJob
@ChacTrigger(cron = "*/5 * * * * ?")
public class CronJob {

	@Run
	public Object execute(JobKey jobKey, Object attachment, Logger log) throws Exception {
		log.info("DemoCronJob is running at: {}", DateUtils.format(System.currentTimeMillis()));
		return RandomUtils.randomLong(1000000L, 1000000000L);
	}

	@OnSuccess
	public void onSuccess(JobKey jobKey, Object result, Logger log) {
		log.info("DemoCronJob's return value is: {}", result);
	}

	@OnFailure
	public void onFailure(JobKey jobKey, Throwable e, Logger log) {
		log.error("DemoCronJob is failed by cause: {}", e.getMessage(), e);
	}

}

Example 2

@Component
public class MemoryCheckJob extends ManagedJob {

	@Override
	public Object execute(JobKey jobKey, Object arg, Logger log) {
		log.info(info());
		return UUID.randomUUID().toString();
	}

	@Override
	public Trigger getTrigger() {
		return GenericTrigger.Builder.newTrigger("*/5 * * * * ?").setStartDate(DateUtils.addSeconds(new Date(), 30)).build();
	}

	private String info() {
		long totalMemory = Runtime.getRuntime().totalMemory();
		long usedMemory = totalMemory - Runtime.getRuntime().freeMemory();
		return FileUtils.formatSize(usedMemory) + "/" + FileUtils.formatSize(totalMemory);
	}

	@Override
	public long getTimeout() {
		return 60L * 1000;
	}

}

Example 3

public class CommonJob implements NotManagedJob {

	@Override
	public Object execute(JobKey jobKey, Object attachment, Logger log) {
		log.info("JobKey:{}, Parameter: {}", jobKey, attachment);
		return null;
	}

}

How to define DAG?

Example 1

flowchart TD;
     Task-A-->Task-B;
     Task-B-->Task-C;

Loading

TaskA.java

package com.test.chaconne;

@ChacJob(name = "taskA")
@ChacTrigger(cron = "0 */1 * * * ?")
public class TaskA {

	@Run
	public Object execute(JobKey jobKey, Object attachment, Logger log) throws Exception {
		return "TaskA Finished";
	}

}

TaskB.java

package com.test.chaconne;

@ChacJob(name = "taskB")
@ChacTrigger(triggerType = TriggerType.DEPENDENT)
@ChacDependency({ @ChacJobKey(className = "com.test.chaconne.TaskA", name = "taskA") })
public class TaskB {

	@Run
	public Object execute(JobKey jobKey, Object attachment, Logger log) throws Exception {
		System.out.println("TaskA's state: " + attachment); // Print 'TaskA Finished'
		return "TaskB Finished";
	}

}

TaskC.java

package com.test.chaconne;

@ChacJob(name = "taskC")
@ChacTrigger(triggerType = TriggerType.DEPENDENT)
@ChacDependency({ @ChacJobKey(className = "com.test.chaconne.TaskB", name = "taskB") })
public class TaskC {

	@Run
	public Object execute(JobKey jobKey, Object attachment, Logger log) throws Exception {
        System.out.println("TaskB's state: " + attachment); // Print 'TaskB Finished'
		return "TaskC Finished";
	}

	@OnSuccess
	public void onSuccess(JobKey jobKey, Object result, Logger log) {
        System.out.println("TaskC's state: " + attachment); // Print 'TaskC Finished'
	}

	@OnFailure
	public void onFailure(JobKey jobKey, Throwable e, Logger log) {
        // Do something
	}

}

Run result:

TaskA's state: TaskA Finished
TaskB's state: TaskB Finished
TaskC's state: TaskC Finished

Example 2

flowchart TD;
     Task-A-->Task-C;
     Task-B-->Task-C;

Loading

TaskA.java

package com.test.chaconne;

@ChacJob(name = "taskA")
@ChacTrigger(triggerType = TriggerType.SIMPLE)
public class TaskA {

	@Run
	public Object execute(JobKey jobKey, Object attachment, Logger log) throws Exception {
		return "TaskA Finished";
	}

}

TaskB.java

package com.test.chaconne;

@ChacJob(name = "taskB")
@ChacTrigger(triggerType = TriggerType.SIMPLE)
public class TaskB {

	@Run
	public Object execute(JobKey jobKey, Object attachment, Logger log) throws Exception {
		return "TaskB Finished";
	}
	
}

TaskC.java

@ChacJob
@ChacTrigger(cron = "0 0/1 * * * ?", triggerType = TriggerType.CRON)
@ChacFork({
            @ChacJobKey(className = "com.test.chaconne.TaskA", name = "taskA"),
		    @ChacJobKey(className = "com.test.chaconne.TaskB", name = "taskB") })
public class TaskC {

	@Run
	public Object execute(JobKey jobKey, Object attachment, Logger log) throws Exception {
		JobResult result = (JobResult) attachment;
		JobResult[] forkResults = result.getForkJobResults();
		for (JobResult forkResult : forkResults) {
            // Print TaskA and TaskB's state
			System.out.println(forkResult.getJobKey().getName() + "'s state: " + forkResult.getResult());
		}
		return "TaskC Finished";
	}

	@OnSuccess
	public void onSuccess(JobKey jobKey, Object result, Logger log) {
        System.out.println("TaskC's state: " + attachment); // Print 'TaskC Finished'
	}

	@OnFailure
	public void onFailure(JobKey jobKey, Throwable e, Logger log) {
        // Do something
	}

}

Run result

TaskA's state: TaskA Finished
TaskB's state: TaskB Finished
TaskC's state: TaskC Finished

Example 3

flowchart TD;
     Task-A-->Task-B;
     Task-A-->Task-C;
     Task-B-->Task-D;
     Task-C-->Task-D;
Loading

Using API create Dag Task

@RequestMapping("/dag")
@RestController
public class DagTaskController {

	@Value("${spring.application.cluster.name}")
	private String clusterName;

	@Value("${spring.application.name}")
	private String applicationName;

	@Autowired
	private JobManager jobManager;

	@GetMapping("/create")
	public Map<String, Object> createDagTask() throws Exception {
		Dag dag = new Dag(clusterName, applicationName, "testDag");
		dag.setTrigger(new CronTrigger("0 0/1 * * * ?"));
		dag.setDescription("This is only a demo of dag job");
		
		DagFlow taskA = dag.startWith(clusterName, applicationName, "taskA", TaskA.class.getName());
		DagFlow taskD = taskA.flow(clusterName, applicationName, "taskD", TaskD.class.getName());
		taskD.fork(clusterName, applicationName, "taskB", TaskB.class.getName());
		taskD.fork(clusterName, applicationName, "taskC", TaskC.class.getName());
		jobManager.persistJob(dag, "123");
		return Collections.singletonMap("ok", 1);
	}

}

How to deploy chaconne cluster with decentralized mode?

Example 1

@EnableChaconneEmbeddedMode
@SpringBootApplication
@ComponentScan
public class YourApplication {

	public static void main(String[] args) {
		final int port = 8088;
		System.setProperty("server.port", String.valueOf(port));
		SpringApplication.run(YourApplication.class, args);
	}

}

How to deploy chaconne cluster with centralized mode?

Example 1

@EnableChaconneDetachedMode(DetachedMode.PRODUCER)
@SpringBootApplication
public class ChaconneManagementApplication {

	public static void main(String[] args) {
		SpringApplication.run(ChaconneManagementApplication.class, args);
	}
}

Example 2

@EnableChaconneDetachedMode(DetachedMode.CONSUMER)
@SpringBootApplication
@ComponentScan
public class YourApplication {

	public static void main(String[] args) {
		SpringApplication.run(YourApplication.class, args);
	}

}

About

A lightweight distributed task scheduling framework based on SpringBoot framework. It not only has high availability and scalability but also is easy to customize your business application by API provided.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published