-
Notifications
You must be signed in to change notification settings - Fork 116
Building Your Scheduler
The core of Fenzo is the task scheduler. This page explains how to build a task scheduler that is appropriate to your needs.
Contents:
The TaskScheduler class defines an object that you can use to optimize the assignment of tasks to hosts within a Mesos framework. When you create a task scheduler object, you do so in a way that customizes it for your needs. This section will show you how to do this.
Fenzo uses the Builder Pattern to govern the creation of a task scheduler. If you have not encountered this pattern before, it may seem unusual. To create your TaskScheduler object, instead of calling that object’s constructor, you call a series of builder methods in this fashion:
myScheduler = new TaskScheduler.Builder()
.withSomeCustomization()
.withAnotherCustomization()
⋮
.andSoForth()
.build();
The TaskScheduler.Builder class has a set of methods that you can use to customize a generic task scheduler object. Each of these methods returns the same TaskScheduler.Builder object, but with your customizations. You can therefore chain these methods one after the other to make a series of customizations. When you finally call the build() method at the end of this chain, the Builder returns a TaskScheduler object, made to order as you designed it.
There are many options available to you when you customize your TaskScheduler during this build process:
| customization method | description |
|---|---|
.disableShortfallEvaluation() |
By default, Fenzo evaluates task assignment failures and assesses pending tasks to make an estimate of whether, and which, additional resources will be needed. This is useful for determining how to scale up resources. But this is an expensive calculation and you can disable it if you do not need it. See: Autoscaling. |
.withAutoScaleByAttributeName() |
Call this method to indicate which host attribute you want your task scheduler to use in order to distinguish which hosts are in which autoscaling groups. You must call this method before you call .withAutoScaleRule(). See: Autoscaling. |
.withAutoScaleDownBalancedByAttributeName() |
Call this method to tell the autoscaler to try to maintain a balance of host varieties when it scales down a cluster. Pass the method a host attribute, and the autoscaler will attempt to scale down in such a way as to maintain a similar number of hosts with each value for that attribute. See: Autoscaling. |
.withAutoScalerCallback() |
The callback you pass to this method receives an indication when an autoscale action is to be performed, telling it which autoscale rule prompted the action and whether the action is to scale up or scale down the autoscale group. The callback then initiates the appropriate scaling actions. See: Autoscaling |
.withAutoScalerMapHostnameAttributeName() |
In some circumstances (for instance with Amazon Web Services), the host name is not the correct identifier for the host in the context of an autoscaling action (for instance, in AWS, you need the EC2 instance identifier). If this is the case for your system, you need to implement a function that maps the host name to the identifier for the host in an autoscaling context so that Fenzo can perform autoscaling properly. You provide this function to the task manager by means of this builder method. See: Autoscaling. |
.withAutoScaleRule() |
Call this method to add an autoscaling rule that governs how your scheduler will autoscale hosts of a certain type. You must call withAutoScaleByAttributeName() before you call this method. See: Autoscaling. |
.withFitnessCalculator() |
Call this method to add a fitness calculator that your scheduler will use to compute the suitability of a particular host for a particular task. See: Fitness Calculators |
.withFitnessGoodEnoughFunction() |
Pass this method a function that takes a value between 0.0 (completely unfit) and 1.0 (perfectly fit) that describes the fitness of a particular host for a particular task, and decides, by returning a boolean value, whether that value is a “good enough” fit such that the task scheduler should go ahead and assign the task to the host. If you write this function to only return true for values at or near 1.0, the task scheduler will spend more time searching for a good fit; if you write the function to return true for lower values, the task scheduler will be able to find a host to assign the task to more quickly. (See [[Fitness Calculators: The Fitness “Good Enough” Function |
.withInitialResAllocs() |
Call this method to set the initial limitations on how many resources will be available to each task group. See: Resource Allocation Limits |
.withLeaseOfferExpirySecs() |
Call this method to set the expiration time for resource offers. Your task scheduler will reject any offers that remain unused if this expiration period from the time of the offer expires. This ensures your scheduler will not hoard unuseful offers. The default is 120 seconds. |
.withLeaseRejectAction() |
Required. Call this method to establish a method that your task scheduler will call to notify you that it has rejected a resource offer so that you can, in turn, tell Mesos that you are declining the offer. |
The sample Fenzo-aware Mesos framework implemented as the com.netflix.fenzo.samples.SampleFramework Java class gives an example of how to implement a TaskScheduler object.
scheduler = new TaskScheduler.Builder()
.withLeaseOfferExpirySecs(1000000000)
.withLeaseRejectAction(new Action1<VirtualMachineLease>() {
@Override
public void call(VirtualMachineLease lease) {
mesosSchedulerDriverRef.get().declineOffer(lease.getOffer().getId());
}
})
.build();
In addition to those customizations you can make to your task scheduler as you build it, you can also change its behavior after you have built it and while it is operating. These are the TaskScheduler methods you can use to modify the behavior of your previously-built TaskScheduler:
| method | description |
|---|---|
addOrReplaceAutoScaleRule() and removeAutoscaleRule()
|
Use these methods to change the autoscale rules you established when you built your task scheduler. See: Autoscaling |
setAutoscalerCallback() |
The callback you pass to this method receives an indication when an autoscale action happens, telling it which autoscale rule prompted the action and whether the action was to scale up or scale down the autoscale group. See: Autoscaling |
addOrReplaceResAllocs() and removeResAllocs()
|
Use these methods to change the resource allocation limits you established when you built your task scheduler. See: Resource Allocation Limits |
disableVM() and disableVMByVMId()
|
make a particular host ineligible to accept task assignments |
enableVM() |
make a previously-disabled host eligible to accept task assignments again |
setActiveVmGroupAttributeName() |
set the host attribute that Fenzo uses to determine to which group a host belongs |
setActiveVmGroups() |
establish a set of host groups that are active (hosts that belong to groups not in this list will be disabled, and Fenzo will not attempt to assign hosts to them) — if you do not call this method, or if you pass it a null List, Fenzo will consider all groups to be active |