-
Notifications
You must be signed in to change notification settings - Fork 9
Scalability
By using tasks in this library you can more easily distribute your workloads across servers. Typical workloads that I prefer to put on background processes includes image manipulation, data processing, dequeuing, data analysis, message sending (i.e. email, device push notifications).
It is important to know the nature of the data that you are handling on your web front ends; but where possible I like to just queue data from my web servers, and have a server that is handling the actual work dequeuing and storing the data. If you are able to implement this, then you will be able to have a more fault tolerant and scalable system.
It is important to note the different types of scale typically used when looking at the workloads that you have:
Horizontal scale refers to adding more servers to handle a given workload; if you are dequeuing data and storing it, you may find that dequeuing at a rate of 1 message per second is not keeping up with the number of items in the queue. One way to solve this would be to add more servers to give you higher throughput.
Vertical scale is a term used to describe adding bigger servers to get the work done; take for example image processing where you need a lot of CPU power to crunch through the images. If you find that your servers aren't able to process enough images (throughput) fast enough, then you may need to add a higher frequency CPU (Ghz); or a computer with more CPU cores.
If you can solve your data needs so that you are able to achieve linear scale rates then you are on the right track. That is to say that by scaling Vertically or Horizontally you are able to achieve increased throughput by a significant factor, without introducing bottlenecks (such as data storage). The only other piece to this is to determine what the maximum throughput your system can handle, or when would you start seeing bottlenecks? If you can determine that then you will know which direction to head in.
I typically will start any project with an Extra Small VM on Azure. And start determining the CPU/Memory/Network bottlenecks. The Extra Small's don't usually take me too far primarily due to memory, and network usage. Most of the King.Service projects I have seen end up using Small VM's; as they are handling primarily dequeue tasks and data analysis tasks less frequently.
For the most part there is a lot of headroom left in the Small VM's when running between 2-8 in a production environment. However, if you are planning on running a significant number of images through Small might not be the place to start...
I am happy when I know the throughput of my servers, and I am able to adjust the throughput by scaling the number of servers. On my current deployment I have been scaling the number of servers based on queue length, which has worked well.
Most recently, I have had an audit log which has been filling a queue up and I have moved from dequeuing at a rate of one message every 45 seconds to a rate of 5 seconds. I thought that this throughput would be enough; as it turns out, it isn't. So instead of running on dequeue service on a server we are going to run several, which we can govern by configuration. That gives us the ability to say in development environments we want 1 audit log dequeuer, while in production we can have 5 audit log dequeuers; and by adding more servers it starts giving us substantial throughput. As it is an audit log we don't want to move to event based queues, as we want to control the speed of the data flow as it is low priority data.
Check out the Auto Scale Feature!