Hi Aastikta, I had a question, what would be good use cases for the types of autoscaling mentioned above? I believe the answer lies in the difference between applications where utilization matters more than throughput or vice versa?
Hi Saloni, you are right that usage will depend highly on the application and it's use case. For example, take a machine learning service, let's say it helps in ranking some elements based on certain attributes. Here, the performance of this ML service is measured by the actual work done to correctly rank the new elements per second. Since the useful work done by this service is measured by requests it receives per second, throughput autoscaling can be a good fit for it. As the number of requests start increasing, the work load on the ML service will increase because now it will have to rank more elements and also, train itself in the process. If we use throughput autoscaling when traffic surges, it can help us autoscale based on the demand (number of requests) and how ML service is handling requests if one of the data centers where this service is deployed, goes down (disaster).
Regarding utilization autoscaling, take for example, a chat application that sends message on main thread and spins up a background thread for operations such as file upload. Is this application starts spinning up a lot of background threads due to traffic spike, then the cpu and/or memory utilization goes up. In such scenarios, utilization metrics will be helpful to autoscale the service by say, spinning up new server instances.
Great content. It would be great if you keep posting these in the medium platform too. there would be a bigger reach and many will be benefited with these stuff.
Hi Aastikta, I had a question, what would be good use cases for the types of autoscaling mentioned above? I believe the answer lies in the difference between applications where utilization matters more than throughput or vice versa?
Hi Saloni, you are right that usage will depend highly on the application and it's use case. For example, take a machine learning service, let's say it helps in ranking some elements based on certain attributes. Here, the performance of this ML service is measured by the actual work done to correctly rank the new elements per second. Since the useful work done by this service is measured by requests it receives per second, throughput autoscaling can be a good fit for it. As the number of requests start increasing, the work load on the ML service will increase because now it will have to rank more elements and also, train itself in the process. If we use throughput autoscaling when traffic surges, it can help us autoscale based on the demand (number of requests) and how ML service is handling requests if one of the data centers where this service is deployed, goes down (disaster).
Regarding utilization autoscaling, take for example, a chat application that sends message on main thread and spins up a background thread for operations such as file upload. Is this application starts spinning up a lot of background threads due to traffic spike, then the cpu and/or memory utilization goes up. In such scenarios, utilization metrics will be helpful to autoscale the service by say, spinning up new server instances.
Hi Aastikta,
Great content. It would be great if you keep posting these in the medium platform too. there would be a bigger reach and many will be benefited with these stuff.
Keep Going :)
Hello Jai,
Great suggestion, tried with one of my first articles but will keep it in continuation for remaining too. Thanks for reading :)