Autoscaling

Autoscaling, also spelled auto scaling or auto-scaling, and sometimes also called automatic scaling, is a method used in cloud computing that dynamically adjusts the amount of computational resources in a server farm - typically measured by the number of active servers - automatically based on the load on the farm. For example, the number of servers running behind a web application may be increased or decreased automatically based on the number of active users on the site. Since such metrics may change dramatically throughout the course of the day, and servers are a limited resource that cost money to run even while idle, there is often an incentive to run "just enough" servers to support the current load while still being able to support sudden and large spikes in activity. Autoscaling is helpful for such needs, as it can reduce the number of active servers when activity is low, and launch new servers when activity is high. Autoscaling is closely related to, and builds upon, the idea of load balancing.{{cite web|url = http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.pdf|title = Above the Clouds: A Berkeley View of Cloud Computing|date = February 10, 2009|publisher = Berkeley EECS|accessdate = March 21, 2015}}{{cite web|url=http://aws.amazon.com/autoscaling/|title = Auto Scaling|publisher = Amazon Web Services|accessdate = March 21, 2015}}

Advantages

Autoscaling offers the following advantages:

For companies running their own web server infrastructure, autoscaling typically means allowing some servers to go to sleep during times of low load, saving on electricity costs (as well as water costs if water is being used to cool the machines).
For companies using infrastructure hosted in the cloud, autoscaling can mean lower bills, because most cloud providers charge based on total usage rather than maximum capacity.
Even for companies that cannot reduce the total compute capacity they run or pay for at any given time, autoscaling can help by allowing the company to run less time-sensitive workloads on machines that get freed up by autoscaling during times of low traffic.{{cite web|url = http://techblog.netflix.com/2015/09/creating-your-own-ec2-spot-market.html|title = Creating Your Own EC2 Spot Market|last1 = Park|first1 = Andrew|last2 = Denlinger|first2 = Darrell|last3 = Watson|first3 = Coburn|date = September 18, 2015|accessdate = December 16, 2016|publisher = Netflix}}
Autoscaling solutions, such as the one offered by Amazon Web Services, can also take care of replacing unhealthy instances and therefore protecting somewhat against hardware, network, and application failures.
Autoscaling can offer greater uptime and more availability in cases where production workloads are variable and unpredictable.

Autoscaling differs from having a fixed daily, weekly, or yearly cycle of server use in that it is responsive to actual usage patterns, and thus reduces the potential downside of having too few or too many servers for the traffic load. For instance, if traffic is usually lower at midnight, then a static scaling solution might schedule some servers to sleep at night, but this might result in downtime on a night where people happen to use the Internet more (for instance, due to a viral news event). Autoscaling, on the other hand, can handle unexpected traffic spikes better.

Terminology

In the list below, we use the terminology used by Amazon Web Services (AWS).{{cite web|url = http://docs.aws.amazon.com/autoscaling/latest/userguide/WhatIsAutoScaling.html|title = What Is Auto Scaling?|publisher = Amazon Web Services|accessdate = December 16, 2016}} However, alternative names are noted and terminology that is specific to the names of Amazon services is not used for the names.

class="wikitable" border="1" ! Name (used in AWS, unless otherwise noted) !! Meaning !! Alternative names (used in Google Cloud Platform, Microsoft Azure, or other platforms)
Instance	A single server or machine that is part of the group of machines subject to autoscaling
Autoscaling group	The collection of instances subject to autoscaling, along with all the associated policies and state information	Managed instance group (Google Cloud Platform)
Size	The number of instances currently part of the autoscaling group
Desired capacity (or desired size)	The number of instances that the autoscaling group should have at any given point in time. If the size is less than the desired size, the autoscaling group will try to launch (provision and attach) new instances. If the size is more than the desired size, the autoscaling group will try to remove (detach and terminate) instances
Minimum size	A number of instances below which the desired capacity is not allowed to fall
Maximum size	A number of instances above which the desired capacity is not allowed to rise
Metric	A measurement (such as CPU utilization, memory usage, network usage) associated with the autoscaling group, for which a time series of data points is generated regularly. Thresholds for metrics can be used to set autoscaling policies. Metrics can be based on aggregates of metrics for instances of the autoscaling group, or based on load balancers associated with the autoscaling group
Scaling policy (or autoscaling policy)	A policy that specifies a change to the autoscaling group's desired capacity (or sometimes, its minimum and maximum size) in response to metrics crossing specific thresholds. Scaling policies can have associated cooldown periods, which prevent additional scaling actions from occurring immediately after a specific scaling action. Changes to desired capacity could be incremental (increase or decrease by a specific number) or could specify a new value of the desired capacity. Policies that increase the desired capacity are called "scaling out" or "scaling up" policies, and policies that decrease the desired capacity are called "scaling in" or "scaling down" policies
Health check	A way for the autoscaling group to determine if the instances attached to it are functioning properly. A health check may be based on whether the instance still exists and is reachable, or it could be based on whether the instance is still registered and in service with an associated load balancer
Launch configuration	A description of the parameters and scripts used when launching a new instance. This includes the instance type, purchase options (such as spot versus on-demand in the case of AWS), possible availability zones for launch, machine image, and scripts to run on launch	Instance template (Google Cloud Platform)
Manual scaling	A scaling action executed manually
Scheduled scaling	A scaling policy that is executed at a specific time, for instance, time of day or week or month or year. See #Scheduled scaling for more

Practice

=Amazon Web Services (AWS)=

File:AWS Simple Icons Compute Amazon Elastic MapReduce Auto Scaling.svg

Amazon Web Services launched the Amazon Elastic Compute Cloud (EC2) service in August 2006, that allowed developers to programmatically create and terminate instances (machines).{{cite news|url = https://techcrunch.com/2006/08/24/exclusive-amazon-readies-utility-computing-service/|title = Almost Exclusive: Amazon Readies Utility Computing Service|last = Cubrilovic|first = Nik|date = August 24, 2006|accessdate = December 4, 2016|work = TechCrunch}}{{cite web

|url=http://aws.typepad.com/aws/2006/08/amazon_ec2_beta.html

|title=Amazon EC2 Beta

|first=Jeff

|last=Barr

|date=August 25, 2006

|work=Amazon Web Services Blog

|accessdate= May 31, 2013

}} At the time of initial launch, AWS did not offer autoscaling, but the ability to programmatically create and terminate instances gave developers the flexibility to write their own code for autoscaling.

Third-party autoscaling software for AWS began appearing around April 2008. These included tools by Scalr{{cite news|url=https://techcrunch.com/2008/04/03/scalr-the-auto-scaling-open-source-amazon-ec2-effort/|title = Scalr: The Auto-Scaling Open-Source Amazon EC2 Effort|date = April 3, 2008|accessdate = March 21, 2015|work = TechCrunch|last = Work|first = Henry}} and RightScale. RightScale was used by Animoto, which was able to handle Facebook traffic by adopting autoscaling.{{cite news|url = https://www.zdnet.com/article/rightscale-cloud-management-extends-to-mysql/#!|title = RightScale cloud management extends to MySQL. RightScale, which specializes in cloud computing management for the Amazon Web Services platform today announced support for MySQL Enterprise. The service, which goes live July 1, provides automated deployment, management and scaling, coupled with MySQL Enterprise premium-level support for large database applications.|last = Howlett|first = Dennis|date = June 25, 2008|access-date = December 16, 2016|work = ZDNet}}{{cite web|url = http://www.rightscale.com/blog/enterprise-cloud-strategies/animotos-facebook-scale|title = Animoto's Facebook Scale-Up|last = von Eicken|first = Thorsten|date = April 23, 2008|accessdate = December 16, 2016|archive-url = https://web.archive.org/web/20161220212607/http://www.rightscale.com/blog/enterprise-cloud-strategies/animotos-facebook-scale|archive-date = December 20, 2016|url-status = dead}}

On May 18, 2009, Amazon launched its own autoscaling feature along with Elastic Load Balancing, as part of Amazon Elastic Compute Cloud.{{cite web |url=https://aws.amazon.com/blogs/aws/new-aws-load-balancing-automatic-scaling-and-cloud-monitoring-services/ |title=New Features for Amazon EC2: Elastic Load Balancing, Auto Scaling, and Amazon CloudWatch |publisher = Amazon Web Services |date=May 18, 2009|last = Barr|first = Jeff|accessdate= June 15, 2016}} Autoscaling is now an integral component of Amazon's EC2 offering.{{cite web|url = http://searchcloudapplications.techtarget.com/definition/autoscaling|title = What is autoscaling?|publisher = TechTarget|accessdate = March 21, 2015|archive-date = April 29, 2019|archive-url = https://web.archive.org/web/20190429044346/https://searchcloudapplications.techtarget.com/definition/autoscaling|url-status = dead}}{{cite web|url=https://aws.amazon.com/blogs/aws/auto-scaling-update-lifecycle-standby-detach/|title = Auto Scaling Update – Lifecycle Management, Standby State, and DetachInstances|last = Barr|first = Jeff|date = July 30, 2014|accessdate = March 21, 2015|publisher = Amazon Web Services (official blog)}} Autoscaling on Amazon Web Services is done through a web browser or the command line tool.{{cite web|url=https://aws.amazon.com/developertools/2535|title = Auto Scaling Command Line Tool|accessdate = March 21, 2015|publisher = Amazon Web Services (community-edited page)}} In May 2016 Autoscaling was also offered in AWS ECS Service.{{Cite web|url=https://aws.amazon.com/blogs/compute/automatic-scaling-with-amazon-ecs/|title=Automatic Scaling with Amazon ECS|date=18 May 2016|access-date=12 February 2019|archive-date=25 September 2019|archive-url=https://web.archive.org/web/20190925091242/https://aws.amazon.com/blogs/compute/automatic-scaling-with-amazon-ecs/|url-status=dead}}

On-demand video provider Netflix documented their use of autoscaling with Amazon Web Services to meet their highly variable consumer needs. They found that aggressive scaling up and delayed and cautious scaling down served their goals of uptime and responsiveness best.{{cite web|url=http://techblog.netflix.com/2012/01/auto-scaling-in-amazon-cloud.html|title = Auto Scaling in the Amazon Cloud|date = January 18, 2012|accessdate = March 21, 2012|publisher = Netflix Tech Blog|last1 = Orzell|first1 = Greg|last2 = Becker|first2 = Justin}}

In an article for TechCrunch, Zev Laderman, the co-founder and CEO of Newvem, a service that helps optimize AWS cloud infrastructure, recommended that startups use autoscaling in order to keep their Amazon Web Services costs low.{{cite news|url=https://techcrunch.com/2012/04/22/amazon-web-services-mistakes/|title = The 10 Biggest Mistakes Made With Amazon Web Services|last = Laderman|first = Zev|date = April 22, 2012|accessdate = March 21, 2015|work = TechCrunch}}

Various best practice guides for AWS use suggest using its autoscaling feature even in cases where the load is not variable. That is because autoscaling offers two other advantages: automatic replacement of any instances that become unhealthy for any reason (such as hardware failure, network failure, or application error), and automatic replacement of spot instances that get interrupted for price or capacity reasons, making it more feasible to use spot instances for production purposes.{{cite web|url=https://cloudonaut.io/5-aws-mistakes-you-should-avoid/|title = 5 AWS mistakes you should avoid|last = Wittig|first = Michael|publisher = cloudonaut|date = December 26, 2015|accessdate = December 16, 2016}}{{cite web|url = https://wblinks.com/notes/aws-tips-i-wish-id-known-before-i-started/|title = AWS Tips I Wish I'd Known Before I Started. A collection of random tips for Amazon Web Services (AWS) that I wish I'd been told a few years ago, based on what I've learned by building and deploying various applications on AWS.|date = February 3, 2014|accessdate = December 16, 2016|last = Adams|first = Rich}}{{cite web|url = https://www.wikihow.com/Use-Amazon-EC2-Spot-Instances|title = How to Use Amazon EC2 Spot Instances|publisher = wikiHow|accessdate = December 16, 2016}} Netflix's internal best practices require every instance to be in an autoscaling group, and its conformity monkey terminates any instance not in an autoscaling group in order to enforce this best practice.{{cite web|url = http://techblog.netflix.com/2011/07/netflix-simian-army.html|title = The Netflix Simian Army|date = July 19, 2011|accessdate = December 5, 2016|publisher = Netflix}}

=Microsoft's Windows Azure=

On June 27, 2013, Microsoft announced that it was adding autoscaling support to its Windows Azure cloud computing platform.{{cite news|url=https://techcrunch.com/2013/06/27/microsoft-adds-auto-scaling-to-windows-azure/|title = Microsoft Adds Auto Scaling To Windows Azure|last = Lardinois|first = Frederic|date = June 27, 2013|accessdate = March 21, 2015|work = TechCrunch}}{{cite news|url=https://www.zdnet.com/article/microsoft-to-add-autoscaling-alerts-to-windows-azure/|title = Microsoft to add autoscaling, alerts to Windows Azure|date = June 27, 2013|access-date = March 21, 2015|work = ZDNet}}{{cite web|url = http://www.networkworld.com/article/2168844/cloud-computing/google--microsoft-play-catch-up-to-amazon--add-load-balancing--auto-scaling-to-their.html|title = Google, Microsoft play catch up to Amazon, add load balancing, auto-scaling to their clouds|last = Butler|first = Brandon|date = August 7, 2013|access-date = March 21, 2015|publisher = Network World|archive-date = May 18, 2018|archive-url = https://web.archive.org/web/20180518193924/https://www.networkworld.com/article/2168844/cloud-computing/google--microsoft-play-catch-up-to-amazon--add-load-balancing--auto-scaling-to-their.html|url-status = dead}} Documentation for the feature is available on the Microsoft Developer Network.{{cite web|url=https://msdn.microsoft.com/en-us/library/dn589774.aspx|title = Autoscaling Guidance| date=26 August 2015 |publisher = Microsoft Developer Network}}{{cite web|url=https://msdn.microsoft.com/en-us/library/hh680892%28v=pandp.50%29.aspx|title = The Autoscaling Application Block|accessdate = March 21, 2015|publisher = Microsoft Developer Network}}

= Oracle Cloud =

Oracle Cloud Platform allows server instances to automatically scale a cluster in or out by defining an auto-scaling rule.{{Cite web|url=https://docs.oracle.com/en/cloud/paas/psmon/automatic-scaling.html|title=Administering PaaS Services|website=Oracle Help Center|language=en-us|access-date=2018-05-16}} These rules are based on CPU and/or memory utilization and determine when to add or remove nodes.

=Google Cloud Platform=

On November 17, 2014, the Google Compute Engine announced a public beta of its autoscaling feature for use in Google Cloud Platform applications.{{cite web|url=http://googlecloudplatform.blogspot.com/2014/11/autoscaling-welcome-to-google-compute.html|title = Autoscaling, welcome to Google Compute Engine|date = November 17, 2014|accessdate = March 21, 2015|publisher = Google Cloud Platform blog|last = Balejko|first = Filip}}{{cite news|url=https://venturebeat.com/2014/11/17/google-compute-engine-gets-autoscaler-to-adjust-app-resources-based-on-varying-traffic-and-workloads/ | title = Google Compute Engine gets Autoscaler to adjust app resources based on varying traffic and workloads|last = Protalinski|first = Emil|date = November 17, 2014|accessdate = March 21, 2015|work = VentureBeat}}{{cite news|url=https://techcrunch.com/2014/11/17/google-brings-autoscaling-to-compute-engine/|title = Google Brings Autoscaling To Compute Engine|date = November 17, 2014|accessdate = March 21, 2015|work = TechCrunch|last = Lardinois|first = Frederic}}{{cite web|url=http://www.datacenterknowledge.com/archives/2014/11/17/google-launches-autoscaling-beta-on-compute-engine/|title = Google Launches Autoscaling Beta on Compute Engine|last = Verge|first = Jason|date = November 17, 2014|accessdate = March 21, 2015|publisher = Data Center Knowledge}} As of March 2015, the autoscaling tool is still in Beta.{{cite web|url=https://cloud.google.com/compute/docs/autoscaler/|title = Autoscaler|accessdate = March 21, 2015|publisher = Google Cloud Platform}}

=Facebook=

In a blog post in August 2014, a Facebook engineer disclosed that the company had started using autoscaling to bring down its energy costs. The blog post reported a 27% decline in energy use for low traffic hours (around midnight) and a 10-15% decline in energy use over the typical 24-hour cycle.{{cite web|url=https://code.facebook.com/posts/816473015039157/making-facebook-s-software-infrastructure-more-energy-efficient-with-autoscale/|title = Making Facebook's software infrastructure more energy efficient with Autoscale|last = Wu|first = Qiang|publisher = Facebook Code Blog|date = August 8, 2014|accessdate = March 21, 2015}}

=Kubernetes Horizontal Pod Autoscaler=

Kubernetes Horizontal Pod Autoscaler automatically scales the number of pods in a [https://kubernetes.io/docs/concepts/workloads/controllers/replicationcontroller/ replication controller], [https://kubernetes.io/docs/concepts/workloads/controllers/deployment/ deployment] or [https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/ replicaset] based on observed CPU utilization (or, with beta support, on some other, [https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#scaling-on-custom-metrics application-provided metrics]){{cite web|url=https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/|title = Horizontal Pod Autoscaler Walkthrough|language=en-us|accessdate = June 21, 2018}}

Alternative autoscaling decision approaches

Autoscaling by default uses reactive decision approach for dealing with traffic scaling: scaling only happens in response to real-time changes in metrics. In some cases, particularly when the changes occur very quickly, this reactive approach to scaling is insufficient. Two other kinds of autoscaling decision approaches are described below.

= Scheduled autoscaling approach =

This is an approach to autoscaling where changes are made to the minimum size, maximum size, or desired capacity of the autoscaling group at specific times of day. Scheduled scaling is useful, for instance, if there is a known traffic load increase or decrease at specific times of the day, but the change is too sudden for reactive approach based autoscaling to respond fast enough. AWS autoscaling groups support scheduled scaling.{{cite web|url = http://docs.aws.amazon.com/autoscaling/latest/userguide/schedule_time.html|title = Scheduled Scaling|publisher = Amazon Web Services|accessdate = December 16, 2016}}

= Predictive autoscaling =

This approach to autoscaling uses predictive analytics. The idea is to combine recent usage trends with historical usage data as well as other kinds of data to predict usage in the future, and autoscale based on these predictions.

For parts of their infrastructure and specific workloads, Netflix found that Scryer, their predictive analytics engine, gave better results than Amazon's reactive autoscaling approach. In particular, it was better for:{{cite web|last1=Jacobson|first1=Daniel|last2=Yuan|first2=Danny|last3=Joshi|first3=Neeraj|title=Scryer: Netflix's Predictive Auto Scaling Engine|url=http://techblog.netflix.com/2013/11/scryer-netflixs-predictive-auto-scaling.html|website=The Netflix Tech Blog|publisher=Netflix|accessdate=28 May 2015}}{{cite web|url = https://www.morpheusdata.com/blog/2016-11-02-autoscaling-how-the-cloud-provides-a-tremendous-boost/|title = Autoscaling: How the Cloud Provides a Tremendous Boost|date = November 2, 2016|accessdate = December 16, 2016|publisher = Morpheus}}

Identifying huge spikes in demand in the near future and getting capacity ready a little in advance
Dealing with large-scale outages, such as failure of entire availability zones and regions
Dealing with variable traffic patterns, providing more flexibility on the rate of scaling out or in based on the typical level and rate of change in demand at various times of day

On November 20, 2018, AWS announced that predictive scaling would be available as part of its autoscaling offering.{{cite web|url = https://aws.amazon.com/blogs/aws/new-predictive-scaling-for-ec2-powered-by-machine-learning/|title = New – Predictive Scaling for EC2, Powered by Machine Learning|last = Barr|first = Jeff|date = November 20, 2018|accessdate = November 23, 2018|publisher = Amazon Web Services}}

References

Category:Network management

Category:Servers (computing)

Category:Cloud computing