Papers
Serverless Computing: One Step Forward, Two Steps Back
Joseph M. Hellerstein, Jose Faleiro, Joseph E. Gonzalez, Johann Schleier-Smith, Vikram Sreekanti, Alexey Tumanov, Chenggang Wu · CIDR 2019 (Conference on Innovative Data Systems Research), 2019
The thesis: one step forward, two steps back
The paper analyzes the public Function-as-a-Service (FaaS) offerings, such as AWS Lambda, and argues that they get one thing right and two things wrong. The step forward is autoscaling with pay-per-use: the developer does not think about servers or capacity planning, and billing tracks actual usage. The two steps back, according to the authors, are that current FaaS is bad for data-intensive computing and bad for general-purpose distributed computing. The reason is that the model isolates code in short, stateless functions, and leaves data and communication to slow storage services. The authors do not say that serverless is useless. They acknowledge a large market of narrow applications, many of them little more than business logic over a database, and say that FaaS serves those cases well. The criticism is that the current model does not deliver the cloud's potential to those who want to program data systems and distributed algorithms. The stated goal is to change the question from 'will serverless win?' to 'how do we design infrastructure and programming models that unleash innovation in data-rich systems at cloud scale?'.
The four limitations of current FaaS
The paper identifies four limitations of FaaS offerings. The first is short lifetimes: a Lambda function lives at most 15 minutes, which prevents keeping long-lived state inside it. The second is I/O bottlenecks: measuring network bandwidth, a single function reached 538 Mbps, but with 20 functions in parallel the average per function dropped to 28.7 Mbps. The third is communication through slow storage: functions are not network-addressable, so two functions can only talk by passing data through an intermediary service. The fourth is the absence of specialized hardware, such as GPUs, which hurts model training. Together, these limitations lead to the pattern the paper criticizes: code is shipped to the data, instead of bringing the data closer to the code, and the machine running the function does not keep a cache between invocations with any guarantee. The authors treat this arrangement as the root cause of the poor performance they show in the following case studies, and not as an implementation detail that fine-tuning would fix.
Case studies: model training and prediction
To make the criticisms concrete, the authors compare FaaS with EC2 machines on machine learning tasks. In training a model, the function-based version took 465 minutes, versus about 22 minutes on an EC2 m4.large instance. In cost, it was $0.29 versus $0.04. That makes the version roughly 21 times slower and 7.3 times more expensive. In prediction serving, the latency per request was 559 ms when the model was fetched on every call, 447 ms with the model embedded in the function and SQS used for input, 13 ms with EC2 and SQS, and 2.8 ms with EC2 and ZeroMQ. In other words, even eliminating the model fetch, the function stayed far behind a machine that communicates directly. The point is not that the function's hardware is bad in itself, but that the execution model (short-lived, no reliable cache, communication via a service) is expensive for this kind of workload. The numbers illustrate the argument that data and communication, not computation, dominate the cost.
Communication, latency, and the cost of coordination
The paper measures latencies for moving 1 KB of data over different paths: about 303 ms to invoke a function, 108 ms from a Lambda function to S3, 11 ms from Lambda to DynamoDB, 106 ms from an EC2 instance to S3, 11 ms from EC2 to DynamoDB, and only 290 microseconds between EC2 instances using ZeroMQ. The difference of orders of magnitude shows the price of forcing communication to go through storage. The paper also discusses distributed coordination. Because functions are not directly addressable, a protocol such as leader election has to be done through storage: each round took 16.7 seconds, which amounts to 1.9% of the aggregate time, and for 1000 nodes the cost was at least $450 per hour. An example of messaging cost: sustaining 1 million messages per second on SQS would cost $1,584 per hour, versus $27.84 per hour for 290 EC2 instances (about 57 times less). These numbers support the thesis that FaaS, as offered today, is unsuitable for algorithms that depend on frequent communication.
What FaaS does well and the initial objections
The authors list three easy cases in which FaaS works well: embarrassingly parallel workloads (each task is independent of the others), service orchestration (the function coordinates calls to other services) and function composition. They cite as a success story a company, Autodesk, whose user onboarding flow came to take about 10 minutes. The paper also anticipates objections. One is that 'serverless' has a broader meaning than FaaS; the authors respond that delivering a specific autoscaling service does not solve the problem of enabling general-purpose programming in the cloud. Another is 'wait for the next networking announcement': they accept that networks will improve, but say that the memory hierarchy will continue to exist and that it will be necessary to be able to place code and data on the same side of a boundary, which FaaS does not offer. The third is 'economics makes serverless inevitable': they agree that there is a huge market of narrow applications, but fear that this will discourage new stateful services and reinforce the dominance of the providers' proprietary products.
Proposals: the way forward
In the last part, the paper proposes challenges for a truly programmable cloud. The first is fluid placement of code and data: the infrastructure should be able to physically bring code and data closer together for performance, but keep them logically separate to allow elasticity. Declarative, data-centric languages, such as SQL with UDFs, MapReduce, and TensorFlow, make this easier. The second is support for heterogeneous hardware, with the provider compiling high-level programs to the cheapest hardware that meets the user's goal. The third is long-running, addressable virtual agents, with a known identity and network performance comparable to the standard, but remappable across physical machines; actors, tuplespaces, pub/sub, and DHTs are examples of ideas along these lines. The fourth is disorderly programming: languages that encourage small units of data and computation, easy to move in time and space. The authors also suggest a common intermediate representation, service level objectives (SLOs) with an associated price, and attention to security. They conclude that the future of cloud programming is 'fluid', whether or not it is called serverless.
See the ideas at work
Practice the designs that use this paper in the System Design school.