Concepts
FaaS limits: stateless functions and data far from the code
Intuition
Imagine an office where each task is done by a freelancer who shows up, works for a short time, keeps nothing from one visit to the next, and has no phone. If a freelancer needs the papers from a distant archive, someone has to carry the papers to them every time; if two of them need to talk, they can only do so by leaving notes in a slow mailbox in the hallway. For independent, short tasks this is great, because you hire as many as you need and pay only for the work done. For tasks that depend on many papers and a lot of conversation, all the time goes into carrying paper and exchanging notes.
How it works
The paper analyzes Function-as-a-Service (FaaS), such as AWS Lambda, and points out four limitations. First, short lifetimes: a function lives at most 15 minutes, so it does not keep long-lived state, and the machine that runs it does not guarantee a cache across invocations. Second, I/O bottlenecks: a single function reached 538 Mbps of network bandwidth, but with 20 functions in parallel the per-function average dropped to 28.7 Mbps. Third, functions are not network-addressable, so communication between them goes through a slow storage service; moving 1 KB takes about 108 ms from Lambda to S3 and 11 ms to DynamoDB, versus 290 microseconds between EC2 instances with ZeroMQ. Fourth, there is no specialized hardware such as GPUs. The combined effect is the pattern the paper criticizes: data shipping, that is, the code is sent to the data and the data is brought to the function on every execution, instead of bringing the data closer to the code (code shipping). In the case studies, training a model took 465 minutes on functions versus about 22 minutes on an EC2 m4.large (US$ 0.29 versus US$ 0.04, about 21 times slower and 7.3 times more expensive). In prediction serving, per-request latency was 559 ms fetching the model on every call and 447 ms with the model embedded and SQS, versus 13 ms with EC2 and SQS and 2.8 ms with EC2 and ZeroMQ. The authors treat this as a consequence of the execution model, not a detail that fine-tuning would solve.
In practice
The paper lists three easy cases where FaaS works well: embarrassingly parallel workloads (tasks independent of each other), service orchestration (the function coordinates calls to other services), and function composition. It also acknowledges a large market of narrow applications, little more than business logic over a database, and cites Autodesk's user sign-up flow, which came to take about 10 minutes. When choosing serverless, the practical question is where the data is and how much communication between functions the workload requires: model training, low-latency serving, and distributed algorithms with frequent chatter tend to be better on machines that hold state and are directly addressable. As a direction, the authors propose fluid placement of code and data and long-running addressable agents.
Use it in a design
Practice the cases that rely on this concept in the System Design school.