Video platform (YouTube/Netflix)

01 · CONTEXT

Upload and watch: two buttons, two problems

The classic exercise asks for a video sharing platform like YouTube, and the same solution applies to Netflix or Hulu. The scope is deliberately narrow: upload a video and watch a video, on mobile apps, browsers, and smart TVs. The rest of the product is left out of the design.

The goals that guide the design are fast upload, smooth streaming, quality switching during playback, and low infrastructure cost, all with high availability and the ability to grow. File size, however, depends on the source: one fixes a ceiling of small and medium videos, the other requires supporting files of tens of GB, which changes how they are uploaded. As for scale, one source starts from millions of daily active users and the other from about one million videos uploaded and one hundred million watched per day.

The storage math shows the size of the problem: with five million daily active users, ten percent uploading a 300 MB video, that is 150 TB of new data per day. On the other side, serving those videos through the CDN is expensive, and that cost will weigh on the decisions at the end of the case.

02 · UPLOAD

Blob storage, CDN, and a queue in the middle

Building blob storage and a CDN from scratch is complex and expensive, so the design uses ready-made cloud services. The system has three pieces: the client, the CDN, which the video is served from, and the API servers, which everything other than the streaming itself passes through. Since the API servers are stateless, the layer scales horizontally.

1. The client asks the API servers for a presigned URL. The load balancer spreads the requests, and the API authorizes the exact destination where that file may be written.

2. With the URL, the client sends the file straight to the original storage, without passing through the application servers. In parallel, the API writes the metadata to the database and the cache.

3. The transcoding servers fetch the original and convert it into several formats and resolutions.

4. When done, the result goes to the transcoded storage and, at the same time, a completion event enters the completion queue.

5. The completion handler consumes the queue and updates the metadata database and cache, so that the API now knows the video is ready.

6. The transcoded files are distributed to the CDN, from where they will be streamed to viewers.

The upload splits into two parallel processes: the file goes to storage while the client sends the metadata. When transcoding finishes, a message queue carries the event to the handler, which updates the metadata database, kept partitioned and replicated for performance and availability.

The client does not send the video through the API: it receives a presigned URL and uploads straight to object storage. This keeps large files from crossing the application servers and, because the URL is issued by the API, only those who are authorized write to the right place.

The queue also decouples the stages. Without it, the encoding module would sit waiting for the output of the download module; with it, the module processes in parallel the events that are in the queue.

03 · TRANSCODING

From the raw file to the formats each screen understands

Transcoding is indispensable for four reasons: raw video takes up a lot of space, many devices and browsers only accept certain formats, people with broadband should get higher quality than people with low bandwidth, and the network changes during playback, especially on mobile, which requires switching quality midway.

A video format has two parts. The container bundles video, audio, and metadata, like a basket, and is usually recognized by the file extension. The codec compresses and decompresses to reduce size while preserving quality. Streaming protocols, in turn, differ in the encodings and players they support, so you need to choose one that fits your use case.

The work is expensive and slow, and each creator may request different steps, such as a watermark or a custom thumbnail. To provide flexibility and parallelism, the design adopts a directed acyclic graph model, the DAG, which describes which tasks exist and which depend on which. The most expensive part is converting the video chunks, which consumes CPU and must run on many machines at the same time, coordinated by an orchestrator.

1. The preprocessor takes the original video and splits it into short chunks that can be played independently, and also generates the DAG from the creator's configuration.

2. The chunks and metadata sit in temporary storage, so that a failure does not force starting over from scratch.

3. The DAG scheduler breaks the graph into stages and puts the tasks in the resource manager's queue.

4. The manager matches each task with the most suitable worker, and the task workers run the tasks in parallel.

5. If a worker goes down in the middle of a task, the task is repeated on another worker, reusing the data already stored.

6. Once the tasks are complete, the encoded files are written as the pipeline output.

04 · STREAMING

Segments, manifest, and adaptive bitrate

Watching is not downloading. Downloading copies the whole video to the device; streaming receives the video continuously, a little at a time, and starts playing right away. The difference matters: downloading a 10 GB file would take more than 13 minutes on a 100 Mbps connection, and a network drop midway would lose the progress.

In adaptive bitrate, a main manifest lists the available versions of the video and points to manifests for each version. The client fetches the video metadata, which points to the manifest, picks a format according to the network and the user's preferences, and downloads the first segment. If the network worsens or improves, it changes the format of the following segments.

1. The player asks the API for the video metadata, which responds with the manifest address, reading from the cache when possible.

2. The player downloads the manifest, which lists the versions of the video, and picks one according to the network and preferences.

3. It downloads the first segment, starts playing, and keeps loading the next ones in the background.

4. If the network worsens, the player starts requesting segments of a lighter version, to avoid interrupting playback.

5. When the CDN does not have the segment, usually from a less popular video, it comes from the high-capacity storage.

With manifests and segments on the CDN, the nearest edge server delivers the video and the streaming continues without touching the backend. Popular metadata sits in a cache, which spares the database.

05 · COST AND SECURITY

Where the money goes and how to protect the video

CDN cost is the main villain, and the way out comes from the distribution of accesses: a few videos are very popular and many have few viewers or none. From this observation come the levers for custo.

All these decisions depend on the access pattern, so the viewing history should be analyzed before optimizing.

For upload speed, the client splits the video into chunks, which allows resuming an upload that failed, and upload centers spread around the world, built with the CDN itself, bring the upload closer to the user. For security, presigned URLs restrict who can write. To protect the content, there are three paths: DRM systems, AES encryption with an authorization policy, and visual watermarking.

06 · FAILURES AND EXTENSIONS

When something breaks and what comes next

Error handling separates two types. For recoverable errors, such as a segment that fails to transcode, the general idea is to retry the operation a few times. For unrecoverable errors, such as a malformed format, the system stops the tasks for that video and returns an error code to the client. If a task worker goes down, the task goes to another, and if the metadata database primary goes down, a replica is promoted.

To grow, the API layer scales horizontally. The metadata can move to a partitionable database, and the task queue serves as a signal to increase the number of workers. There are also two topics for the end of the interview: live streaming, which requires lower latency and perhaps another protocol, and the removal of videos that violate copyright or are illegal.