Photo sharing (Instagram/Flickr)

01 · CONTEXT AND SCOPE

What a photo app needs to do

A photo-sharing app lets people post images and videos and see them back, fast, anywhere. In the Hello Interview version of the interview, the core requirements are creating posts with a photo, video, and caption and viewing a chronological feed of the people you follow. In an exercise in requirements gathering, what makes the case interesting is not the list of posts, but the weight of the files.

The same source asks for media to be displayed almost instantly and sets maximum sizes for photos and videos. With files of that size, the question becomes where the bytes travel and where they live.

02 · THE NAIVE PATH

Passing the file through the API

The first design that comes to mind has the API server receive the file and pass it on to storage. Hello Interview describes exactly this flow and points out that it works for small files and breaks when they grow. Storing the file in the database is even worse: databases are great with structured data and bad with large binary objects.

1. The app sends the whole file to the API server.

2. The API passes the same bytes on to object storage.

3. Only then does it write the post metadata to the database. Each large upload ties up API connections and bandwidth for the entire duration of the transfer.

The problem is not correctness, it is cost: the layer that should only decide and authorize becomes a transport channel, and scaling uploads means scaling API servers.

03 · DIRECT UPLOAD

Presigned URL: the API authorizes, storage receives

The large-blob pattern takes the API out of the path of the bytes: it generates a presigned URL and the client sends the file directly to object storage, and then downloads it through a CDN. On S3, this URL allows an upload without the sender having cloud credentials.

The URL is valid for the period set when it was generated, and works like a bearer token: whoever has it has the access. That is why the expiry should be short and the object key should be chosen by the server, never by the client.

1. The app asks the posts service for permission to upload a photo. The service decides the object key and returns the presigned URL.

2. In the same call, the service records the post in a pending state. Not a single byte of the photo has passed through here.

3. The app sends the file directly to object storage using the URL.

4. Storage notifies the queue that there is a new object, and the image worker consumes the notification.

5. The worker generates the size variants, writes them to storage, and marks the post as ready in the database.

04 · LARGE FILES AND PROCESSING

Parts, events, and durability

For large files, multipart upload splits the object into pieces sent independently. If one part fails, only that part is resent, without affecting the others. The S3 documentation recommends this mode for objects of 100 MB or more, which covers videos and is usually not needed for ordinary photos.

After the upload, processing does not need to block the user. Storage can publish an object-created event to a queue, a topic, or a function, and a worker generates the size variants. These events are delivered at least once, so the worker must tolerate the same notification twice. This is the classic role of message queues.

Object storage also solves durability. S3 standard storage is designed for 99.999999999% durability and stores objects redundantly in at least three availability zones. Losing a photo's file is damage the user will not forgive, and that is what justifies not keeping it on application server disks.

05 · DELIVERY

The CDN in front of storage

On reads, object storage becomes the origin, and a CDN sits in front. If the image is already at the nearest point of presence, it is delivered immediately; if not, the CDN fetches it from the origin, and the next people in the region already find the copy.

1. When opening a post, the app receives the metadata with the URL of the variant suited to the screen size.

2. The app requests the image from the CDN, not from storage. The CDN answers from its copy, if it has one.

3. If the CDN doesn't have the image, it fetches it from the origin and keeps the copy for the next requests.

Two precautions make the cache effective: give each variant its own object key and never overwrite it, so the CDN copy does not go stale, and serve the smallest variant that fits the screen, so you don't spend bandwidth on pixels nobody sees.