Concepts
Zero-copy I/O
Intuition
Imagine a waiter who has to carry a dish from the kitchen to the table: the slow way is to pass the dish across three intermediate counters before it reaches the customer, even though nobody needs to touch it along the way. The fast way is to hand the dish directly from the kitchen to the table. When a server sends a file from disk over the network, the traditional "slow" path copies the data several times between operating system memory and the application's own memory before it goes out over the network — copies that exist only because of how the system is organized, not because the application needs to do anything with those bytes along the way.
How it works
In the traditional path for sending a file over the network, data is copied from disk to a kernel buffer, then to an application buffer in user space, and then back to a kernel buffer associated with the network socket — several copies and context switches to move bytes that the application never needs to examine. Kafka avoids this path by using the sendfile system call, which transfers data directly from a file on disk to the network without passing through the application's memory space, eliminating the unnecessary intermediate copies. This choice is consistent with another Kafka design decision: instead of maintaining its own in-application message cache in memory, the broker relies on the operating system's page cache, since producers and consumers access the segment files mostly sequentially. This avoids duplicating the same data in two buffers (one in the application, one in the OS) and has the extra advantage that the cache stays "hot" even if the broker process restarts, because it lives in the OS and not in the process's memory. Together, these two decisions — sendfile for transfer and the page cache as the only caching layer — reduce the number of memory copies and context switches per message, which is exactly the kind of cost that dominates throughput when the data volume is on the order of hundreds of gigabytes per day.
In practice
That is why distributed log and high-throughput streaming systems like Kafka prefer aggressive simplicity on the hot data path (sequential append to disk, sendfile, OS page cache) over a sophisticated application cache: at scale, the cost of avoidable memory copies outweighs the benefit of any "smarter" caching logic done in the application. This is the same reasoning behind zero-copy optimizations in other systems that serve static files or streams at high volume, such as web servers and proxies.
Use it in a design
Practice the cases that rely on this concept in the System Design school.