Simulado AWS Machine Learning Specialty MLS-C01

Kinesis Data Streams guarantees ordering only within a single shard, and the partition key is what determines which shard a record lands in. Using the driver's unique ID as the partition key ensures every ping from that driver maps to the same shard and is therefore consumed in arrival order, which is exactly what a correct rolling window requires. Using a random partition key (the current bug) scatters a driver's pings across shards, destroying per-driver order. Increasing shard count does not assign one shard per driver; the hash distribution still spreads keys, and a single random key would still scatter records, so more shards alone fixes nothing. Enhanced fan-out only gives each consumer a dedicated read throughput pipe; it does not reorder records and ordering is still per-shard. On-demand capacity mode changes how throughput scales, not how partition keys map to shards, so it does not route a producer's records to one shard or fix ordering.

A multi-model endpoint (MME) is built for exactly this case: many models that share a common framework live in S3, the endpoint dynamically loads a model into memory when first requested, caches frequently used ones, and routes each request to the right model by the TargetModel name — so you pay for one fleet instead of 2,000 endpoints, and rarely used tenant models impose almost no idle cost. Running 2,000 separate real-time endpoints is precisely the cost and operational explosion MME is designed to eliminate. A single shared production variant would serve one model to everyone, which violates the requirement that each tenant has its own model. Batch Transform is offline file scoring and cannot serve the per-request, by-tenant online predictions the platform needs, and scheduling 2,000 hourly jobs is operationally absurd for sparse, unpredictable traffic.

RecordIO-protobuf is the optimized binary format for SageMaker built-in algorithms like linear-learner and factorization-machines, and its protobuf tensor encoding supports a sparse representation that stores only non-zero index/value pairs — exactly what a 500-feature mostly-zero dataset needs to cut memory and speed up training. JSON Lines is verbose text, is not a high-throughput training format for these algorithms, and does nothing special for sparsity. Parquet is a fine analytics/storage format and some algorithms accept it, but linear-learner's efficient training path and explicit sparse-tensor support is RecordIO-protobuf, not Parquet RLE — the premise that linear-learner reads Parquet sparse natively is the misconception. A dense .npy file by definition stores every zero, so it neither exploits sparsity nor is the recommended SageMaker input format; memory-mapping does not skip zeros that are physically present in a dense array.