Nubank: real-time ML alerts and automatic retraining

01 · CONTEXT

Models that fail silently

Nubank's ML team starts from an observation: a model served on demand is software, but with an important difference, because it can fail silently, producing wrong predictions without any exception or error message. That is the starting point for talking about monitoring and observability in an ML system.

This case study brings together complementary Nubank topics: how to quickly warn when something is wrong with the served model, and how to replace it carefully with a retrained version.

Beyond the usual failures of any software, the source lists typical ML failure modes: missing or broken features and sudden shifts in the population distribution, both leading to wrong predictions.

02 · ALERTS

From the served model to the on-call

1. The diagram shows business requests reaching the model, which reads its features from its own source.

2. Even without exceptions or error messages, the model may be returning wrong predictions.

3. Missing or broken features and sudden shifts in the population distribution are among the causes of wrong predictions in ML.

4. In the diagram, logs and metrics feed alert rules. The alert text should say what was expected and the interval evaluated, and give the responder a clear link or course of action.

5. If the service stops completely, there may be no logs, averages, counts, or sums. A heartbeat alert fires when the expected ping does not arrive within the configured period.

The source describes ways an alert can stop working over time: becoming too sensitive, which causes alert fatigue, or becoming dulled and never firing again. That is why anyone should be able to edit the threshold to calibrate signal and noise.

Scaling tips: a minimum sample size avoids firing without enough data, and using a sample of the data instead of the full set contains cost, as long as the underlying distribution is large enough. And, like any code, the alert should be tested.

03 · RETRAINING

Retraining without giving up control

The source states that models lose performance because they were trained on a static snapshot of the world, and that often the simplest way to update them is to retrain with newer data, without new features.

1. The diagram shows the retraining path: data, pipeline, pull request, CI/CD, and the model in production.

2. Before automating, the source requires foundations: CI/CD, model monitoring, and a data pipeline that generates training and test sets programmatically and reliably.

3. The pipeline retrains with newer data and compares baseline and challenger on an unbiased test set that is not part of the training sets.

4. Instead of deploying on its own, the pipeline's last step can open a pull request with the compared performance of baseline and challenger.

5. The pull request still needs manual approval before the deployment that replaces the model in production.

6. The way other teams use the model's predictions is called policy. When retraining, the policy may need adjusting too.

04 · PITFALLS

What to check before swapping the model

The policy is often the neglected part. According to the source, depending on complexity, it is harder to retrain or readjust the policy than the model, and using calibrated probabilities makes these adjustments easier.

Feedback loops deserve attention: use of the model affects the future training set. In credit, for example, you only observe default among those who received money, which adds a noticeable bias to the data collected after the model was introduced.

Other cautions from the source: the label of an example scored today may only become available after some time, and this should be considered when choosing training and validation periods. Retraining costs should be weighed against the expected benefit, and the pipeline should not be coupled to a single model or use case.

05 · LESSONS

How to bring this into an interview

In a design discussion, it is worth showing that you anticipate how the model will be watched, how the swap will be proposed and reviewed, and how the policy that consumes the score will be kept in sync with it.