nuFormer: transaction foundation model for credit and fraud

01 · CONTEXT

Plenty of data, value hard to extract

A bank accumulates large volumes of data (transactions, in-app events, customer interactions), but extracting value from them remains hard. Because they are sequential and loosely structured, these data have historically led teams to tabular models built on hand-crafted features. This is the classic starting point of ML systems design at Nubank.

According to one of Nubank's researchers, using traditional models means building an entire engineering pipeline to extract value from the data, and this requires people with deep domain knowledge working the data by hand. This dependence makes the process less scalable and more expensive as volume and complexity grow.

02 · FROM A MODEL PER PROBLEM TO A BACKBONE

A shared foundation for many applications

nuFormer is a foundation model developed by Nubank's AI Core team to learn representations of financial data at scale. The goal is not to solve a single problem, but to build a reusable foundation for different applications across the bank.

1. The raw data goes through a pipeline of hand-crafted feature engineering.

2. Without a foundation model, each problem requires a model trained from scratch.

3. This work depends on domain experts who manipulate the data by hand, which becomes more expensive and less scalable as volume and complexity grow.

The alternative is what the blog describes: instead of training a model from scratch for each problem, nuFormer learns a representation of financial behavior that can be reused across many contexts, giving applications a shared starting point.

1. Transactions are converted into sequences of tokens.

2. A transformer, with self-supervised learning, builds representations directly from the raw data.

3. The result is a representation of financial behavior that can be reused across many contexts.

4. On top of this foundation, the model allows fine-tuning for tasks such as credit, fraud detection, and product recommendation.

5. Among the uses already in place, the blog cites credit, loans, income prediction, and cross-selling.

03 · TRANSACTIONS AS LANGUAGE

Treating the history as a sequence

The paradigm shift is in how the data is viewed. Instead of treating transactions as isolated records, the idea is to interpret them as sequences with structure, context, and meaning, and converting these sequences into tokens makes it possible to capture deeper behavioral patterns.

The work combines Transformer architectures with self-supervised learning to build representations directly from the raw data. nuFormer mixes structured and textual transaction attributes and allows fine-tuning for tasks such as credit scoring, fraud detection, and product recommendation.

The work started with transactions, and the team's current vision is multimodal: also integrating behavioral signals, in-app interactions, and other sources of information.

04 · FROM TRAINING TO PRODUCTION

A systems problem, not just an ML one

According to the blog, the biggest challenge is not the model itself, but the engineering needed to make it work. This includes pipelines, GPUs, and distributed training, but also the production stage, where regulatory governance comes into play.

1. Training requires well-structured pipelines, GPU management, and distributed training.

2. In the financial sector, the model goes through several rounds of validation before going to production, to meet regulation and internal standards.

3. At inference time, Transformer-based models tend to have higher latency, a sensitive factor in financial applications.

4. With specialized infrastructure and teams, it is possible to reach performance levels comparable to those of traditional models.

5. Ensuring that the model learns correctly remains among the biggest challenges, and requires continuous monitoring and metrics aligned with business impact.

05 · RESULTS AND LIMITS OF THE SOURCE

What Nubank says it achieved

The blog states that bringing these models to existing systems delivered significant gains in key metrics within a few months, surpassing improvements accumulated over years with traditional approaches. The model is already applied to credit, loans, income prediction, and cross-selling.

In Nubank's earnings release, the latest generation of nuFormer quadrupled context length, training speed, and inference speed, and reduced the cost of running models in production. The model is in production in three portfolios: credit cards in Brazil and Mexico and unsecured credit in Brazil, with small and medium businesses and cards in Colombia in testing.

The release describes nuFormer as supporting credit underwriting decisions, customer service, and growth across the company, backed by more than a decade of transaction history from more than a hundred million customers.