From Black Box to Glass Box: Extracting Interpretability from Nori

Today, getting to those insights is slow and manual. Data scientists reach for a stack of complex statistical tools (correlation matrices, regression coefficients and variance-inflation checks, partial-dependence plots, feature-selection pipelines, and post-hoc explainers like SHAP) to tease out which of the hundreds of features actually drive a variable of interest. The machinery is brittle when features are correlated, it takes real expertise to wield, and the explanations approximate the model rather than being it. The question shows up in every domain:

• Credit: Across hundreds of bureau, transaction, and behavioral fields, which few really drive default, and do they raise risk or lower it?
• Healthcare: Among thousands of labs, vitals, diagnoses, and medications, which signals predict a 30-day readmission?
• Insurance: Of all the underwriting attributes on a policy, which ones move a claim's expected severity?
• Manufacturing: Across dozens of sensor streams on a line, which readings foreshadow a defect before it happens?
• Retail & telecom: Among hundreds of account and engagement features, which actually predict churn?

Synthefy-Nori is our tabular foundation model. It solves regression in a single forward pass (no training, no tuning) and beats tuned gradient boosting head-to-head, digesting all of those features at once. Like any deep model its weights are opaque. So, on its own it is a black box.

What's new is that we can pull the reasoning back out of it.

In high-stakes settings like credit, healthcare, and insurance, where "the model said so" is not an answer and regulators demand a reason, that is exactly what you need. Nori doesn't dodge this by being transparent; it earns it a different way. Because it isolates each feature's effect cleanly, probing it recovers the true contribution of every feature:

• Extracted, not approximated. We measure importance on Nori's own predictions, through its full pipeline. Because Nori is an accurate predictor, that probe recovers the real contribution of each feature, not a surrogate's guess about it.
• Clean feature separation. Nori attributes each feature's marginal effect precisely, so the extracted importances are trustworthy and every sign is checkable against domain knowledge.
• Rebuilt as a glass-box. The structure we pull out distills into a transparent additive model you can read and defend feature by feature.

Seeing It: Credit-Card Default

We use a public dataset of 30,000 credit-card customers, the UCI Default of Credit Card Clients set, with 23 features covering credit limit, demographics, and six months of bills, payments, and repayment status. We split 70/30 and lay out three insights.

Here is the payoff up front: the glass-box model we pull out of Nori, live. It keeps only the handful of most important features Nori flags, which together recover ~95% of its accuracy. Drag any feature's slider and its marker walks that feature's shape function while the predicted default probability at the top updates. The whole model runs in your browser; the three insights below show how we get there.

Insight 1: We can extract what matters

Nori won't hand us a feature ranking directly, but we can extract one by probing it. Shuffle one feature across the held-out customers, re-predict, and measure how far Nori's AUC falls. That drop is Nori's reliance on the feature, read straight off its predictions, through its entire transformer, not a proxy. Because Nori separates each feature's contribution cleanly, this recovers each one's true importance. To decide what counts as real, we calibrate against noise: inject a few random columns and flag anything at or below the strongest of them.

The most recent repayment status dominates everything else by roughly 3×, credit limit and the latest bill follow, and the weakest features sink below the noise floor.

The demographic columns (SEX, MARRIAGE, EDUCATION) barely move a real customer's risk and fall below the noise floor. Nori promotes the behavioral signals and demotes the demographics on its own.

Insight 2: The calls are right

Importance says how much; the model's shape functions say which way, and every sign is credit-sensible:

• Repayment status (latest): any recent missed payment sharply raises default risk, the textbook signal.
• Credit limit: larger limits, extended to safer customers, lower it.
• Demographics: essentially flat.

Nori isn't just accurate; it's accurate for reasons you can check. The signs line up with how credit risk actually works, which is what lets you trust the number.

Insight 3: A few features carry everything

Keep only the top features Nori flags, refit, and watch the accuracy. A single feature already reaches AUC 0.70, and the top 25% (6 of 23 features) recovers ~95% of the full-model AUC. The long tail adds almost nothing.

That is the whole case for a glass-box: the signal lives in a handful of features, so a small, transparent model is enough all along.

The Whole Model, on One Page

Use those top features with a transparent additive model (the prediction is a sum of one plottable curve per feature, plus a few pairwise interactions) and it matches Nori. Below is that model end to end: the inputs, each feature's shape function (with data density and effect direction), the interaction heatmaps, and the sum that yields the default probability. To score a customer, you read the curves and add them up.

How to read it. Each middle panel is one feature's shape function: the x-axis is the feature's value, the y-axis how much it moves the predicted default probability, and the sum of all seven (plus the baseline and the interactions) is the final prediction. Take repayment status (latest): while the customer pays on time or merely revolves (values ≤ 0) the curve sits slightly below zero, then the instant it crosses into arrears it leaps up (about +2.8 in log-odds at the extreme), which is why a single recent missed payment is the biggest driver in the whole model. The right-hand heatmaps are pairwise interactions: the extra effect of two features acting together, beyond their individual curves (red raises the probability, blue lowers it). In the top one, credit limit × latest bill, a large balance sitting on a small credit limit is tinted blue: the model tempers the risk there, because a near-full balance is normal for a small line, not a red flag. Everything flows into the Σ node, through a sigmoid, to the default probability on the right.

Try Nori on Your Data

Grab the code, weights, and docs:

• GitHub: github.com/Synthefy/synthefy-nori
• Hugging Face: huggingface.co/Synthefy/Nori
• Notebook: the full interpretability walkthrough on this credit dataset: view on GitHub or open in Google Colab

Point Nori at the dataset you're working on right now, and extract the reasoning behind its predictions. If you build something interesting, tell us on Discord or at insights@synthefy.com.