Avoiding a Knock-out: A Client Story That Inspired Me at Ensemblex

I’m Matt Slonaker, and I recently joined Ensemblex as Chief Revenue Officer, stepping into a role that’s already opened my eyes to the incredible work happening here. With a background in fintech and a passion for driving growth through innovative solutions, I was thrilled to dive into our client projects. One case study, in particular, has left a lasting impression on me—a challenging yet rewarding journey with a consumer lender in Latin America that turned a model rollout disaster into a success. I’m sharing this story because it encapsulates why I’m excited to be part of Ensemblex and what I believe sets us apart in the industry.

A Rollout Gone Wrong

When I joined, I learned about a client—a consumer lender with over $200 million in assets across multiple Latin American markets—whose core product is personal loans ranging from $400 to $15,000. Before 2018, they relied on a generic off-the-shelf credit score for underwriting. Eager to modernize, they partnered with a reputable data and modeling firm to create a custom origination score. The new model promised better performance and the potential to grow their business without added risk, launching with optimism in mid-2018.

But the excitement was short-lived. Within a month, early payment delinquencies surged, forcing the client to pull back originations and apply emergency rules-based overlays to manage spiraling portfolio risk. As I reviewed the details, I discovered the culprit: the model was built on a dataset skewed by inconsistent “knock-outs” (KOs)—rules-based applicant rejections that had shifted over time. This left the development data unrepresentative of the client’s current applicant base. The model shone within its narrow data confines but was blind to these gaps, and the implementation plan didn’t address how KOs should evolve. It was a stark lesson in the dangers of context-blind modeling, and I was intrigued by how Ensemblex stepped in to fix it.

Stepping In to Turn Things Around

The client’s analytics team had acted swiftly, using univariate and multivariate analysis to identify risky segments and add more KOs to stem the damage. It worked, but at a cost—KOs are blunt tools that limit risk differentiation, complicate underwriting, and stifle growth. That’s when they turned to Ensemblex, and I found myself captivated by the challenge they brought us: build a smarter ML underwriting model to control losses, simplify the process, and drive expansion, all while avoiding the pitfalls of the past.

What Drew Me In: A Thoughtful Approach

What struck me most as I explored this case was Ensemblex’s meticulous, context-driven strategy. The client’s underwriting funnel had 19 KOs—ranging from age minimums to credit risk filters and segment-specific rules tied to the old model. Rolling them all back at once was too risky, given the lack of performance data from rejected applicants. Instead, the team collaborated with the client to prioritize seven KOs for initial rollback, blending their expertise with data-driven insights. This partnership approach—something I value deeply—resonated with me and highlighted why I’m proud to be here.

The data work was equally impressive. The team analyzed KO history to assess blind spot risk, favoring rollbacks where inconsistent application provided some performance data. They replicated current KOs in the evaluation dataset, ensuring analysis focused on eligible applicants—a detail I’ve seen overlooked elsewhere but which proved critical here. They even leveraged “natural tests”—empirical opportunities from inconsistent KO application over time—to validate the model’s risk-sloping across populations. It was this blend of rigor and creativity that hooked me.

The Swapset Analysis That Sealed It

I was particularly fascinated by the swapset analysis—a deep dive into where our model diverged from the incumbent. The team examined swapset sizes, distributions, and top features like positive references, delinquency days, and application channels. With 84% agreement with incumbent decisions and swap-ins skewing toward riskier scores, the results aligned with the client’s recent risk stabilization efforts. Top feature values showed swap-ins outperforming swap-outs but lagging in-ins, building trust in the model’s logic.

Deep dives into segments like younger and non-upgrade borrowers—initially concerning due to higher risk—revealed a smart twist. The model swapped out toxic subsets while adding worthwhile applicants, dropping bad rates from 23% to 12% for age and 21% to 13% for bank classification. Using empirical data from natural tests, the team confirmed swap-in performance, cementing confidence in the rollback plan. This level of detail and adaptability is why I’m excited to contribute to Ensemblex’s mission.

A Success That Inspires

The results speak for themselves. The model reduced risk by 11% versus the current rules-based system and 40% versus the post-rollout chaos, while eliminating seven KOs to streamline underwriting. Six months after launch, risk indicators beat projections, and the client is poised to hit pre-COVID volumes faster than expected, with plans to test further KO rollbacks. This turnaround—from a near-disaster to a growth engine—shows the power of a model built with context, and it’s a story I’m eager to share.

Why I’m Sharing This

This case study embodies what drew me to Ensemblex: a team of elite data scientists, with decades of experience from companies like Capital One and CitiBank, who fuse cutting-edge ML with a deep understanding of client businesses. A model detached from context is a liability; here, it’s a trusted lever for growth. As a neobank or fintech, if you’re looking to unlock ML’s potential, I invite you to explore www.ensemblex.com or contact us at [email protected]. This story inspires me daily, and I can’t wait to help more clients achieve similar success.

— Matt Slonaker, Chief Revenue Officer, Ensemblex