At Meta, they’ve got this iconic poster of a rocking horse with the words, “Do not mistake motion for progress.” Man, does that hit home. We’re seeing it firsthand with clients in the banking and financial sector jumping on the AI bandwagon. There’s this study showing teams using AI tools felt 20% more productive but were actually 19% less productive.

Why? They’re burning hours tweaking prompts, waiting for AI to spit something out, and then nitpicking the results. Less than 44% of AI suggestions even get used without heavy edits. Meanwhile, everyone’s high-fiving because it feels like they’re moving faster, even as the numbers tank.

The kicker? Without hard data, these teams would’ve kept pouring gas on the fire. They felt productive. Their bosses saw more reports, more drafts, more “output.” Everyone’s smiling—except the business metrics, which are screaming for help.

Here’s what we’re telling clients to avoid becoming another AI cautionary tale in 2025:

1. Pick one metric that actually matters.
   Not a dozen KPIs or some fancy dashboard no one understands. One number that drives the business. For one client, it’s fraud detection rate. For another, it’s hours of manual review automated. Whatever it is, make it the North Star for the quarter.
   Look at JPMorgan Chase—the $600 billion bank behind Chase cards and countless financial services. They don’t mess around with “vibes.” For their COIN platform, an AI tool for legal document review, the primary metric was hours automated. It crushed 360,000 hours of manual work annually. Clear pass/fail: Does it save time and reduce errors? No fluff.
   Motion is just noise. Progress moves the needle.
2. Test AI tools like you’re running a science experiment.
   The best companies we work with don’t just roll out AI and pray. They run tight, time-boxed trials with control groups. Think Swift, the global payments network, collaborating with banks like BNY Mellon and Deutsche Bank. They didn’t deploy AI fraud detection across the board. Instead, they ran pilots using live transaction data against historical patterns, measuring anomaly detection accuracy site-by-site. The goal? Tackle the industry’s $485 billion annual fraud hit by spotting patterns no human could. Only scale after proving it cuts fraud without false positives exploding.
   Define success before you start. Write it down. Measure it. If the tool doesn’t deliver, kill it fast.
3. Limit the chaos—cap work-in-progress.
   AI makes it stupidly easy to start stuff. Need a risk assessment model? Done. Ten compliance report drafts? Poof, generated. But starting isn’t shipping. We see clients drowning in half-baked ideas because AI lets them spin up endless variations.
   Take a page from agile transformations in banking, like those at ING or through lean practices we’ve implemented. Limit what your team’s working on—say, no more than three AI initiatives in flight at once. You don’t get to start a new project until you finish or ditch the last one. Why? Because ten half-built models are worth less than one that actually launches and reduces operational risks. AI just makes this trap bigger—more motion, less progress.

2025’s shaping up to be the year of AI-fueled chaos. But the clients we see winning? They’re not the ones moving fastest. They’re the ones making it harder to ship. They demand proof. They add guardrails. They stop more than they start.

That rocking horse poster? It’s a warning. Motion without progress is just a fancy way to waste time and money. While everyone else is rocking back and forth, our clients are the ones actually getting somewhere.

By: Matt Slonaker

3x Author & Publisher