AI Fundamentals: Why Your AI Implementation Falls Apart And How to Fix It

In the AI Fundamentals - What Do These Different Systems Do?, I talked through how the four core blocks - Inference, Data, Automation and Agency - get assembled into the AI products that you see today, and how to use combinations of those blocks to build different types of functionality. AI is getting baked into everything, and whether you're assembling it yourself or paying for AI as a service, it's becoming both better and more prevalent.

But there's a trap that many companies are walking into right now: AI sprawl.

The Democratization Trap

AI tools have democratized building stuff. Where it used to take a team of designers and developers weeks to put together a tool to solve a problem or connect disparate systems, today non-technical users can use a tool like Claude Code, Loveable or Codex to build complex tools to create something they need in a few hours. And that's amazing. Problems get solved without the need for a huge investment, and people can get on to the next problem.

That's why what comes later feels so bad.

The problem is that because it's easy, everyone builds independently. Each person solves a real problem on their own, but unless someone's carefully monitoring what's being built and how, you wind up with lots of little independent, often redundant systems. You're building business infrastructure - systems and services that your team will come to rely on to get stuff done - but without a central plan or governing authority.

That generally only works for a while.

The Slow Cost of Quick Wins

So what does sprawl look like?

Different or similar sets of data are being saved in different places and don't talk to each other.
Parvin's created a client database in one service, Michelle's got a similar data set in a Google Sheet. HubSpot had the canonical record, but neither of them are using it, so the data is getting out of date.

Multiple inference services are being used to do similar things but on different cost centers.
Marketing is using ChatGPT, IT is using Claude, most of the rest of the org is using Copilot, the Leadership team is testing Reclaim.ai, meetings are being recorded by TL;DV. Each of those inference services are being paid for independently, can probably be consolidated into a single system and likely don't talk to each other. It's also more complicated to track overall token costs when the services are all over the place.

Various automations are being set up and relied upon, with different levels of logging, reporting and error management.
Automations are being set up using Zapier or an Agent service in order to help speed up more mundane tasks, but they're not built to a reporting standard. Most don't report errors. Some report errors by email or Slack to the owner (and nobody else). When people get sick, go on vacation or leave the company, you lose access not only to the error reporting, but often to the system itself, which could be running agentic workflows against your systems with no human in the loop.

Codebases built with different conventions, on different software stacks that nobody owns.
If your business infrastructure runs on Microsoft, and people are off building tools in Claude Code or Loveable or elsewhere, unless the authors are being deliberate about exactly what they want and to what standard, they're going to get whatever that system is most comfortable building, or thinks is best for the situation. Claude might build one user a Python FastAPI app. Loveable's going to make a React application for another user. They're all going to use different data storage techniques and services. The front-ends will all be built in different ways. Who's going to be expected to clean that up when anything goes wrong? Your techs that were hired to work on the Microsoft stack?

Each of these scenarios introduces cost, risk and a support surface that your business wasn't intentional about creating - it was democratized. And it shows up in 3-6 months after people really get humming with building new things.

So What Do We Do?

So what? Don't use AI? Slow down? How do you realistically manage sprawl?

First of all, this piece isn't here to fear-monger. We're here to find a path to sustainable problem solving, so that's going to be our approach.

In a fast-moving market, slowing down can look tone-deaf. If you think you're behind competitors in terms of AI or service offerings, you're weighing two kinds of risk at once — technical and financial. If a little AI sprawl helps you land that big sale or ship that new feature, you could beat your competition on that client or customer and deal with technical debt later.

That's real math. We all do it.

But you need to plan for how to manage that debt. It's like a credit card. The bill comes due. You can take on debt now, but you need to start saving and putting systems in place to rework that debt at the same time, so they you're not crushed by the cost later.

Solving for Sprawl

In order to do this right we need to understand who understands and owns what lane.

The reason why AI-driven tools and democratization are so powerful is because the end-user of the tool is the designer. There's no gap in requirement understanding, and no broken telephone.

• Your salesperson knows they need to track a deal better than your software developer does.
• Your data analyst knows what information they want to see and the pattern they're looking for - they just need the tools to find it.

These bottom-up decisions are driven by domain-knowledge, and those in the field know their field better than anyone, so they build great tools.

What they don't know is how to build a sustainable IT infrastructure. How data governance works. How to implement security safeguards. How cost centers work. It's system knowledge. It's not their field, and until now, it's never been their problem.

Before AI tooling, to get something built, you had to bring the domain knowledge and the system knowledge together. You'd start a project, you'd do discovery, your engineers would build something and you'd have user acceptance testing. That process delegated responsibility to those who knew best. Domain knowledge provided the requirements, and system knowledge set the constraints.

You still want your IT team making decisions around how IT infrastructure is built. You still want your subject-matter experts getting as close to the outcome they want as possible to do their best work. What's changed is the speed it has to happen at. Your builders and your infrastructure owners are usually after the same thing: something that works, that lasts, that doesn't create a mess for someone else to clean up. They just tend to trip over each other getting there, because one side moves at builder-speed and the other may still be built for meeting-speed.

Old Skills, New Speed

The reality is that sprawl isn't new. It happened (and still happens) in IT infrastructure all the time. There were two governance approaches to IT infrastructure.

One was to enforce all decisions and approvals through human gatekeepers. This works to enforce the rules, but makes change happen slowly. At the speed of AI, this doesn't really work anymore.

The other was to codify infrastructure decisions in "infrastructure as code" (IaC) and orchestration frameworks that set the rules in advance and then allowed elastic deployment of new resources as they were required. This same philosophy is what's required to manage sprawl.

Users with AI tools are making infrastructure decisions in your organization right now. They're spinning up Claude Code and making that dashboard reporting app that you've been asking for for 18 months and they'll have it done before they clock out for the weekend. It might save data differently, pay for AI differently and be written in a programming language nobody at your office uses, but it'll be done.

If you pipe all your infrastructure decisions through human gatekeepers, those gatekeepers will only learn about the app once it's live and demoed to leadership. All they can do at that point is punitive enforcement.

IaC and Coding policies let you feed your infrastructure decisions to the agent in advance.

When Claude Code fires up, it can read your policy documentation and, if we assume your infrastructure is built on Microsoft:

• It can only use CoPilot inference, using these API keys
• All data has to live inside the Azure Cosmos DB. It has to create it's own DB and user with scoped permissions, but data housing all happens in one place.
• Error monitoring is non-negotiable. There's an org-standard way to implement it, and standard reporting channels that must be followed.
• Applications need to run on Azure Static Web Apps, need to leverage NextJS with Typescript, use Tailwind for CSS, and include the styling bootstrap that ensures brand consistency.

All of these decisions can be made by your IT team, codified, distributed and read every time someone starts on a new tool, feature or service. But to be applied, these decisions need to be made now, before the next tool gets built, so that it can follow the conventions you set.

Moreover, with those pollicies codified, you can often refactor the existing tools to fall in-line. The closer they are to policy the easier this is. Using the same tool that built it, each tool can review itself against the newly set policies and make modifications to bring itself inline with spec.

This is actually a strong case for getting policy in place early, even if it's not complete. If you've got a partial ruleset (say you've worked out what stack you want things built in, but not exactly where data should be housed yet), any tool built during that period will at least be the right stack. That means less debt later.

The Real ROI

At some point someone in the room asks what this is actually worth. Fair question.

The tempting answer is headcount. AI's supposed to mean doing more with less. For most companies that math doesn't hold up. You don't have five people doing one job you can trim to three. You've got three people doing the work of five, and no line left to cut.

The real ask isn't "replace people with AI." It's "let the people we have go further." Better decisions, less time lost to friction. We already know AI tools can help domain experts do more. The goal is to truly understand the cost of what's being built without getting surprised later.

Sprawl is what stands between you and that. Every hour spent figuring out which spreadsheet has the real numbers, or why the dashboard doesn't match the CRM, was supposed to go toward the job you hired someone to do. Nothing about it looks urgent. It just quietly eats the outcome you were trying to buy.

Where This Leaves You

None of this means self-hosting everything or writing a fifty-page policy manual before anyone touches Claude Code. That's its own trap. The fence gets so tall people build around it instead of inside it.

Your system thinkers need to make a handful of decisions, intentionally, before the next tool gets built instead of after: where data lives, who owns the automation layer, what the model strategy is, who's accountable for each piece. Consolidation isn't cleanup you do later. It's a choice you make now, before the mess is big enough to need cleaning up.

Like most topics in life, there's more below the surface. This piece is about governing what gets built, the stack, the data home, the inference costs. What happens once agents are live and acting on their own in production is a bigger conversation. We'll get into that later.

The next piece in this series gets practical: how you actually start. No matter where you are on your AI adoption journey, I'll try to give you an area worth attacking that can pay real dividends.