AI Fundamentals: What Do These Different AI Systems Actually Do?

In the last article, I talked through the basic building blocks of what we've come to call AI. What makes up "AI", what are they doing and how do you use them. The four blocks we covered were Inference, Data, Automation and Agency. If you're looking for a place to start - I'd recommend reading that first.

I also promised to use the next article to talk about how these building blocks work together and how you can practically use them. Most of the AI services that you're seeing hyped are various orchestrations of the four core blocks. Whether that's "agents", "workflows", "co-work" or whatever they call it, you can bet it's basically these four kids in a trenchcoat - so let's see how they stack up.

Stacking the Lego Blocks

Inference + Data Retrieval: AI That Knows Your Business

If you're just using the basic web interface, every time you open a ChatGPT chat, you're basically starting from scratch. The data sources that it defaults to generally come from its trained data or from a tool-based web search. This means you need to bring ALL the input required to accomplish your task if you're going to see value. For trivial tasks this can be fine, but people new to AI underestimate how much information they carry around in their heads that they assume the AI should know - and it doesn't, until you make that data available. Internal processes, conventions, problem solving approaches, historical data, previous requests, none of that is available by default.

To make that available, you need to capture that information, and then make it available to the Inference to use. There are a few ways to do this.

ChatGPT or Claude "Projects" are the simplest way to do this. They do two things:

• First, they have access to all of the chat history in that project, so if you told it last week that when you're reviewing a dataset to explicitly ignore columns M and N, it'll remember that in today's chat.
• Second, it introduces permanent "sources" for data that apply to the whole project. That can be as informal as a call transcript, or as formalized as a database connection but you can add data sources, and you can provide lots of them. They're source data that every chat in that project can access.

This same pattern — curated data sources plus inference — shows up across a lot of different products and each of them has value. We just talked about projects and sources. Custom GPTs often do the same thing, sometimes with some Agency sprinkled in. Chatbots are basically projects with declared sources that your customers can interact with.

What It Looks Like

If you're trying to figure out what to make for dinner tonight and you ask an AI, it's going to try to provide some recommendations based on popular things people eat, or maybe simple meals because it's trained to try to make life easy, but it doesn't know what you like, nor what you have available to use. Are you going to get a good recommendation? You definitely won't get a personal one.

Imagine before you get to cooking, you go through your fridge, freezer and cupboards and you created a list of every ingredient and amount, the cooking appliances and utensils you have, your cuisine preferences and your list of allergens. Now ask the AI the same question. Are you going to get better answers? Of course you are.

And yet you might still just make a PB+J. You do you.

How to Approach It

The Inference + Data stack is adept at a few things:

• Minimizing context loss when initiatives pass between individuals (think Omar takes over one of Adam's accounts while he's on vacation and doesn't want to drop the ball)
• Surfacing specific answers from larger content pools (think a customer asks specific questions about product service or warranty)
• Making recommendations based in collected data (think product development wants to know the most common customer service inquiries to direct the work in each sprint)

Once you've identified a use-case within your organization, you're going to need a knowledge-owner.

The knowledge-owner's job is to assess the current state of the data and its readiness for consumption by your Inference, as well as to manage and maintain its upkeep over time. When the data is human processes or documentation, there's often unwritten rules and conventions that aren't explicitly stated. For data, it's often haphazardly collected, or lives all over the place, and formalization and consolidation will help.

In reality, you'll probably deal with both. When a customer asks about Warranty information, you've probably got two different sets of data - the written text of the Service and Warranty, and a procedure and convention document that customers shouldn't see, but would guide recommendations, next steps and escalation protocols. It's the same training you'd give a call center worker, but just for the AI system.

Data and processes change over time, and the knowledge-owner's job is to maintain freshness. This governance is critical. Without it, the investment in tooling withers on the vine as the data expires.

Inference + Data Collection + Automation: AI That Learns and Improves its Recommendations

With Inference talking to Data, you start to get output that makes sense to your business and the world you work in. If you consistently give it feedback on how it's performing, it's going to improve over time, which is great, but let's be clear about what that feedback is - it's just more data. Because project-based tools can preserve context, source material, and prior feedback across conversations, your feedback can become part of the data-pool the system draws from later. Simple enough, but it still takes work on your part. What if you could do that less explicitly?

By adding in Automation - specifically around whether you approve of and use the output it provides - you can create a system where the data collection improves over time without you continually having to personally train it.

What it Looks Like

Let's ground this in something you're probably used to - Netflix recommendations. When you first kick off Netflix, it just shows you what's popular. It might ask you to choose some shows you liked before, or some genres that appeal to you, but it's working with nothing. Where it gets smart is by paying attention to what you actually do.

Netflix will generate a set of recommendations. If it keeps recommending the same thing to you over and over, and you never engage with it, you probably aren't interested.

It'll track that you've been offered "Love is Blind: Sweden" 12 times, and you've never watched it, so it's probably not worth showing you again.

If you're offered "Don't Look Up" and you watched 17 minutes of it and never went back, that probably didn't work for you either (and I probably won't invite you over for movie night).

But if you watched Bridgerton - every season - end to end - like 3 times, whether you hit that "Love It!" button on the feedback screen or not, it can confidently recommend to you Queen Charlotte, Pride and Prejudice and maybe even Legend of Shen-Li... and send you an invite to my masquerade watch party.

This is the part people usually mean when they talk about machine learning: the system observes behavior, compares outcomes, and adjusts future recommendations based on what seems to work.

How to Approach It

Practical ML is embedded ML. Because we're measuring implicit or explicit feedback, it makes it almost impossible to survive context-switching. You want it to feel effortless, not like additional training. To use this, you need an existing system or tool that someone is using, and you need to design the automated feedback loop into the system.

Within that system, you've got two options:

1. Identify an implicit signal during that system's use and assign it a clear pass or fail grade.
2. Make it dead easy to provide an explicit signal (thumbs up/down buttons) so that it actually gets used.

With either of those systems in place, you feed that data back to your system during its own recommendations. When someone searches your website for "how do I pay my bill" if they leave without clicking anything, the responses you gave probably weren't a good fit for that query. If they clicked a link and then clicked back and kept searching, that listing probably didn't help them. If they clicked a link out to the payment gateway, that's probably a "pass" that should be considered.

Work with your devs and system administrators in order to get these feedback loops in place, as they can provide meaningful improvements over time.

Agency + Automation: Getting Busy Work Off Your Plate

Are there any tasks you have where you do the same task over and over again? Check if X happened, if it didn't, update something. Get information from one system, move it to another system in an ordered way. They eat up your time, but they won't go away.

These kinds of tasks are quite literally what computers were built to make easier. Traditionally, there were a few barriers standing in the way of finally scratching that itch.

• All the involved systems have to be willing to talk to external services
• The data passing between these systems has to be clearly and reliably structured
• You needed to actually write code to make the two systems talk and pass data between them.

There were entire software companies dedicated just to getting popular systems to talk to each other reliably. And they made billions.

Today, all three of those barriers have started to come down and automation has become accessible. Automation services like Zapier, Make and n8n have allowed less technical users to build bridges between different sets of software. If you want an invoice in QuickBooks to trigger a new row in an Excel spreadsheet and an email to a mailing list you can do that - and it's become easier than it's ever been.

What it Looks Like

Automation works on triggers - events that happen either based on a clock or based on another thing happening. Automation services give you access to "When X, Do Y" style workflows that don't require you to check in, they just happen. When the trigger fires, they've built a library of API connectors that can talk to almost any program or service you can think of and make real changes where you need them. They've built the bridge between these services, making them work together instead of in silos.

When people think automation these days, they tend to think that Inference has to be in the loop. Agent systems offered by the AI providers tout this as a benefit - and it totally can be - but it's often not required, or only needs to play a tiny interpretive role. Instead of having a whole agent framework, having your n8n workflow send "look at this email, judge if it's a feature request, a sales inquiry or a support ticket and return one of feature, sales, support" and then having it route the email to the appropriate person can cost you less than a penny instead of the $20/month for their agent platform.

How to Approach It

There are two decisions to make when it comes to Automation - one is top-down, the other is usually bottom up.

The top-down decision is service selection and architecture. What jobs can be automated and which can't? What governance is in place to ensure quality of service delivery? What happens when an automation goes wrong? None of these questions should be answered on a per-workflow basis; there should be organizational rules that implement approval, outcomes and error-handling so that your team has confidence that the "silence" that comes from automating a noisy task doesn't mean it stopped happening.

The bottom-up decision is going to be the actual implementation of the automation, and the best source of that information is generally the person doing the noisy task. If the guys on the floor in fulfillment and shipping are spending 3 minutes on every order manually pulling down the shipping and tracking information and pasting it into the "Shipping Notification" emails depending on what shipping method the customer selects, you're going to want to ask them exactly what they're doing so you can get that off their plate. That doesn't mean they're doing the implementation - your automation owner can do that - but they definitely need to be consulted.

Agency + Data + Automation + Inference: AI that Figures Out How to Solve Problems

The synthesis of all of these services coming together is where things really start to get wild.

• Take the user-independence of Automation (do things when other things happen, not when I tell you.)
• Add the Agency of the ability to connect to a variety of real-world services that you interact with
• Give it the Data and context it needs to understand not only what you're working with, but also what you're trying to accomplish
• And then give it a "brain" from an Inference LLM to be able to understand messy data, language semantics and logical thinking.

Now you've got a semi-autonomous digital Agent - and this is where "Agent" comes from - that has the potential to solve real-world problems.

Automation by itself solves problems where you fully understand the problem. You have to be able to walk through each step, identify what's needed and tell the automation to "do X, not Y". When you add in inference, you're able to ask an Agent to look at a problem and then reason through how to solve it by itself - without the explicit instructions required of plain automation.

This opens up a universe of questions that can be surprisingly hard to answer:

• What can you think up for a computer to do when you're not paying attention?
• How tight of a leash do you want on it?
• How much data access are you willing to give it?
• How much are you willing to spend as a service to have it do its thing?

"Hold on a second - that's basically a person" you'll say. And that's exactly right. And it comes with all the same risks, rewards and responsibilities.

• You have to give it a job description, including limits on its authority.
• You have to set up access, and figure out what it can see and what it can't.
• You have to come up with work for it to do - this might seem super easy, but just like pulling in a new employee, can you actually find work for it to do all day? That can be harder to do than you think.
• You have to integrate its input/output into existing human flows, so you're not doing double work or disrupting things people do.
• You have to watch it, assess its effectiveness, provide feedback, continue to train it and build its capacity.
• You have to live with the consequences of when it screws up - especially if it has any live access to clients, production systems or other outcomes.

The Catch

If you've made it this far, you're probably realizing that setting these things up requires a lot of work, training and effort. It does.

There's a reason why at this point in AI's development, people are still hesitant about it. There's absolutely value to be had, but the technology is being pitched as an instant cure-all to everything that ails you, when in reality it's a significant lift to get it operating properly and maintain it over time. That's not a reason to avoid it. It's a reason to go in with your eyes open.

AI tools make it easier than ever to build a quick fix to a common problem. That is useful. It is also dangerous.

Because once a few teams start building those fixes independently, you are no longer just experimenting with AI. You are quietly creating infrastructure: data flows, automations, access permissions, business logic, and dependencies that people will eventually rely on.

That works great right up until it doesn’t.

In the next article, I’m going to talk about the trap most companies are walking into right now: AI sprawl. Not because people are being careless, but because the tools have made it incredibly easy to build without first deciding what the system is supposed to become.

‍