How AI Works¶
Three properties explain most of what your teams will encounter when they use AI.
1. Probabilistic: It Is Making Its Best Guess¶
AI is trained on massive datasets, but it does not have everything in its knowledge base. Under the hood, it predicts the most probable next response based on patterns it has learned. The results often feel like expertise.
AI can look things up, read files, search the web, and use tools, but whether it does depends on what tools are available and how you prompt it. By default, it is likely to generate a response from what it already knows rather than go verify something first. That means it may be applying probability to fill gaps rather than flagging that it is uncertain.
The challenge: AI does not signal what it has high confidence about versus what it is guessing at. A low-confidence response looks just as polished as a high-confidence one. AI can be confidently wrong.
What This Means for You
- AI output requires verification. Always. There is no confidence threshold where verification becomes optional.
- You should prompt AI to flag uncertainty, cite sources, or verify its claims. It will not always do this by default.
- In specialized or classified domains where information is not freely available online, AI has less to draw from and is more likely to fill gaps with guesses. Your own organizational and domain context becomes even more important.
2. Non-Deterministic: Same Input, Different Output¶
Ask AI the same question twice and you will get two different answers. Both may be valid, but they will not be identical. This is called being non-deterministic.
Think of it like asking five colleagues to summarize the same meeting. All accurate, all different. This is not a bug. It is a property of how the system generates responses.
You can use this to your advantage. Ask AI the same question a few times and compare. Where the responses agree, you can have higher confidence. Where they diverge, look more carefully.
What This Means for You
- You cannot rely on AI producing the same output twice, even with identical inputs.
- Consistency across a team or organization requires giving AI explicit constraints, not hoping it will repeat itself.
- The more specific your instructions, the tighter the cluster of possible outputs. Specificity is your primary tool for controlling variation.
3. Limited Memory: What AI Retains and What It Uses¶
Every new conversation with AI starts from zero. It has no memory of previous interactions. The moment you close a conversation and open a new one, everything from before is gone. This is called being stateless. Within a single conversation, AI tracks what you have said. Across conversations, it retains nothing.
A note on memory: Some AI tools have started to incorporate cross-conversation memory, retaining certain information between sessions. But these work by saving summary notes, not replaying the full conversation.
The Context Window: An Oxygen Tank¶
Even within a single conversation, AI can only hold so much in working memory at once. This is called the context window. Think of it like an oxygen tank: every message you send and every response AI generates uses up air. As the tank runs low, AI starts paying less attention to things earlier in the conversation.
When the tank runs low, you have two options:
- Start new. Wipe the conversation and start fresh. Your project files are untouched, but the conversation is gone.
- Summarize and continue. Condense the conversation into a summary and keep working. The gist is preserved, but the details are lost.
In Your AI Assistant
Your AI coding assistant has built-in commands for managing the tank:
| Command | What It Does |
|---|---|
/new |
Starts a new conversation and resets the tank. Your project files are untouched. |
/compact |
Condenses the conversation to free space while keeping the thread alive. |
/sessions |
Browse previous conversations and resume where you left off. |
Your AI coding assistant has built-in commands for managing the tank:
| Command | What It Does |
|---|---|
/clear |
Wipes the conversation and resets the tank. Your project files are untouched. |
/compact |
Condenses the conversation to free space while keeping the thread alive. |
/resume |
Returns to a previous conversation where you left off. |
Your AI coding assistant has built-in commands for managing the tank:
| Command | What It Does |
|---|---|
/clear |
Wipes the conversation and resets the tank. Your project files are untouched. (You can also press Ctrl+L.) |
/compress |
Condenses the conversation to free space while keeping the thread alive. |
/resume |
Returns to a previous conversation where you left off. Launches an interactive session browser. |
Your AI coding assistant has built-in commands for managing the tank:
| Command | What It Does |
|---|---|
/clear |
Wipes the conversation and resets the tank. Your project files are untouched. (Or /new to start fresh without clearing the terminal.) |
/compact |
Summarizes the conversation to free space while keeping the thread alive. |
/resume |
Opens a session picker to return to a previous conversation where you left off. |
What This Means for You
- If you need AI to know something, you must tell it every time, or give it a file that contains what it needs to know.
- Long conversations lose detail over time. If AI starts missing things you told it earlier, the tank is running low.
- The fix is not a better model. The fix is manufacturing findable context: giving AI the information it needs, reliably, every time it starts.
Know the Properties, Know the Risks¶
| Property | The Risk | The Guardrail |
|---|---|---|
| Probabilistic | AI may guess rather than verify, and will not signal low confidence | Prompt for verification; establish review processes |
| Non-deterministic | Results vary across team members and sessions | Specific instructions and standardized formats |
| Limited memory | AI loses organizational context between conversations and detail within them | Context files and standing instructions |
These are not flaws. They are properties, and they tell you where your practices should focus to mitigate the risks.
See It for Yourself
Partners | ~3 minutes total | Split into partners. Each person uses their own machine.
Both open a new conversation with your AI assistant and send the same prompt:
Tell me what it means to explore a new frontier.
Compare your results. Notice:
- Non-determinism: Same prompt, different responses. Both valid, neither identical.
- Probabilistic: Are there statements that sound confident but you do not trust?
- Limited memory: Neither conversation knows the other exists. Each started from zero.
The rest of this Lesson teaches you how to control it.
Key Insight
These three properties are not flaws to work around. They are the surface area you need to manage. Knowing where AI guesses, varies, and forgets tells you exactly where your practices need to go.