Human Judgment¶
The Guardrails Are Not the Whole Answer¶
You have now learned several layers of guardrails:
- Specificity shrinks what AI can guess at. User stories and acceptance criteria define exactly what to build and what "done" looks like.
- Context gives AI the information it needs about your project, your standards, and your domain.
- Automated tests verify that the software works the way your acceptance criteria say it should.
- Quality gates ensure that only tested, secure code reaches production.
These are powerful. But they are not the whole answer. There are decisions that no automated check can make for you. When should a system act on its own? When should a person review every action? How much autonomy is appropriate for a given task, in a given environment, with a given set of consequences?
These are judgment calls. And they are yours to make.
Two Models for Human Oversight¶
There are two well-established models for how people interact with systems that can act on their own:
Human-in-the-loop means a person reviews and approves every action before it happens. The system proposes; the person decides. Nothing happens without explicit human approval.
This is the safest starting point, and it is the one most organizations reach for first. But it has a scaling problem. When someone is approving ten actions a day, the reviews are thorough. When they are approving five hundred, the reviews become rubber stamps. The safety you thought you had quietly erodes under volume.
Human-on-the-loop means the system acts within defined boundaries, and a person monitors for problems. The system has autonomy within constraints. The person watches for anomalies and steps in when something looks wrong.
This model scales, but it requires trust. That trust is not built on hope. It is built on the guardrails you just learned about: automated tests that verify behavior, evals that check AI quality, and quality gates that enforce standards. You trust the system to act because you can verify that it is acting correctly.
Think of it like...
Think about how air traffic control works. Controllers do not manually fly every aircraft. Autopilot systems fly the aircraft within established parameters: altitude, heading, speed, airspace restrictions. Controllers monitor the whole system and intervene when something unusual happens or when conditions require a human call.
That is human-on-the-loop. The autopilot has autonomy within constraints. The controllers are not micromanaging; they are monitoring. And the entire system depends on instrumentation: radar, transponders, and protocols that make every action visible and verifiable. Without that infrastructure, the system does not work.
The Autonomy Slider¶
In practice, it is not a binary choice between in-the-loop and on-the-loop. It is a slider, and different tasks, even within the same project, can sit at different positions:
| Position | Human Role | AI Role | Example |
|---|---|---|---|
| Hands-on | Directs every action | Executes instructions | First time working in a new domain |
| Collaborator | Works alongside AI | Co-produces with guidance | Complex work requiring ongoing judgment |
| Approver | Reviews before execution | Proposes, waits for approval | Actions with real-world consequences |
| Monitor | Watches for problems | Acts within constraints | Routine tasks with good instrumentation |
| Observer | Reviews after the fact | Fully autonomous | Low-stakes, well-understood, well-measured |
Where you set the slider depends on two things:
- Can you undo it? If the action is reversible, higher autonomy is safer. If it is irreversible, you want more human involvement. Drafting a document that a person will review before sending is low-risk; the draft can be rewritten. Sending an email to a thousand people, deploying a policy change to production, or deleting records cannot be taken back.
- What happens if it gets it wrong? A formatting issue in an internal document is not the same as an incorrect alert that redirects a crew. The stakes determine where the slider sits.
For many of the organizations in this room, that second question carries serious weight. When missions are on the line, when compliance frameworks exist for good reason, when errors have consequences beyond inconvenience, the slider position is not a casual decision. It is a risk call.
And here is the key point: neither position is safe without the infrastructure. Human-in-the-loop degrades into rubber-stamping at high volume. Human-on-the-loop degrades into not watching if there is no instrumentation. Both require the automated tests, evals, and quality gates you have been learning about to function properly.
Discussion: Where Does the Slider Sit?
Team Discussion | ~3 minutes total | Discuss at your table.
Think about a system or process in your organization. If AI could assist with or automate part of it, where would you put the slider? Would you want a human approving every action? Monitoring from a dashboard? Reviewing after the fact?
What would need to be in place for you to move the slider further toward autonomy? What would need to be true before you felt comfortable?
Key Insight
Automated guardrails handle the checks that should be consistent and tireless. Human judgment handles the decisions that depend on context, stakes, and consequences. The question is not whether to trust AI. It is where to set the slider, given what happens if something goes wrong and what infrastructure you have in place to catch it. That answer will be different for every project, and it may be different for every task within the same project.