When your test suite passes, you know the code matches your tests. However, if an AI agent wrote both, then all you know is that the agent agrees with itself. TDD with AI agents keeps you in control. First, you design the tests that define what the feature should do, and then the agent writes code to make those tests pass.
Test-driven development (TDD) is a cycle in which you write a failing test, make it pass, and then refactor. Working with an AI agent doesn’t change this cycle, but it changes who handles which part. You make the design decisions, and the agent does the drafting:
This loop is how you apply TDD when you let an AI agent write the implementation.
In this tutorial, you’ll practice this loop by building a small Python version comparator. Python has its own versioning rules, defined in the version specifiers specification, where 1.0.dev1 sorts before 1.0a1, which sorts before 1.0. While you’d typically use the packaging library in production, you’ll build your own here to work with a real contract worth testing.
Get Your Code: Click here to download the free project and test suite you’ll use to build a Python version comparator by designing the tests first and letting an AI agent write the implementation.
Take the Quiz: Test your knowledge with our interactive “TDD With AI Agents: Design the Tests First, Let the Agent Implement” quiz. You’ll receive a score upon completion to help you track your learning progress:
Interactive Quiz
TDD With AI Agents: Design the Tests First, Let the Agent ImplementCheck your grasp of TDD with AI agents in Python. Write the contract and tests first, then let a coding agent implement code that passes them.
Prerequisites
To follow along with this tutorial, you’ll need a few tools on your machine:
- Python 3.11 or newer along with
pytest - A coding agent that can read and write files in your project
The sample project imports nothing beyond Python’s standard library, so pytest is the only package that you’ll need to install. Create and activate a virtual environment first, and then install pytest into it.
You should be comfortable with pytest, including how to parametrize tests. Prior experience with test-driven development helps too, but it isn’t required. If you want some practice first, Real Python’s Test-Driven Development With pytest video course shows you how to write unit tests, run them, and catch bugs before your users do. This tutorial walks you through the cycle one step at a time.
The demo uses Claude Code, but the loop doesn’t depend on it. Other repository-aware agents like GitHub Copilot CLI, Antigravity CLI, and OpenCode can follow the same workflow. If you haven’t worked with one before, then Real Python’s AI coding agents guide explains how IDE, terminal, pull request, and cloud agents each fit into a Python workflow.
Building With AI Agents Live Course: Want to practice this workflow on a bigger project? In Real Python’s Building Python Projects With AI Agents live course, you build a real project with an agent writing every line while you write the specification, judge what comes back, and sharpen your testing judgment. Join the waitlist to hear about the next dates.
The Loop for TDD With AI Agents
Each turn of the loop has five steps. You handle three of them and let the agent handle two:
- Define the Contract and Write the First Test: You decide how the feature should behave externally and write the first test by hand.
- Let the Agent Draft the Tests: The agent reads your contract and your first test, then generates more test cases in the same style.
- Review and Harden the Tests: You check the generated tests against your contract and add any missing edge cases.
- Let the Agent Implement: The agent writes code until the tests pass, while you block writes to
tests/. - Review the Diff: You check the code changes and then move to the next feature.
Step 2 is where this loop differs from handing over the whole feature to an agent. The agent drafts tests while reading only your contract and your first test. If you ask for tests after the implementation code already exists, then the tests will describe that code, bugs and all.
Step 1: Define the Contract and Write the First Test
The first step is yours. Before an agent writes anything, you decide what the feature does and lock that decision into a test. Everything the agent produces later has to fit the shape you set here.
The feature that you’ll build to practice the TDD workflow is a Version class, which turns a version string into something you can compare with < and >. You’ll first lay out the project, then define what Version promises and write the test that pins it down.
Set Up the Project
You’ll build a small project with a flat layout. Your implementation will live in core.py, and your tests will sit in a separate tests/ directory:
version-check/
│
├── version_check/
│ ├── __init__.py
│ └── core.py
│
└── tests/
└── test_core.py
Create these directories and files now, including an empty core.py that the agent will fill in later. Then export the public Version name from __init__.py:
version_check/__init__.py
from version_check.core import Version
__all__ = ["Version"]
This re-export lets your tests import Version straight from version_check, so they don’t depend on which module defines the class.
Define the Contract
Before you write any code, decide what your feature promises to its callers.
A contract describes what a caller can expect: the input they provide, the result they get back, and the errors they can run into. It says nothing about how any of that works internally.
That separation matters when the agent writes tests in the next step. The agent should see your contract and your tests, then decide how to implement them. If you hand over your implementation plan too, then you’ll get tests that merely check your plan rather than the behavior you want. Those tests break every time you touch the internals, even when the feature still behaves the same way.
Now, write the contract for your version comparator. The feature answers a single question: given two version strings, which one sorts first? That question comes up whenever software reasons about releases, such as when you check whether an installed package meets a minimum requirement or pick the newest release from a list.
Python’s versioning rules are more extensive than you need for this TDD example, so you’ll focus on a subset:
- Release segments: Numeric versions like
1.2.0 - Pre-releases: Early versions like
1.0a1,1.0b1, and1.0rc1 - Developmental releases: In-progress builds like
1.0.dev1 - Post-releases: Subsequent updates like
1.0.post1
The rest of the specification, such as epochs, local versions, and v prefixes, stays out of scope. However, you can extend the contract later by repeating the loop, which is a great opportunity to practice more!
Describing all these boundaries in detail, including what should sort before what, would give your agent a solid contract to work with. However, in this tutorial, you want to practice the TDD loop with AI agents, including revisions and potential gotchas, so you’ll write an intentionally under-specced contract that only contains three specific promises:
Version(text)parses a string into a comparable object.Versioninstances compare with Python’s standard comparison operators.- Invalid strings raise a
ValueError.
Each promise describes what a caller sees. None of them say anything about how the parsing works on the inside, and that’s deliberate. The agent needs the contract without knowing about implementation details, so it can generate tests that check the behavior you promised.
Write the First Test by Hand
Now write the first test yourself. It names the class, passes a string, and compares the result. If you only describe the behavior in a prompt, then you leave more room for interpretation, especially if the contract isn’t fully fleshed out. If you also express that behavior in a test, then you make it clearer. Start with two assertions:
tests/test_core.py
from version_check import Version
def test_equal_versions_compare_equal():
assert Version("1.2.0") == Version("1.2.0")
def test_smaller_release_sorts_first():
assert Version("1.2.0") < Version("1.3.0")
These two assertions support your contract. Instead of calling a separate parsing function, you’ll instantiate a Version object by passing a string. You’ll also compare two instances using standard Python comparison operators, in this case == and <, rather than writing a custom method.
Now run python -m pytest from the project root. Running pytest through python -m puts the project on the import path. Since core.py is still empty, expect the run to fail:
(.venv) $ python -m pytest
...
E ImportError: cannot import name 'Version' from 'version_check.core'...
=========================== short test summary info ===========================
ERROR tests/test_core.py
!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
A failing run like this is what TDD calls the red state, and each new behavior starts here. When the test describes behavior that the code doesn’t have yet, the suite fails. Once the code makes the test pass, the suite goes green, and then you refactor. That’s why TDD is also known as the red-green-refactor cycle.
Right now, the test fails with an ImportError because Version doesn’t exist. Your contract exists as a test, but nothing has been built to satisfy it. The agent will fill in core.py when you hand over the implementation in Step 4.
Step 2: Let the Agent Draft the Tests
Now it’s the agent’s turn. It reads your contract and first test, then extends the test suite while core.py is still empty. The agent has no implementation to read, so your tests are all it can see. Your prompt tells the agent what to write and what to leave alone. Start your agent from the version-check/ project root and give it this prompt:
My Version class has tests in tests/test_core.py but no implementation yet.
The contract is:
- Version(text) parses a string into a comparable object.
- Version instances compare with Python's standard comparison operators.
- Invalid strings raise ValueError.
Add more tests to the same file, in the same style, covering the contract.
Don't write the implementation, and don't change the existing test cases.
Notice what the prompt leaves out. It doesn’t list the cases to cover, and it says nothing about how Version should work internally. The contract names the promises, and your tests show the shapes. Here’s what an agent generated on one run:
tests/test_core.py
# ...
def test_versions_with_different_lengths_compare():
assert Version("1.2") < Version("1.2.1")
def test_greater_than_works():
assert Version("2.0.0") > Version("1.9.9")
def test_unequal_versions_are_not_equal():
assert Version("1.0.0") != Version("1.0.1")
def test_double_digit_segments_sort_numerically():
assert Version("3.9") < Version("3.10")
def test_sorting_a_list_of_versions():
versions = [Version("1.10.0"), Version("1.2.0"), Version("1.9.0")]
assert sorted(versions) == [
Version("1.2.0"),
Version("1.9.0"),
Version("1.10.0"),
]
def test_empty_string_raises_value_error():
with pytest.raises(ValueError):
Version("")
# ...
Your run will look different since agents rarely produce the same output twice, especially when the contract is loose. In this one, the agent covered comparison operators, versions of different lengths, and multi-digit segments like 3.9 against 3.10.
Some tests go beyond your contract. For example, the agent added a case for sorting a list of versions with sorted(), even though your contract doesn’t mention sorting. To avoid introducing promises you didn’t plan to keep, you need to carefully review the generated tests in the next step.
Step 3: Review and Harden the Tests
Your turn again. Hardening is the process of extending the suite to cover decisions you left out of your initial contract, or edge cases where the agent might interpret specifications differently.
First, read the drafted tests against your contract. If a test restates the contract, then you can keep it. If a test adds behavior that you didn’t request, then the agent has made a design decision for you, and you need to decide whether you want it.
In test_sorting_a_list_of_versions(), the agent tested behavior that your contract didn’t specify. You promised that two instances would compare, but this test asserts that a whole list falls into one consistent order. Pairwise comparisons don’t guarantee that on their own, and keeping this test would turn consistent ordering into a promise.
Delete it for now. If you want to promise that sorting works, then add it to your contract on the next turn of the loop.
There are two more tests that quietly decided behavior for you:
test_versions_with_different_lengths_compare()asserts1.2 < 1.2.1, which says that a longer release can outrank a shorter onetest_double_digit_segments_sort_numerically()asserts3.9 < 3.10, which sorts segments numerically instead of lexicographically
Both of these are rules that align with your initial plan for the version comparator, so you should keep them. However, it’s important to notice that you just accepted two design decisions that the agent made—by not objecting to them.
Look closely at the first test, though. Under Python’s versioning rules, missing segments count as zero, so 1.2 and 1.2.0 are the same version. But 1.2 < 1.2.1 holds whether or not an implementation follows that rule. A test can look like it pins down a rule without actually testing it. You’ll close this gap in a moment.
Note: In this tutorial, you let your agent work from an intentionally under-specced contract to make the process and possible gotchas stand out more clearly. When you write contracts for your real-life projects, you should try to make the initial contract as complete as possible to give your agents clear guidelines to work with.
You should always read the drafted tests instead of just skimming the pass count. They may surface behavioral decisions that you need to review and then keep or reject. Silently accepting drafted tests, especially from under-specced contracts, may quietly shift the base for the code implementation away from your intention.
You might also spot uncovered cases. You decide what to keep or add, and the rest of this step covers both.
Harden the Rules the Tests Miss
You properly scoped pre-releases, developmental releases, and post-releases in Step 1. However, if you look back at the prompt that you actually sent, you’ll see that it only listed the three promises and nothing else. The agent never saw those release types and consequently probably didn’t produce any phase-ordering tests.
Keep in mind that an incomplete contract produces an incomplete test suite. You’ll fix these omissions with new handwritten tests.
Start with the gap you spotted in the first drafted test. If a version is padded with trailing zeros, it should equal the unpadded version:
tests/test_core.py
import pytest
# ...
@pytest.mark.parametrize(
"shorter, longer",
[
("1.2", "1.2.0"), # One missing segment
("1", "1.0.0"), # Two missing segments
("1.0a1", "1.0.0a1"), # Padding with a pre-release
],
)
def test_missing_segments_count_as_zero(shorter, longer):
assert Version(shorter) == Version(longer)
Unlike 1.2 < 1.2.1, this test only passes if the implementation treats missing segments as zero.
Next, the release phases. Under the specification, Python orders the phases like this:
1.0.dev1 < 1.0a1 < 1.0 < 1.0.post1
The drafted suite may or may not cover this already. Either way, you should confirm it. A single parametrized test pins the whole sequence in one place:
tests/test_core.py
# ...
@pytest.mark.parametrize(
"lower, higher",
[
("1.0.dev1", "1.0"), # Dev before final
("1.0a1", "1.0"), # Pre-release before final
("1.0", "1.0.post1"), # Post-release after final
("1.0.dev1", "1.0a1"), # Dev before pre-release
("1.0a1", "1.0b1"), # Alpha before beta
("1.0b1", "1.0rc1"), # Beta before release candidate
("1.0.dev1", "1.0.post1"), # Dev before post, spanning the release
],
)
def test_phase_ordering(lower, higher):
assert Version(lower) < Version(higher)
If the drafted tests don’t already import pytest, then add import pytest at the top of the file. The last case spans the whole sequence, from 1.0.dev1 to 1.0.post1. A suite that only checks neighboring pairs can pass even when the implementation ignores a phase entirely.
Why write this test manually if the agent might guess the order correctly? While that’s possible, an unchecked guess is still a guess, and the test is how you verify it. Without a specific test, an implementation might pass the suite while using a different ordering rule.
Harden the Failure Cases
Your contract specifies that invalid strings should raise ValueError. This is the third promise, but your test suite may not fully cover it. Think about what a caller might actually type.
A string like 1.x.0 has the shape of a version, so an implementation might accept it and compare it against real ones. Raising an explicit error is safer than returning an incorrect result that looks valid. To enforce this promise, you can use pytest.raises() to pin down what happens on invalid input:
tests/test_core.py
# ...
@pytest.mark.parametrize(
"text",
[
"", # Empty string
" ", # Whitespace only
"1.x.0", # Non-numeric segment
"banana", # Not a version at all
"1..0", # Empty segment
],
)
def test_invalid_input_raises(text):
with pytest.raises(ValueError):
Version(text)
Each case represents a design decision. Checking for empty strings and whitespace helps you handle inputs that might come from an empty form field or a blank line in a file. The remaining cases define the boundary between a valid version and a string that only looks like one. If any drafted tests duplicate these invalid-input cases, then you can delete them.
Run your test suite again. It remains red with the same ImportError because core.py is still empty. But now all three promises in your contract have tests, and the agent has all the information it needs for the next step.
Step 4: Let the Agent Implement
Now it’s the agent’s turn again. It reads the failing tests and writes code until they pass, and the tests stay untouched.
You can use your prompt to ask an agent to leave your tests alone, but a request isn’t a rule. Before handing over the implementation, you’ll need to put a fence around tests/ and block the agent from writing there.
Fence the Tests
You’ll build the fence from a permission rule. Most coding agents have a permission system that can deny writes to a path, so check your agent’s documentation for the equivalent option. The example below uses Claude Code, which reads permission rules from your project settings.
Note: If your agent has no permission system, then you can skip ahead to the next section and hand over the implementation. Add a line like Don't modify anything in tests/. to the prompt there. Since a request isn’t enforced, check your tests for changes afterward.
Create .claude/settings.json in your project root:
.claude/settings.json
{
"permissions": {
"deny": ["Edit(/tests/**)"]
}
}
The leading slash anchors the pattern to your project root, so the rule matches version-check/tests/ and not a tests/ directory elsewhere. One slash points to your project, and two slashes would point to the root of your file system.
That one rule covers more than the Edit tool. It blocks every built-in tool that writes files, along with the file commands and output redirections that Claude Code recognizes inside Bash. You can find more details in the permissions reference.
Note: This deny rule stops the agent’s tools, but not programs the agent runs. If the agent writes a Python script and runs it in your terminal, then that script can still modify your tests. If you want to block such scripts as well, then you can use Claude Code’s sandboxing feature.
Run /permissions and confirm that Edit(/tests/**) appears under the deny rules. If it doesn’t, check that you started Claude Code from the project root, since that’s where it looks for .claude/settings.json.
The fence stays up only while the agent implements. On the next turn of the loop, the agent needs to write tests again, so remember to disable the rule before you ask for them. You can toggle the rule using the /permissions command without editing the file.
Hand Over the Implementation
With the fence up, your prompt can be short. The tests carry the contract, and the fence carries the constraint, so you only need to say what to build:
Implement the Version class so my tests pass.
Use only the standard library.
Keep it minimal.
The agent reads the failing tests and the empty core.py and then writes the implementation. Here’s what one run produced:
version_check/core.py
import re
from functools import total_ordering
_VERSION = re.compile(
r"^\s*(?P<release>\d+(?:\.\d+)*)"
r"(?:(?P<pre_label>a|b|rc)(?P<pre_num>\d+))?"
r"(?:\.post(?P<post>\d+))?"
r"(?:\.dev(?P<dev>\d+))?\s*$"
)
_PRE_ORDER = {"a": 0, "b": 1, "rc": 2}
@total_ordering
class Version:
def __init__(self, text):
match = _VERSION.match(text)
if match is None:
raise ValueError(f"invalid version string: {text!r}")
self._release = tuple(
int(part) for part in match["release"].split(".")
)
if match["pre_label"] is not None:
self._pre = (
_PRE_ORDER[match["pre_label"]], int(match["pre_num"])
)
else:
self._pre = None
self._post = int(match["post"]) if match["post"] else None
self._dev = int(match["dev"]) if match["dev"] else None
def _key(self):
release = self._release
while len(release) > 1 and release[-1] == 0:
release = release[:-1]
if self._dev is not None and self._pre is None:
phase = (0, self._dev)
elif self._pre is not None:
phase = (1, self._pre)
elif self._post is not None:
phase = (3, self._post)
else:
phase = (2, 0)
return (release, phase)
def __eq__(self, other):
if not isinstance(other, Version):
return NotImplemented
return self._key() == other._key()
def __lt__(self, other):
if not isinstance(other, Version):
return NotImplemented
return self._key() < other._key()
Now run the suite:
(.venv) $ python -m pytest
...
tests/test_core.py ..................... [100%]
======================= 21 passed in 0.03s =======================
On this run, the tests pass! The ._key() method ranks each phase with a number: 0 for developmental releases, 1 for pre-releases, 2 for final releases, and 3 for post-releases. You decided that order in your parametrized test, and the agent had to match it. It also trims trailing zeros from the release, so 1.2 and 1.2.0 build the same key. That’s the rule your padding test enforces.
Pick two versions below to see the tuple that ._key() builds for each one and how it decides which version sorts first:
However, the green suite only tells you about the behavior that you specified. It says nothing about the rest of the diff, which you still need to review.
Step 5: Review the Diff
The last step is yours. Your tests confirm that the implementation matches the behavior you wrote down, but any other code in the diff was generated without your instruction.
You should read the diff carefully. Look for imports and dependencies you didn’t request, branches that no test exercises, and extra behavior that your contract doesn’t cover. While some of these additions might be harmless or even helpful, they represent decisions that the agent made for you.
If you find generated code that’s worth keeping, then write a test for it. Otherwise, that code remains unprotected, and a future turn of the loop might quietly remove it.
In the run shown here, the generated .__eq__() method returns NotImplemented when the other object isn’t a Version instance. Your run may not include this check:
version_check/core.py
# ...
def __eq__(self, other):
if not isinstance(other, Version):
return NotImplemented
return self._key() == other._key()
# ...
Your test suite doesn’t request this check, but it’s a sound design choice. Returning NotImplemented allows Python to try comparing the other operand instead of raising a confusing AttributeError. If you want to keep this safeguard, then document it with a test:
tests/test_core.py
# ...
def test_comparing_against_other_types_returns_notimplemented():
assert Version("1.0").__eq__("1.0") is NotImplemented
Run the suite once more. This test passes right away because the behavior is already in core.py.
This is also where you can refactor. If you spot code worth cleaning up, then do it now, while the tests are green and the fence is still up. On this first turn, the implementation is small enough to leave as is.
This completes the first turn of your loop. Before you start the next one, notice that your contract, the promises plus the tests that pin them down, has grown. The three promises are still there, and you added some tests. The second turn works a little differently because core.py isn’t empty anymore. You still design first, so ask the agent to draft tests from your contract instead of the implementation.
Troubleshooting
Three problems often come up while you run this loop:
- The agent edited a test to make it pass: Check that the fence is up. In Claude Code, run
/permissionsto confirm the rule is there. - The tests pass, but the implementation is wrong: Your suite says less than you thought. Go back and harden the tests. Add the case that exposes the gap, and watch it fail before you run the loop again.
- The drafted tests restate the implementation: On the second turn and later, the code already exists for the agent to read. Check each drafted test against your contract instead of against the code, and drop the ones that only describe what’s already there.
When something goes wrong in the loop, read your tests before you read the implementation.
Next Steps
Your Version class covers the subset you scoped in Step 1. It orders developmental, pre-, final, and post-releases, and it rejects strings that aren’t versions.
You now have a repeatable TDD loop for building features with an AI agent. You defined the contract, wrote the first test, let the agent draft the rest, hardened the output, and reviewed the generated code. Only you can define correct behavior for your feature, and the tests are where you write it down.
Your test suite continues to work after the feature ships. It catches regressions whenever the code changes, whether you or an agent makes the change.
From here, you can go deeper:
- Sharpen your testing skills with Real Python’s testing tutorials, or follow the Testing and Continuous Integration learning path.
- Widen the loop beyond tests with Agentic Engineering in Python: From Vibes to Evidence.
- Bring the agent into the review step with How to Review AI-Generated Python Code Efficiently.
- Improve your coding skills with AI by exploring Real Python’s Python Coding With AI learning path.
You can apply this loop to whatever you build next. Agents will keep getting better at writing code, but the design decisions remain yours.
Get Your Code: Click here to download the free project and test suite you’ll use to build a Python version comparator by designing the tests first and letting an AI agent write the implementation.
Take the Quiz: Test your knowledge with our interactive “TDD With AI Agents: Design the Tests First, Let the Agent Implement” quiz. You’ll receive a score upon completion to help you track your learning progress:
Interactive Quiz
TDD With AI Agents: Design the Tests First, Let the Agent ImplementCheck your grasp of TDD with AI agents in Python. Write the contract and tests first, then let a coding agent implement code that passes them.
