This post is also available as a video on the Modern Software Engineering Channel
Both Daniel Terhorst-North and Matt Wynne have suggested that what we call Test Driven Development should be called “Example Guided Development”, because that captures the essence of all of BDD, ATDD and Specification by Example. It’s a particularly good framing for working with agentic AI tools too, they often seem to do better with a concrete example of than only a theoretical description of what you want. Examples are at the heart of the EXACT coding workflow from Marco Emrich and Ferdinand Ade – and – they have data to back up that choice. I’ll be interviewing them about it in today’s video, and how you can get the benefits of TDD with agentic AI.
On the Modern Software Engineering channel we have a series of videos we call ‘AI Briefings’ where we interview some of the innovators in this space. Do check out our playlist for previous editions. My particular focus is on how Test-Driven-Development is being re-imagined – because it clearly doesn’t work the same way when AI tools are writing all the code. Today I’m delighted to be talking with Marco Emrich and Ferdi Abe. Marco is a Primary IT consultant and Software Architect with Codecentric. Ferdi is a Senior Consultant and Developer, also with Codecentric. Together they have developed a set of AI skills and a workflow that they call “EXACT”. They showed me this approach in action on an exercise and I was impressed with the empirical way they’ve developed this approach, that lets them get the benefits of TDD when using agentic AI.
Where does EXACT come from?
Studio Emily – I began by asking Marco and Ferdi what led them to develop the EXACT coding workflow.
Marco – So we did some experiments, mostly at conferences, but also at client projects. And what we found out is that when vibe coding came around, we had these issues that mostly everybody has who tried vibe coding. And what you get is something that produces code, but maybe not exactly what you want. And also you have a lot of issues with code quality. So the ideas and concepts that we used in the past, we learned to do this as humans using test-driven development, using refactoring, still works. And we use this as a counterpoint to vibe coding, so actually profit from the benefits the LLMs give us, but still end up with good quality.
Marco – I mean, of course it’s a word play. So we want to get exactly what we want, not what you get with vibe coding.
Studio Emily – I think we have all experienced poor quality from LLMs with vibe coding, and many of us have been looking for good solutions to get exactly what we want. Marco and Ferdi have been working on this over a period of several years, and getting some great results.
Studio Emily – Before the interview, Marco and Ferdi shared a picture of the EXACT workflow with me. It begins with Example mapping – that’s a technique from Matt Wynne of BDD fame – for collaborating with stakeholders to map out a user story. Then you work with the LLM to turn that into a test list – that’s an idea from Kent Beck’s Canon TDD – where you keep track of the whole problem you need to solve in a natural language list of ideas for test cases. You use this list in the inner TDD loop – one item from the list at a time, going through red, green, refactor, building up a working implementation.
I asked them to explain each step, starting with Example Mapping.
Ferdi – in this example we used example mapping as a collaborative modelling technique, which basically is made for humans. It’s really engaging, it’s motivating, you have to think hard about what you’re building. And so it builds up the mental model really quickly, we think. And then also we can see in the test list step, there’s just two icons, one is a human and one is kind of this technical brain, which is showing the LLM basically. And so this also means that here we are collaborating. So for example, in our exact coding harness, the LLM is checking if we are missing any obvious permutations, if you’re missing any obvious examples in our suite.
Studio Emily – I like this emphasis on collaboration with other humans to build a shared understanding, and using the strengths of the LLM to improve it and turn that into an actionable list of test cases.
Measuring the effect of Example Mapping
Studio Emily – Another thing that impresses me about Marco and Ferdi’s approach is that it is empirical – driven by experiments with different LLMs and programming languages and harnesses. They have developed a whole test lab to try out different approaches and see what works. They have also invented a couple of new code katas, (I love it! New code katas!) which of course aren’t previously in the training data for the LLM. They have a problem description which they present to the LLM to solve, and a complete test suite of acceptance tests which they don’t show the LLM, that they can use to assess how well it implemented the requirements.
I asked Marco and Ferdi to show some of their results.
Emily – Would you talk me through what these numbers mean?
Marco – The first one is basically looking at the lever of the example mapping. We tried to describe the Katas, basically just the rules, the acceptance criteria in prose, and run it through the LLM. So this is basically vibe coding. And then we did the same thing, but did example mapping before that. So we gave the LLM clear examples, what we want to achieve in the Kata. And you can see that the correctness increases by 60, 70%, 80%
Studio Emily – These data are clear – compared just giving it a problem description in prose, the LLM builds something much closer to what you want when you do example mapping first. But is it realistic – wouldn’t it be better to work on giving the LLM a better description of the problem in the first place? Ferdi explained that this test is pretty similar to the situation he faces in his normal work.
Ferdi – I see a lot of success stories from parts of the software world, where, for example, the team at Claude Code, they’re basically building a tool themselves for themselves. So they’re the users and the stakeholders and everything. But in our daily business, we usually are, I don’t know, insurances, banks, e-commerce companies, whatever. And usually, we come in and we are not the experts. And so there is always a lot of ambiguity in the requirements.
Comparing with Spec-Driven Development
Studio Emily – Ambiguity in prose – yes – We’ve talked before on the Modern Software Engineering channel about the value of acceptance tests to specify what to build – and Example mapping is a complementary approach. I asked Ferdi to compare that with other augmented coding approaches, like spec-driven development.
Ferdi – before we had the phase of a vibe coding, and then later on all these spec-driven development frameworks pop up on GitHub with like lots of stars and everything. These frameworks, for example BMAD or others, have these different phases and in each phase they make a big picture plan or they make an architecture plan or whatever. After a second or third wave and it becomes more and more lines of markdown. But we think that this does not really improve the UX of the people that are building the stuff a lot – you kind of have the same “looks good to me” feeling you have you always had it with a really big Pull Request, and so we thought we have to basically front load or shift left some of the thinking and we thought that example mapping is a great tool for that. We also like stuff from the DDD world, like event storming or domain storytelling.
Inner Loop TDD
Studio Emily – The first part of the EXACT workflow is all about human understanding and collaboration, and not too much in the way of markdown files. Rather – examples, sticky notes and conversations. Ideas from Domain Driven Design and Behaviour Driven Development.
Let’s look more closely now at the inner loop and how we’re getting the LLM to do TDD. Marco showed me this data about another one of their experiments:
This table shows a comparison of “Plain TDD” with “Exact Coding”, and a third variant which we’ll come to in a moment. Basically “Plain TDD” means simply telling the LLM to ‘do TDD’ with no further explanation. This is much cheaper in token cost, but the EXACT coding workflow does better in other areas.
“Longest function” is perhaps obvious what it means – lower is better, but I’ll need to explain the others. Basically, cognitive max is a static analysis measurement of code complexity – lower is better – and mutation score is how good the tests are at finding bugs – higher is better.
What we see here is the code written using the Exact workflow, is much higher code quality, much better unit tests, although at a higher cost in tokens.
Researching the benefits of SubAgents in TDD
Studio Emily – Those are results from a very recent experiment. Marco and Ferdi started doing these experiments way back. Ferdi told me about the first iteration of their approach and how the testing lab came about.
Ferdi – I mean, we basically started out with one command back then. There were no skills back then. We started out with one command, a TDD command. And then I read more and more about how LLMs work and how Context Engineering works. And so then I basically asked like, “Hey Marco, shouldn’t we actually have kind of sub agents?” Because I mean, there’s this famous loss in the middle paper, right?
Which basically says the longer the context gets, the more gets lost in the middle of the context. So I thought, okay, it would make sense basically because you have these clear steps in TDD. It would make sense to have each phase in a single subagent with a fresh context, right? And this is basically how Marco said, okay, at one like, I think Sunday evening, he said, okay, I’m gonna research this quickly. And so this whole framework, research framework, came to life then.
And basically to answer the question, would it actually be better to have separate subagents with separate context for each step?
Studio Emily – A Sunday evening project that turned into an entire test lab for agentic harnesses. I’m sure you’ve all had projects that got out of hand like that! Anyway, that’s where the third line in this data table comes in. Marco and Ferdi have done lots of experiments. For this case, for Opus 5.5 and Claude code, they get better results with two agents- the main agent doing most of the TDD workflow, and a separate subagent for the ‘refactoring’ step. This gets them measurably better code quality. Although not by much. The difference used to be bigger.
Marco – So when we started with this approach, it was Opus 4.6. And with 4.6, the spread was much larger even. So with 4.6, the dedicated refactoring agent brought a lot of additional value. Nowadays, the additional value from the refactoring agent is not so strong, but the additional value from having a dedicated TDD approach is still visible.
Predictive TDD
Studio Emily – Another detail of the specifics of how they ensure good code quality, is to ask the LLM to do ‘predictive TDD’. This is an idea popularized by Ted M. Young in his TDD game, and you may have seen it in the training tool Cyber-dojo. Marco and Ferdi explained what Predictive TDD is and why it helps:
Marco – The important point is that before you do any TDD step, you have to predict what the result would be. So if I do a red step, I first have to say, this test will be red. And for what reason will it be failing? What is the error message that I want to get? So this is something we did even before LLMs, because it helps us as humans to confirm that we still are in touch with reality, that we still understand what’s actually going on in the code and in the environment. And now this helps the LLM to check if it’s still in alignment with reality.
Ferdi – we would need to bring a LLM researcher here, but we think that this works because basically the prediction … brings the problem to the front of the context for the LLM in a sense, right? So we think that’s why it makes a difference in our data.
Human in the Loop
Studio Emily – This all sounds like a pretty sophisticated TDD process. I wanted to know how much supervision the agent needs to succeed. Do you still need a human in the loop?
Marco – actually I think there is no clear answer to this question right now. And it shifts all the time also. So what we built into our harness is that you actually get the choice to select the autonomy level that you want to have. So you could decide to go into every step, look after every red, after every green, after every refactor, maybe only after the refactor, maybe only after three tests, or you want to run it completely autonomously. This is also a possibility. So I think at least you need to decide as a human how much human loop you want to have. And for me personally, I found that the more I actually look after it, stay in the loop, understand what the LLM is doing and make course corrections, the better the result actually is.
Ferdi – for a lot of these problems, the human loop is still really important because we first have to kind of build a mental model of the information. We have to kind of get our mental model clear of what we actually kind of need, what the world looks like. And then we can build something. There the human loop is still really important.
How to start with the EXACT workflow
Studio Emily – It sounds like you still need quite a bit of human skill to get the best results. I asked Marco and Ferdi how easy it would be for someone with little experience of TDD to pick up these agentic skills and be successful.
Ferdi – Marco often says he tried teaching TDD for 20 years, and now finally something listens! It’s a joke, of course, but I do think that the hard, one of the hard things always has been the thinking and the learning aspect of TDD.
Marco – I think you could start without having full TDD knowledge and get a deeper understanding the more you use it. Having a deeper TDD knowledge, of course, would also help because then you could improve the harness, feed your own knowledge into it. But it’s not so hard to get started.
Studio Emily – If you’re interested in trying out the harness that Marco and Ferdi have developed, it’s all open source on Github, and actually so is the test lab that they use to evaluate and develop their approach. They are also working on a book, originally in German but soon to be released in English.
Conclusions
Marco and Ferdi have an approach to firstly get an LLM coding agent to build the right thing – with Example mapping or another kind of collaborative modelling – and also to build the thing right – with Predictive TDD and perhaps separate refactoring subagents. Most of all though, we’ve met two engineers using empirical methods to continually refine and improve their way of working. Great stuff. This is published as a video on the Modern Software Engineering Channel, after all!
