Coding Is Like Cooking

I’ve been favouring an Approval Testing approach for many years now, since I find it pretty useful in many situations, particularly for acceptance tests. Not many people I meet know the term though, and even fewer know how to use the technique. Recently I’ve put together some small exercises – code katas – to help people to learn about it. I’ll be going through them at a couple of upcoming conference workshops, but for all you people who won’t be there in person, I’m publishing them on github as well.

I’ve got three katas set up now, Minesweeper, Yatzy and GildedRose. If you’ve done any of these katas before, you’ll probably have been using ordinary unit testing techniques. Hopefully by doing them again, with Approval Testing, you’ll learn a little about what’s different about this technique, and how it could be useful.

Before you can do the katas, you’ll need to install an approval testing tool. I’m one of the developers of TextTest, so that’s the tool I’ve set up right now. Below are some useful commands for a debian/ubuntu machine for installing it.

I’m still developing these exercises, and would like feedback about what you think of them. For example I have Python versions for all three, but only one has a Java version as yet. Do people want more translations? Do let me know how you get on, and what you think!

Installation instructions

You will need to have Python 2, and TextTest. (Unfortunately TextTest uses a GUI library that doesn’t support Python 3). For example:

$ sudo apt-get install python-pip
$ sudo pip install texttest


For more detailed instructions, and for other platforms see the texttest installation docs. For more general documentation, see the texttest website.

You need to have an editor and a diff tool configured for texttest to use. I recommend sublime text and meld. Install them like this:

$ sudo add-apt-repository ppa:webupd8team/sublime-text-3
$ sudo apt-get update
$ sudo apt-get install sublime-text-installer
$ sudo apt-get install meld


Then you need to configure texttest to use them:

$ cd
$ mkdir .texttest
$ touch .texttest/config
$ subl .texttest/config

Enter the following in that file, and save:

[view_program]
default:subl
[end]
[diff_program]
default:meld
[end]

For convenience, I also like to create an alias ‘tt’ for starting TextTest for these exercises. Change directory to one of the exercise repositories, then a ‘tt’ command should start the TextTest GUI and show the tests for that exercise. Define such an alias like this:

alias tt='texttest -d python -c .'


Two of the exercises start with a small test suite for you to build on. There should be instructions in the README file of each respective exercise, to help you to get going. If you really can’t work out what to do, have a look at the sample solutions and see if that helps. These are also on github: Minesweeper-sample-solution, Yatzy-sample-solution, GildedRose-sample-solution

I’ve been interested for a while in the relationship between TDD and good design for a while, and the  SOLID principles of Object Oriented Design in particular. I’ve got this set of 4 “Racing Car” exercises that I originally got from Luca Minudel, that I’ve done in coding dojos with lots of different groups. If you’ve never done them, I do recommend getting your editor out and having a go, at least at the first one. I think you get a much better understanding of the SOLID principles when you both know the theory, and have experienced them in actual code.

I find it interesting that in the starting code for each of the four Katas there are design flaws that make it awkward to write unit tests for the code. You can directly point to violations of one or more of the SOLID principles. In particular for the Dependency Inversion Principle, it seems to me there is a very direct link with testability. If you have a fixed dependency to a concrete class, that is always going to be harder to isolate for a unit test, and the Tyre Pressure exercise shows this quite clearly.

What bothers me about the 4 original exercises is that there are actually 5 SOLID principles, and none of them really has a problem with the Liskov Substitution Principle. So I have designed a new exercise! It’s called “Leaderboard” and I’ve put it in the same git repository as the other four.

I tried it out last week in a coding dojo with my colleagues at Pagero, and it seemed to work pretty well. The idea is that the Liskov principle violation means you can’t propely test the Leaderboard class with test data that only uses the base class “Driver”, you have to add tests using a “SelfDrivingCar”. (Ok, I confess, I’ve taken some liberties with what’s likely in formula 1 racing!) Liskov says that your client code (ie Leaderboard) shouldn’t need to know if it has been given a base class or a subclass, they should be totally substitutable. So again, I’m finding a link between testability and good design.

Currently the exercise is only available in Scala, Python and Java, so I’m very open to pull requests for translations into other programming languages. Do add a comment here or on github if you try my new Kata.

Recently I became intrigued with something Seb Rose said on his blog about ‘recycling’ tests. He talks about first producing a test for a ‘low fidelity’ version of the solution, and refining it as you learn better what the solution should look like. In a follow-up post he deals with some criticisms that other posters had of the technique, but actually seems to agree with Alistair Cockburn, that it’s probably not important enough a technique to need a name. I disagree, it’s a technique I use a lot, although most often when using an approval testing approach. I prefer to call it simply iterative development. A low fidelity version of the output that is gradually improved until the customer/product owner says “that’s what I want” is iterative development. It’s a very natural fit with approval testing – once the output is good enough to be approved, you check it in as a regression test that checks it never changes. It’s also a very natural fit for a problem where the solution is fundamentally visual, like printing a diamond. I also find it very helpful when the customer hasn’t exactly decided what they want. In this kata, it’s not such an issue, but in general, quickly putting out a low-fidelity version of what you think they want and then having a discussion about how to proceed can save you a lot of trouble.

The other posters seemed to be advocating a TDD approach where you find ‘universal truths’ about the problem and encode them in tests, so you never have to go back and revisit tests that you made pass earlier. In order to take small steps, you have to break down the problem into small pieces. Once you have identified a piece of the problem and solved it, it should stay solved as you carry on to the next piece. That seems to be what I would call ‘incremental’ development.

There’s a classic explaination of the difference between iterative and incremental that Jeff Patton came up with a few years ago using the Mona Lisa painting. It’s a good explaination, but I find experiencing abstract concepts like this in an actual coding problem can make a world of difference to how well you can reason about and apply them. So I thought it would be interesting to look at these two approaches to TDD using the Diamond Kata.

I have a regular coding dojo with my team these days, so a few weeks ago, I explained my thinking about incremental and iterative, showed them Jeff Patton’s picture, and asked them to do the kata one way or the other so we could compare. I probably didn’t explain it very well, because the discussion afterwards was quite inconclusive, and looking at their code, I didn’t think anyone had really managed to exclusively work one way or the other. So I decided to try to force them into it, by preparing the test cases in advance.

I came up with some starting code for the exercise, available here. I have two sets of unit tests, the first with a standard incremental approach, where you never delete any test cases. The second gets you to ‘recycle’ tests, and work more iteratively towards the final solution. In both cases, you are led through the problem in small steps. The first and last tests are the same, the difference is the route you take in between.

When I tried this exercise with my team, it went a lot better. I randomly assigned half the pairs to use the ‘iterative’ tests, and the rest to use ‘incremental’ tests. Then after about 45-55 minutes, I had them start over using the other tests. After another 45 minutes or so I stopped them and we had a group discussion comparing the approaches. I asked the ‘suggested questions for the retrospective‘ I’d prepared, and it seemed to work. Having test-driven the solution both ways, people could intelligently discuss the pros and cons of each approach, and reason about which situations might suit one or the other.

As Seb said, ‘recycling tests’ is a tool in your developer toolbox, and doing this kata might help you understand how to best use that tool. I’d love to hear from you if you try this excercise in your coding dojo, do leave a comment.

I’ve been working on this Kata “Gilded Rose” at a few different coding dojos lately. There is even a video of a session I did at the “Tampere Goes Agile” conference recently. In the video, you can see me talking about my Principles of Agile Test Automation, which I have just written about, and updated in my last blog post.

I think these test automation principles are useful to think about when you’re doing the Gilded Rose kata. The basic plot of the Kata is that you’ve just been hired to look after an existing system, and the customer wants a new feature. Having a look at the code, you can see you’re going to want to refactor it a little before adding the new feature, and before you do that, you’re going to want some automated tests.

So the first part of the Kata is to add automated tests to the existing code. You’ve got a requirements document the customer has given you, and you can use it to identify test cases. You’ve also got the code which you can read and execute and work out what it does. The customer is happily using the code in production right now, so you can assume that the behaviour it has is the behaviour they want to keep, whatever it says in the requirements document. (hint!)

Warning – spoilers lie ahead! You should probably try the Gilded Rose kata for yourself before reading on!

When I’ve done this exercise with various groups, I’ve spent a lot of time discussing with people how to make their test cases really readable, and express the requirements clearly, and at the same time useful as regression protection when refactoring the code later.

When you design a test suite you have two main aims – to help you understand what the code should do, (and what it does now), and protection from regression failures when you update it. It can be a bit tricky to do both with the same test suite. If you focus solely on describing the requirements in an executable way, you tend to miss edge cases and there are gaps in the regression protection. If you focus only on regression protection, you’ll spend time analysing the edge cases, and measuring code coverage to see how well you’re doing, but the test cases can become quite hard to read and understand.

You can see for yourself by comparing this test case by Bobby Johnson with this text-based approval test. (It was written by several people at a GothPy meeting). Bobby’s test case is extremely readable and expresses the requirements clearly. He’s done pretty well on the edge cases, but I think he’s missing one or two*. With the text-based approval tests, it’s not so easy to understand what the underlying business rules are, although the regression protection is very good.

When I do this kata with a group, we spend some time discussing the various test cases we’ve come up with, and showing them on the projector. When we did this last week at the Booster Conference, people commented that showing these different test cases had given them a better understanding of “readability” and “regression protection”, and many went on to improve their test suites.

Once you’re reasonably happy with your test suite, the next task is to do the refactoring and add the new feature. How useful are your test cases for regression protection? It’s very easy to make refactoring mistakes in this kata, and you will be testing your tests! You may discover while refactoring that there are more test cases that you want to add. Version control can be pretty useful, so you can run the new test cases against the original code.

There’s also an interesting restriction on your refactoring options – the “Item” class is owned by a nasty-sounding goblin and he doesn’t want you to change his code, so if you do, you have to be prepared for some serious consequences! When comparing refactored solutions at the end of the dojo, this is often an interesting discussion point – did you change the Item class? Is your new design so great that you’re prepared to argue with the goblin for it?!

I havn’t tried this, but I would actually like to try running the text-based approval test against all the refactored solutions at the end of the coding dojo, as input to the retrospective. I think this test covers all the edge cases very well, and would reveal any refactoring mistakes that were not caught by the tests people had developed themselves. That would be interesting feedback to have!

If you havn’t tried the Gilded Rose kata yourself, I do recommend it for practicing writing good test cases. I’d be happy to get a pull request from you if you want to translate the exercise into your favourite programming language, or you can do it in the original C#, as Bobby suggests.

If you’re interested in taking part in a coding dojo with me, I’ll be at several conferences later this year: ACCU in Bristol, XP2013 in Vienna and Test Automation Day in the Netherlands.

* I believe he’s missing a check that the quality of backstage passes doesn’t increase past 50

The Cyber-Dojo tool was designed by Jon Jagger as en environment where you can practice your coding skills. I’ve used it a few times now with groups at coding dojos and code retreats, and I think it’s a pretty useful tool for those contexts. (See also my last post which talks about using Cyber-Dojo during Global Day of Code Retreat).

One of the advantages of Cyber-Dojo for a Coding Dojo, (or Code Retreat), is that you don’t waste much time at the start of a coding session setting up a coding environment. The session facilitator creates a Cyber-Dojo instance in advance, and puts the practice-id up on a whiteboard or projector where everyone can see it. Participants just point their browsers at cyber-dojo.com, enter the practice-id, and very quickly get coding.

Cyber-Dojo supports about a dozen programming languages, and has starting positions set up for about 30 code katas. What is less known, is that it also allows you to set up any kata or starting position you like. I thought I’d take this opportunity to create some documentation for this feature:

1. create a new cyber-dojo instance by going to http://cyber-dojo.com/ and pressing “setup”
2. Select the programming language you want to use
3. Select “Verbal” from the list of katas
4. Click “OK”, then make a note of the “practice-id” – it’s also in the url. Press “Start” to enter this cyber-dojo instance.
5. Edit the code files with your starting position, and update the instructions with the details of the kata exercise. Basically get the cyber-dojo instance set up to the position you want people in your Coding Dojo to start from. Run the tests as often as you like until you have everyting as you want it.
6. Click on the “fork” icon on the left hand side to create a new cyber-dojo instance starting from this position, and note the new practice-id. You can give this id to your Coding Dojo participants.
7. You can also publish a url that will automatically create a new cyber-dojo instance from this position, so people can create their own cyber-dojo instances. The form of the url is:

http://cyber-dojo.com/forker/fork/(practice-id)?avatar=(your animal)&tag=(number of the traffic light to fork)

I’ve used this feature to set up a number of Refactoring katas in cyber-dojo, for example the Tennis kata:

• Tennis in Python
• Tennis in Ruby
• Tennis in Java
• Tennis in C++
• Tennis in C#

Perhaps sometime Jon will add a page on cyber-dojo.com that lists these kinds of additional available starting posistions, (hint!), but for now, you’ll have to keep track of them yourself.

By the way, do let me know if you try out this Tennis Refactoring Kata in Cyber-Dojo and how you get on with it. I welcome comments on this blog or on my github repo.