Archive for the ‘Uncategorized’ Category

When you inherit difficult code it can take weeks to become productive. Having the right tools for the job and knowing how to use them makes a huge difference. These videos show you how.

Sometimes you don’t know what a piece of code is supposed to do, but you do know that people are using it in production, and that it in some sense ‘works’. One approach I often use in this situation is Approval testing. It can get you test coverage quickly without having to understand the code.

Since you don’t know what the code is supposed to do, you can’t define in advance what results you expect. But, what you can do is run the code, accept whatever it does as ‘correct’, then invent scenarios that will exercise all the code branches. I’ve made a video of me doing just this on some rather hairy legacy code – The Gilded Rose Refactoring Kata. With the right tools the tests fall into place relatively easily.

I’d like to credit Llewellyn Falco who showed me this way to solve this exercise.

I recorded a screencast in three parts. This is the first part.

Part 1: Introducing the Gilded Rose kata and writing test cases using Approval Tests

About the Gilded Rose code

One of the exercises I’ve used for years to help programmers improve their skills is the Gilded Rose Kata. It’s a refactoring kata – the code needs cleaning up and tests adding so you can build a new feature. That is a realistic scenario that programmers often face in everyday work, but this exercise adds a fantasy twist. The code you have to work with keeps track of various magical items stocked at the Gilded Rose establishment. The new feature concerns support for “Conjured” items that have slightly different magical properties from the other items. The scenario is just weird enough to be fun and just realistic enough to be a useful exercise.

I didn’t design the kata originally, that was Terry Hughes. I spruced up the code a little to make it a better exercise and added some extra instructions to get you going. I also translated the starting code into a few different programming languages and put it up on GitHub. In the 5 years since then I have been delighted to see how popular it’s become. I’ve had over 50 contributors chip in with various translations and improvements, and at least 800 people have forked the project and presumably had a go at the refactoring.

I think the appeal of the exercise is partly the wacky scenario it throws you into, and partly how utterly terrible the code is at the start. If you do the refactoring well it actually looks really neat at the end, which is very satisfying.

Lift-Up Conditional

When you inherit difficult code it can take weeks to become productive. I’d like to show you the difference it can make when you have the right tools for the job and know how to use them.

Once you’ve got good tests in place you can refactor much more confidently. In my previous post I showed how to get good tests using Approval Testing. I’m pretty confident in these tests, so I’ve made a second video showing some initial refactorings I’d do to get this code cleaned up a little.

One of the techniques I’m using is called ‘Lift-Up Conditional’. It’s a manipulation of a long complex conditional statement that will let you group together all the statements related to one particular conditional. I haven’t seen this particular refactoring described in the literature before – it was Llewellyn Falco who showed it to me originally. It’s perfect for the Gilded Rose code which basically comprises one big complex conditional.

The other star of this show is IntelliJ. It has a lot of automated refactorings that come together to make ‘Lift-Up Conditional’ easy and it makes really short work of cleaning up this code.

This is the second screencast in the series. My aim is to show that with the right tools and refactoring know-how you can quickly become productive with the code, even without fully understanding the byzantine business rules.

Part 2: Refactoring item logic using ‘lift up conditional’

Replace Conditional with Polymorphism

When you inherit difficult code it can take weeks to become productive. I’d like to show you the difference it can make when you have the right tools for the job and know how to use them.

Once you’ve got the code cleaned up to the point where you can see the parts of the logic that belong together, you can start to create a better class structure. A classic refactoring for this is ‘Replace Conditional with Polymorphism’ which was first described in Martin Fowler’s book ‘Refactoring’.

The basic idea is that you create subclasses to encapsulate the logic concerned with each logical case. Your design becomes much more flexible if you need to add new types that are variations on types that are already there – as in this case.

This is the third screencast in the series. My aim is to show that with the right tools and refactoring know-how you can quickly become productive with this code, even without fully understanding the byzantine business rules.

Part 3: Replace Conditional with Polymorphism


Three factors affecting your agility

Let’s talk about cakes and baking. It’s similar to making software. No really! How quickly you can bake a fantastic cake depends on three main factors. Turns out, it’s the same three for software.

Note: This article was first published on Praqma’s blog

In essence, agility is about your ability to respond to change. I can gain a competitive advantage if I can update my software quickly when I see an opportunity in the market. As a software developer, I’m going to work together with other professionals to decide what new features to build at any one time. The important question is, when we’ve agreed on a particular change and realized the cost of delay is significant: how quickly can my team and I make this change?

How quickly can you make a change to your software?

Allow me to answer that question by comparing it with baking. How long does it take to bake a cake? As with software, I think there are three main factors.

The first factor is how complex a cake it is. How many layers? How many kinds of filling? How fancy will the decorations be? Similarly, changes to a piece of software vary in how big and complex they are, and that will affect how long they take.

The second factor is the skill of the baker(s). When you make fancy flowers out of sugar paste how convincing are they? Can you whip cream without it turning into butter? Do you know how to use a thermometer to get perfectly thick and creamy egg custard? If your skills are poor, you might need to throw out your first attempt and make a second batch of custard or cream or flowers, which obviously slows you down. Developing software is similar – skill, experience and knowledge of particular tools all come into play. If your code changes are poorly designed and fail the tests or the code review, that can similarly slow you down.

When baking, the next major factor for how long it will take is the kind of kitchen you’re going to work in. If I’m a contestant on a show like Bake-Off, I will have a sparkling clean workbench and all the gadgets and accessories I could possibly need. Dirty bowls and saucepans vanish as if by magic when the camera is turned the other way. Fantastic mouthwatering creations appear in a very short space of time.

In contrast, if I’m at home, I might need to clear a space on a worktop before I can start. I could waste valuable minutes hunting around for an electric whisk attachment in an overfull drawer. I might belatedly realize my cake tin is too small for the project I have planned or that I set the oven to the wrong temperature. Creating that amazing cake is not going to be as quick and smooth as it could be.

Time to Bake Cake Equation

The worst conditions for baking that I’ve experienced were probably in my shared student house at university. Not only were accessories like whisks and cake tins hard to come by, a huge pile of dirty dishes obscured the sink most of the time. After some weeks, we discovered one of the third years had decided his final exams were so important that he had started to leave all his washing up “for the first years to take care of”, as he put it. I hope you don’t have anyone like that on your team!

Unhappy Chef

In the same way, how long it will take you to build a feature will depend on the state of the environment you’re working in. For a software developer, that environment includes not only the build tools and programming language as you would expect, but also the state of the existing code. Is it clean and easy to understand? Is the current design amenable to the planned change? You might need to spend time cleaning up and fixing things before you can start.

There’s also the build and deployment pipeline the team has set up. Are the regression tests quick to execute and generally passing? Is the deployment automated? Can I observe logs and find bugs in production? If that infrastructure is well oiled and running smoothly, it will make a big difference for how soon my software change can be rolled out.

Time to Build Feature Equation

If you want to be really agile, you need to consider how to improve some or all of the factors in this equation. If you want to build complex, valuable features quickly, you need to make sure you have developers with the skills to build them. You also need to invest in the “kitchen” they are working in.

When I consult with different organizations, I often see problems with developer skill and poor development environments. Investment in both can lead to big gains in agility. You want the time to market for a new feature to be dominated by the essential complexity of that feature, not the accidental complexity that comes from poor code quality, testing and deployment automation. Coding is like baking. Working in a mess is not agile!

A write-up on learning away from work

Note: this post originally appeared on Praqma’s blog

Do you work on any hobby coding projects in your free time? Practice code katas? We all wish we could, but making time for learning away from work isn’t possible for everyone. So, who should pay for learning time?

Recently, I took part in a panel discussion at the Software Craftsmanship Conference in London. One of the questions from the audience was about finding time to learn new skills. Bob Martin, sitting to my right in the picture below, talked about the importance of spending your free time on reading, practicing code katas, and generally sharpening your skills. He said that we need to be able to call upon those skills when there’s a deadline and production software needs to be built. He also said that it’s unrealistic to think these skills can be acquired on the job. That practice needs to be done beforehand, at home, on your own time.

Picture from@cyriux

I agree strongly that software developers need to learn throughout their career. As professional knowledge workers it’s crucial for us to keep up to date and continually expand our toolboxes. Skills like Test-Driven Development (TDD) take time and practice before you become effective with them. Our industry moves very fast and new tools and techniques come along frequently.

I also agree that you need to practice in safe situations before applying a technique in production code. When I see badly designed code it’s often due to developers not fully understanding the libraries and frameworks they use. This causes them to skip testing or do it badly. What’s more, they often also lack techniques for refactoring and can’t improve the design once they’ve learnt more.

The problem is only getting bigger

Software is taking an ever larger role in our society. Bob Martin pointed out in his keynote speech that there is very little you can do in this life without interacting with computers and the software they run. You can’t buy anything, watch tv, do your washing, or probably arrange to meet your friends and go out for a drink without some form of interaction with information technology. Software is everywhere and skilled software developers are needed now in unprecedented numbers.

Not everyone has free time for this

I agree with Bob Martin about the problem: developers need to improve their skills. However, I just don’t think it’s realistic to expect people to spend their free time on this. At least, not in any sustained way and over a whole career. I actually think it’s pretty unhelpful to suggest this should be the main way skill acquisition happens. Several people at the conference agreed with me – not people in the panel, I should say – but regular conference attendees. At least half a dozen people approached me the following day and thanked me for saying this.

At times in my career I have had the luxury of enough free time to spend some of it on learning, reading, practicing and improving my software development skills. But at other times I’ve had family commitments, caring responsibilities, and other priorities that have completely prevented this. Expecting people to learn skills like TDD solely in their free time will cut off a large section of software developers from progressing and becoming more productive. We need those developers too!

Let’s look at better solutions.

One consequence of the current shortage of skilled software developers is our ability to vote with our feet. If your employer does not provide paid on-the-job time for learning, you can probably find another one that will. I am very happy to work for Praqma where our policy is that everyone should spend a good proportion of their time studying, going to conferences, reading, giving internal seminars and generally keeping themselves updated on what’s going on.

Giving developers time to learn makes good business sense in a competitive world where attracting and retaining skilled people is an advantage in the marketplace. You have the power to choose your employer. I don’t think Praqma is the only company that has realized providing time for learning is attractive to smart people and actually a win/win proposition. There may be a similar company close to you too. Have a look around!

What if I can’t change jobs?

Just as having free time for learning can be an unattainable luxury, there may be times in a career where changing jobs is not an option either. In this case you may need to talk persuasively with your manager and affect change from within. I really believe that a developer who improves their skills will work more effectively and be a bigger asset to their team than one who works only on production code and never does any practice. I think you should be able to persuade your boss that a small investment in training time now will reap dividends later.

I’ve also seen people spending a lot of time at work on code that never makes it to production. Have you seen that? Can you bring your boss examples of time you’ve wasted on proof-of-concept projects that never got to production, or long-lived feature branches that were impossible to merge and had to be thrown away? Perhaps you could spend some of that time practicing instead, and avoid some of those costly mistakes.

We need industry-wide skill acquisition

I think Bob Martin is right about the problem but wrong about the solution. When asked how an individual can improve their skills the answer can’t rely on them using their free time. It’s elitist, only those with that luxury of having free time will be able to do it. We need training and skills acquisition on a broad front that will work for the majority of developers. I think employers must put up some time for this as part of the working week. As individual developers we can argue for this, and vote with our feet if necessary. The learning of new skills shouldn’t be a luxury that’s only available to those with free time on their hands. Software, and its role in our world, is too important to rely on such a tenuous and poorly distributed resource.

I wrote a blog post about these tools on my company blog. In swedish.

Bob Martin has just written a post in his blog where he tells the story of a test manager who has 80 000 manual tests, and wishes they were automated instead. Bob writes:

“One common strategy to get your tests automated is to outsource the problem. You hire some team of test writers to transform your manual tests into automated tests using some automation tool. These folks execute the manual test plan while setting up the automation tool to record their actions. Then the tool can simply play the actions back for each new release of the system; and make sure the screens don’t change.”

Bob then goes on to explain why this is such a terrible idea – and blames it all on coupling. That the tests and the GUI are coupled to the extent that when you change the GUI, loads of tests break. Wheras humans can handle a fair amount of GUI changes and still correctly determine whether a manual test should pass or fail, machines fall over all too easily and just fail as soon as something unexpected happens. So you end up re-recording them, which can cost as much as just doing the tests manually in the first place.

These problems are of course bigger or smaller depending on the GUI automation tool you choose. Anything that records pixel positions will fall over when you simply change the screen resolution, let alone when you add new buttons and features in your GUI. More modern tools record the names or ids of the widgets, so they don’t break if the widget simply moves to another part of the screen. In other words, you reduce your coupling.

Geoff has been working on PyUseCase which takes this to another level. Instead of coupling the tests to widget names, you couple them to “domain actions”. This makes your tests even more robust in the face of gui changes. A drop down list can turn into a set of radio buttons and your tests won’t mind, since they just say something like “select airport SFO”. This doesn’t isolate you from the big changes, like moving the order of the screens in a wizard around, but since the tests are written in plain text, in a language any domain expert can read, they are relatively cheap to update.

There is another respect in which machines under-perform compared to manual testers. An intelligent human will usually do a certain amount of exploration beyond the scripted test steps they have infront of them. They try to understand the purpose of the test, click around a bit and ask questions when parts of the system peripheral to the test in hand start to look odd. Machines don’t do any exploration, and in fact often don’t even notice errors on parts of the screen they havn’t been told to look at.

Geoff’s PyUseCase can partly address this kind of a problem. Used together with TextTest, it will continually scan the log the System Under Test produces, and fail the test for example if any stack traces appear. PyUseCase also automatically produces a low fidelity ascii-art-esque log of how the current screen looks, and can compare it against what it looked like last time the test ran. Changes are flagged as test failures, which will bring to your attention the change in an unrelated corner of the screen which says “32nd December” instead of “1st January”.

I know that sounds like we just introduced a huge amount of coupling between the tests and the way the GUI looks, and yes, we have. The difference is that this coupling is very easy to manage. If 1000 tests all fail saying “expected: 1st January, found: January 1st”, TextTest handily groups all the test failures and lets you accept or reject the change en-masse. So it is very little work to update a lot of tests when the GUI just looks different, but you don’t care.

There is still a problem though, that the machine will not explore outside of the scripted steps you tell it to perform. So you will have to do some manual exploratory testing too, not everything can be automated.

So a simplistic lets-just-automate-our-manual-tests is a bad idea because machines can’t handle GUI changes as well as humans can, and because machines don’t look around and explore. Potentially your automated tests will cost more than your manual tests, and find fewer bugs.

So should we stick with our manual test suite then? No, of course not. The value of automated tests is not simply that you can run them more cheaply than manual tests, it is that you can run them more often – at every build, constantly supplying developers with valuable feedback rather than just at the end of the release cycle. It is this kind of feedback that enables refactoring, and lets developers build quality code from the start. That is their real gain over manual tests.

Bob Martin’s suggestion is that you shouldn’t rely on expensive GUI tests for this kind of feedback – only perhaps 15% of your tests should be GUI reliant. The rest run against some kind of api, which is less volatile and hence cheaper to maintain. With the kinds of tools Bob I suspect has been using for GUI testing I’m not surprised he says this. I just think that with tools like PyUseCase and TextTest the costs are much reduced, and call for reconsideration of this ratio. Looking at Geoff’s self tests for TextTest (a GUI intensive tool), around half are testing through the GUI, using pyUseCase. Basically I don’t think GUI tests have to be as bad and expensive as Bob makes out.