Coding Is Like Cooking

How your team actually writes its code is very important if your organization is trying to build products in an Agile way. This post considers the risks associated with insufficiently Agile coding practices and explains how to identify them before they become major issues. 

Nearly every software organization has high ambitions for increasing their agility and yours is probably one of them. It’s common to bring in experienced coaches to help teams operate with more agility and to discover ways of delivering more value. But, if you’re working on software products, you’ll find it’s hard to realize some of the benefits of Agile without changing the behaviour of developers in the codebase. Too many coaching efforts place little or no emphasis there. 

Agile Coaching helps your business to become more successful by improving the way you plan and deliver software. Technical Agile Coaching specifically focuses on how people write code. How do you know if you need Technical Agile Coaching? Here are three signs.

You spend more time fixing bugs than building new features

Look at all the tasks you work on each week. Ask yourself how much time is being spent on innovative new features, and how much on fixing defects in existing features? For some teams the answer can be as little as 5% of time spent on proactive new work. Now ask yourself – how much more money would our organization make if we spent less time on fixes and more on delivering new features? 

Turning this situation around is about increasing the quality of your work so that fewer fixes are needed, resulting in more time for innovation. Often, the root cause of defects is technical debt and code that is hard to understand and change. 

You can mitigate against this by training your developers to build in quality from the start. Technical Agile Coaching helps teams to improve automated tests and reduce technical debt by demonstrating various techniques and practices for evolving design safely.

Delivery of new features is delayed, sometimes more than once

Are your new features around 80% done for long periods and then take forever to be actually ready for release? Often this is the biggest complaint from customers and other stakeholders. How much would it be worth to your organization for new functionality to be available reliably and on time? 

Late deliveries are often caused by developers not collaborating effectively in the code. To build a new feature they need to bring the whole team’s expertise to bear, and integrate their work seamlessly. Typically, the coding work is not divided up in a way that makes collaboration easy, and changes are not integrated until late in the process.

Technical Agile Coaching helps developers to deliver on time: reliable automated regression tests promote better collaboration with a shared view of the current status; teams learn to divide up the coding work and integrate changes far more frequently than the interval between external deliveries; developers learn to use their tools better and to communicate effectively within the team so they can complete tasks more reliably.

Developers complain the code is so bad they need to re-write the whole thing

Sometimes developers ask for a hiatus in new feature development for several months, even a year, so they can start over by rebuilding the same functionality using new tools and frameworks. This is a significant cost to your business, and a large risk. A more staged approach carries far less risk and allows you to build better functionality with much more manageable cost. 

Usually, an organization arrives in this situation after many years of accumulated technical debt. The technologies chosen originally become outdated and are not upgraded in a timely way. Developers make design decisions and then leave without fully documenting or otherwise transferring design knowledge to new team members. New features are added that were not anticipated in the original design. Gradually the code quality declines and it becomes harder and harder to work with.

Technical Agile Coaching helps developers gain better control over existing code, so they can continue working with it and building new features. Teams learn to migrate code to newer technologies in a series of smaller, less risky steps, adding new functionality as they go. Developers learn to pay off technical debt and to communicate better about design decisions.

Benefits of Technical Agile Coaching

Technical Agile Coaching helps developers to change the way they work, day to day, minute by minute, in their codebase. They learn to increase the quality of the code in their product and work more effectively as a team. The coach works closely with the developers while they are writing code to remind, reinforce, teach, and demonstrate Agile practices. With Technical Agile Coaching you should expect to see the following benefits:

• Reductions in technical debt
• Better automated regression tests
• Improved communication between developers within the team
• Smaller, safer changes integrated often and documented well
• Improved effectiveness using team development tools and IDEs.

Technical Agile Coaching helps a team to improve how they write code, and begin to use more effective agile development practices together.

If you’re interested in getting some Technical Agile Coaching, please contact us at ProAgile. We have many coaches with a lot of experience in agile development and several with technical coaching expertise.

Note: this article was originally published on ProAgile’s blog

I only recently joined ProAgile and I feel we’ve got off to a great start. Last week my colleague Fredrik Wendt and I facilitated and hosted a Code Retreat event for a diverse group of software crafters from local companies.

It’s the 10th anniversary of the “Global Day of Code Retreat” event and there were over 150 locations around the globe holding one last week. It’s not the first time I’ve facilitated a Code Retreat but it is the first time I’ve done one in Gothenburg, (the others have been in Stockholm). This year Fredrik and I were keen to bring together some local people, and contribute to the community here. ProAgile has recently invested in a fantastic venue for courses and training at our offices, and we want to take full advantage of it.

We contacted several local companies and specifically invited them to advertise on the event sign up page that they were going to send people. We were delighted when Boeing (Jeppesen), Pagero, Meltwater, Spotify and Greenbyte all agreed to this. Their intention to participate I’m sure helped get the attention of other local software crafters when we put out a general invitation with open sign-ups.

In the end we were a very diverse group drawn from many local companies both large and small. We had a game designer who favoured C++, a Scala enthusiast, a Kotlin proponent, a Python trainer, several Javascript developers, a self-taught PHP and Python developer, a CEO who still codes occasionally, and many local Java, Python and C# developers. Everyone got to pair program with people from other language communities and backgrounds.

Fredrik and I decided to hold a standard code retreat event and based our planning on this article: “Structure of a Code Retreat”. We held 5 sessions working on the “Game of Life” Code Kata through the day. In each session, people work in pairs starting from an empty editor, and come up with a new solution to the problem. Between each session you hold a short retrospective with your pairing partner before taking a short break and switching to a different pairing partner.

In the morning sessions the focus is on getting to know the problem. We all try to find a good way to use Test-Driven Development and simple design rules to get a clean, understandable, flexible design. The sessions after lunch are about using this Game of Life exercise as a vehicle for trying out something new or otherwise challenging. We had some pairs using unfamiliar programming languages and environments. Some pairs were following strict design rules like ‘no if statements, no loops’ and ‘no function longer than 1 line’. Some people accepted the challenge to work on the problem as a mob; we were 4-6 people using one computer with me facilitating.

Throughout the day there were opportunities to reflect on the way we write code: how design guidelines and programming languages and Test-Driven Development and personal preferences all interact to influence the code we end up with. In normal project work with deadlines and tricky business rules and security constraints and so on, you might not get much time to consider the fundamentals of what good software design looks like. Test-Driven Development is a complex skill to learn – it has many facets. I think all programmers benefit from taking some time occasionally to work on practice exercises and discuss software design principles with peers.

At the end of the day we reflected all together in a closing circle. Everyone had the opportunity to say a few words to the group about what they were taking away with them. When you’ve spent time working on a Code Kata, I think there is a complexity gap you need to bridge – you want to carry over the principles and skills you gained from that simple exercise to your actual production code situation. It’s useful to think about what you’ve learnt and explain how you want to apply it later.

In the closing circle, I was happy to hear several positive comments about Test-Driven Development and an intention to use it more. Some people had seen a new programming language and had ideas about when it could be useful to them. Several people mentioned they appreciated the chance to program with people outside of their company and comfort zone. It was a good day, and I don’t think it will be the last such event we hold in Gothenburg.

If you’d like to know about the next events hosted by ProAgile, take a look at our course calendar, keep an eye on ProAgile’s social media channels and perhaps join one of our networking groups.

Note: this article was first published on ProAgile’s blog.

I’ve just joined ProAgile, and it feels great. I’ve known many of the other coaches for some years, and admire their work. My background is as a software developer, and I specialize in the technical side of Agile. Practices like Test-Driven Development, Refactoring, flexible and extensible design, Continuous Integration and so on. If you’re developing software, then you’re going to need to include these kinds of things on your agile journey.

I see great opportunities for collaborating with my new colleagues. We can coach organizations and teams with agile ambitions and cover more aspects if we work together. I work primarily with software developers and help them to write the code in a more agile way. Many of my colleagues are focused on organizational change, agile leadership, team development and other aspects of the agile journey. I think coaches who collaborate will be a win-win situation for everyone.

As well as coaching developers, I am also planning to offer open training courses in agile coding techniques. I wrote a book a few years ago ”The Coding Dojo Handbook” and I love designing small code kata exercises. Many are available on my github page. I’ve worked for many years with an automated testing approach called ”Approval Testing” which I’ve written about on my personal blog. I’ve also worked with testers on strategies when doing Continuous Delivery, and recently developed a card game. I hope to add some new courses to ProAgile’s selection shortly, I have a lot of teaching materials and experience to draw on.

Note: this article was first published on Praqma’s blog

Writing tests for ‘Theatrical Players’

When I read Fowler’s new ‘Refactoring’ book I felt sure the example from the first chapter would make a good Code Kata. However, he didn’t include the code for the test cases. I can fix that!

Martin Fowler recently published a new edition of his classic book ‘Refactoring’. The first chapter is a worked example of how he would go about refactoring a small piece of code that he needs to update with some new functionality.

This is a really common scenario faced by software developers daily. It’s impossible to anticipate every future change and enhancement, so the design of the code will often not be extensible in the direction you need to take it. Refactoring the design in order to make a proposed update easier is an essential skill.

I like using small exercises to practice these kinds of skills, called ‘Code Katas’. I’ve got a whole collection of them in my book ‘The Coding Dojo Handbook’, and more on my github page. When I saw Fowler’s example I felt it would make a really good kata. The particular refactoring techniques he demonstrates are techniques that many developers would benefit from learning and practicing.

I asked Fowler for permission to use his code, which he kindly granted, then I set about transforming this example into a kata.

Creating a new Code Kata

This is Fowler’s example javascript code, which produces a statement for a customer. The new functionality that’s needed is to produce the same information but formatted as HTML.

statement.js

function statement (invoice, plays) {
    let totalAmount = 0;
    let volumeCredits = 0;
    let result = `Statement for ${invoice.customer}\n`;
    const format = new Intl.NumberFormat("en-US",
        { style: "currency", currency: "USD",
            minimumFractionDigits: 2 }).format;

    for (let perf of invoice.performances) {
        const play = plays[perf.playID];
        let thisAmount = 0;
        switch (play.type) {
            case "tragedy":
                thisAmount = 40000;
                if (perf.audience > 30) {
                    thisAmount += 1000 * (perf.audience - 30);
                }
                break;
            case "comedy":
                thisAmount = 30000;
                if (perf.audience > 20) {
                    thisAmount += 10000 + 500 * (perf.audience - 20);
                }
                thisAmount += 300 * perf.audience;
                break;
            default:
                throw new Error(`unknown type: ${play.type}`);
        }
        // add volume credits
        volumeCredits += Math.max(perf.audience - 30, 0);
        // add extra credit for every ten comedy attendees
        if ("comedy" === play.type) volumeCredits += Math.floor(perf.audience / 5);
        // print line for this order
        result += ` ${play.name}: ${format(thisAmount/100)} (${perf.audience} seats)\n`;
        totalAmount += thisAmount;
    }
    result += `Amount owed is ${format(totalAmount/100)}\n`;
    result += `You earned ${volumeCredits} credits\n`;
    return result;
}

As you can see, the logic for formatting the statement is all mixed up with the calculation logic, so as it stands it’s not straightforward to add the HTML formatting feature. What’s needed is to change the design and detangle the logic, via refactoring.

Fowler mentions that the first step in refactoring is always the same – to ensure you have a solid set of tests for that section of code.

However, he did not include the test code he used in the book. (It’s a book primarily about refactoring, and I guess including test code would have taken attention away from that topic). To make this example into a Kata, I wanted to include some tests.

The testing approach I favour in this kind of situation is approval testing. The code exists already and it works, so the easiest way to add a regression test is to find some test data, exercise the code, and approve the result.

This is different from a test-driven development approach where you define the test case before the code exists. (You can use approval testing in that kind of scenario too, but that’s a topic for another article). I usually find it much quicker and easier to add an approval test to existing code than a classic assertion-based test case.

Creating a first approval test

Fowler supplies us with some test data – a sample invoice:

invoice.json
{
  "customer": "BigCo",
  "performances": [
    {
      "playID": "hamlet",
      "audience": 55
    },
    {
      "playID": "as-like",
      "audience": 35
    },
    {
      "playID": "othello",
      "audience": 40
    }
  ]
}

And a list of plays:

plays.json
{
  "hamlet": {"name": "Hamlet", "type": "tragedy"},
  "as-like": {"name": "As You Like It", "type": "comedy"},
  "othello": {"name": "Othello", "type": "tragedy"}
}

The function that we want to test, statement, returns the information to send to the customer, formatted as a plain text string. This statement string is perfect to use as the approved value in the approval test.

This is the test case I designed, it uses the Jest testing framework:

statement.test.js
test('example statement', () => {
   const invoice = JSON.parse(fs.readFileSync('test/invoice.json', 'utf8'));
   const plays = JSON.parse(fs.readFileSync('test/plays.json', 'utf8'));
   const statement_string = statement(invoice, plays);
   expect(statement_string).toMatchSnapshot();
});

First, I read both data files and parse them into javascript objects. Then I call the function we want to test – “statement” and store the result in the ‘statement_string’ constant. The last line of the test checks that this result matches the approved value. What it does is compare the actual value against a stored value which I checked and approved earlier.

This is kept in another file:

__snapshots__/statement.test.js.snap
exports[`example statement 1`] = `
"Statement for BigCo
Hamlet: $650.00 (55 seats)
As You Like It: $580.00 (35 seats)
Othello: $500.00 (40 seats)
Amount owed is $1,730.00
You earned 47 credits
"
`;

Is it called Snapshot or Approval testing?

The Jest testing framework calls the approved value a ‘snapshot’. I prefer to talk about ‘approval’ testing and comparing with an ‘approved’ value. I think these words get to the heart of what you’re doing – deciding what is good enough to test against.

‘Snapshot’ as a word just implies it will be updated again soon. I like to emphasize the human agency involved in inspecting and deciding whether to approve a new value or not.

Is this test good enough to support refactoring?

This test goes a long way to giving us the confidence we need to start refactoring the ‘statement’ code. It generates a full statement with test data that seems realistic enough for this context.

I think it’s a good idea to do a little more analysis of whether this test is sufficient to support the refactoring we want to do. One way to do that is to run the test and measure code coverage. Lines of code that are not covered could be refactored away, or have bugs inserted into them, without the tests failing.

The coverage data shows only one line of code which is not covered by this test:

Line 28 in statement.js:
throw new Error(`unknown type: ${play.type}`);

Our test case is pretty good, to only have one line uncovered. The test data supplied by Fowler includes different kinds of play that already exercise several paths through the code.

This code path is different, however. If you end up on this line then the function aborts, and doesn’t produce a statement. We can’t tweak the data in our existing code to make it also execute this line and still produce a statement.

What we need is an additional test, one that uses an unsupported play type. I invented some new data that continues the Theatrical theme:

new_plays.json
{
 "henry-v": {"name": "Henry V", "type": "history"},
 "as-like": {"name": "As You Like It", "type": "pastoral"}
}

invoice_new_plays.json
{
  "customer": "BigCoII",
  "performances": [
    {
      "playID": "henry-v",
      "audience": 53
    },
    {
      "playID": "as-like",
      "audience": 55
    }
  ]
}

This is the test case that uses them:

statement.test.js
test('statement with new play types', () => {
    const invoice = JSON.parse(fs.readFileSync('test/invoice_new_plays.json', 'utf8'));
    const plays = JSON.parse(fs.readFileSync('test/new_plays.json', 'utf8'));
    expect(() => {statement(invoice, plays)}).toThrow(/unknown type/);
});

This test is not an approval test, it’s a normal assertion-based test. It checks that the error is thrown and the message contains the string ‘unknown type’.

I could have instead used an approval test to check the error message. In this case I chose not to do that. The string to be checked is quite short, and there is little advantage in storing it in a separate file away from the test code.

Confidence in the tests

With that test added, we’re more confident we can use this test suite to support refactoring.

We have 100% coverage now, after all. Unfortunately, that is no guarantee that our tests are perfect, or even reasonably good. One thing I like to do is a spot of (manual) mutation testing. to assess how good my tests are.

I edit the code to insert a bug, then run my tests. If they fail, then that’s a good sign. I might try adding bugs in several different places to increase my confidence.

When I try it on this code, I find any change that affects the way the statement is presented gives me a failing test. I can also easily get a nice diff showing exactly what I broke.

I’m now feeling pretty confident these tests will be good enough to support refactoring.

This is a good starting point for the kata. From here you can practice all the refactorings demonstrated by Fowler in his book, with a safety net to alert you if you make a mistake.

When you come to implement the HTML rendering feature, it should be straightforward to add a new test for it too.

Adding more languages to your tests

In the book, the example language is Javascript, which is widely-used and understood. I often work with teams who also use languages like Java, C# and Python, and the refactoring techniques are just as relevant in those languages. Once I had this JavaScript version working, I added a translation or two. One of the reasons I keep my exercises publicly available on GitHub is so that people can send me pull requests.

If you’d like to do this exercise in a programming language that’s not supported yet, you can fix that for everyone 🙂 I particularly like getting pull requests with translations of my exercises.

The published code kata

The starting point for this new exercise ‘Theatrical-Players-Refactoring-Kata’ is now published and available for download.

The first chapter of ‘Refactoring’ containing a worked solution to this problem is available as a free sample. You have every opportunity to practice and learn these testing and refactoring techniques now.

Happy coding!

A new card game to design Continuous Delivery pipelines

Note: This article was originally published on Praqma’s blog

What testing steps should you include in your Continuous Delivery pipeline? Don’t just string together existing manual processes – use simple, collaborative tools to design something better!

Creating a Continuous Delivery (CD) pipeline is a key development in an organization’s transformation to DevOps. A CD Pipeline covers all the activities needed to transform a code change made by a developer into updated software bringing value to users. Steps in the pipeline include building a new version of the software as well as testing and deploying it. Exactly what kinds of build, test and deployment steps will depend on many factors and there is no ‘one perfect pipeline’ which will suit all situations.

I created the card game ‘Pipeline’ as a quick and fun way to explore alternatives for a CD pipeline without actually building anything. The goal of the game is to design a pipeline for a given scenario and optimize the deployment lead time. You work in a small group and get to discuss what steps are needed and which order you want to do them in. You will run into design tradeoff decisions and may discover people have different risk tolerances. If you play the game a second time with a different scenario, or compare notes with another group, you can learn more about how different scenarios drive different decisions.

Playing the game should help you to build a real pipeline for the real software system you are working on. Building a CD pipeline needs specialist knowledge of particular tools and could be many weeks of work. Playing this game should help you to avoid some costly misunderstandings. It’s also a fun way to engage a small group for an hour or two while you think through the issues you’re facing.

Scenario cards

To play the game each group begins with a sketch of a guiding scenario. For example, imagine you work for a startup aiming to displace an established business, like a social media website or an online bank. The reason for the scenario is that it makes the discussions more concrete and productive. You can discuss some specific risks you need to mitigate through testing. With that in mind it’s easier to reason about how long to spend on each testing step, and also assess the potential impact of excluding a step entirely.

Knowing the deployment lead time of the competitor in the scenario helps guide the group to discussions about tradeoffs and where quality comes from. Can we leave out this slow testing step? Can we mitigate that risk another way?

Pipeline step cards

The ‘pipeline step’ cards form the majority of the cards in the deck. These are the ones you lay out on a table to define the structure of your pipeline. You can choose which steps are important and which can be optional or omitted entirely. You can put several steps in parallel to reduce overall lead time. You can choose to perform a step twice, both before and after building the deployable component for example, or to do some steps before the beginning of the pipeline. The deck includes a few blank cards so that you can invent completely new steps. Just put a new sticky note on one with a text explaining what the new step involves.

Game rule cards

You have a lot of freedom about how to design your pipeline, but in reality there are a few rules about how you must order the steps. For example, you can’t deploy to production before you’ve built a release candidate. The “Game Rule” cards help you to keep track of these rules. One of the cards reminds you to add sticky notes to each step in the pipeline with an expected execution time. You should estimate this – either how long you think it will take, or a timebox for how long it is allowed to take. If it’s a manual step this estimate should include the time spent waiting for the person to be available.

Review

It might take a group of 4-6 people an hour or so to design a pipeline for their scenario and estimate the overall deployment lead time. When this is done (or the time for this part of the exercise is used up!) you should bring out the ‘review’ card. Each group can discuss what the implications of having this pipeline would be for their startup business. How does your lead time compare with your competitor’s? You might discuss how deployment lead time relates to time to market, or whether you would be able to do Lean Product Development, or A/B testing.

Some of the fictitious competitors have rather short deployment lead times. That’s quite deliberate on my part. I’d like people to think about how the competitor is achieving that. What must their pipeline look like? What testing must they be doing differently? You may think they are taking too many risks, and they will be easy to out-compete. Or perhaps they have found a better way, and you’d better imitate them.

Some of the fictitious competitors have much longer deployment lead times. I’d like people to think about whether that is really necessary. Perhaps they have included heavyweight change processes that take time without providing any benefits.

In any case, it’s interesting to think about, and might help you make more informed choices when you come to design a pipeline for your production system. Playing this game could alert you to some of the real life pitfalls that often arise.

Notes for a facilitator

If you want to play this game with your team I hope it will be straightforward for you to organize. The instructions are on the box. Allocate something between 1-2 hours for the exercise, split people into groups of about 3-6 people, with one deck of cards per group. You will also need a block of small sticky notes and pens for each group.

Your job as facilitator is to make sure everyone understands the point of the exercise, follows the pipeline game rules, and correctly calculates their deployment lead time. You should keep time and answer questions as they come up.

One thing a group might do is to decide on a microservices architecture, or front and back ends that are deployed separately. They might want to design a different pipeline for each service or each deployable component. That’s harder to do with only one deck of cards. For the purposes of this exercise, ask them to pick only one pipeline to design to start with. If they complete it quickly, you could give them another deck of cards to design another one later on.

As facilitator you may want to be quite active in the ‘review’ part of the game. Ask leading questions about the pipeline design to get people to explain their reasoning. If there are several groups you’ll want to have them look at one another’s work. This will help people to understand that different scenarios might call for different pipeline design choices. Everyone should come away with a better understanding of how to design a CD Pipeline and the hard choices that sometimes have to be made. Remind them that what they are learning in this game could help you all to avoid costly mistakes when designing a CD pipeline for real.

Throughout, try to keep these goals in mind:

• People should have fun
• Each group should come up with a feasible Continuous Delivery pipeline for their scenario, and they should be able to explain why they designed it that way
• People understand what deployment lead time is, and that it could be significant when it comes to business competitiveness

Acknowledgements and Thanks

The first idea for this game came from a fun exercise at a workshop at the European Testing Conference in February 2018, led by Lisa Crispin and Abby Bangser. They divided us into groups, then handed each a pile of cards showing names of steps you could include in a pipeline. For example, “automated test: unit test”, “manual test: exploratory test”, or “deploy step: deploy to performance testing environment”. We had to collaborate to lay out the cards on a table in the form of a pipeline, and estimate the overall lead time.

I really enjoyed the exercise. There is something about having cards to move about and wave at people that encourages discussion and group engagement. Some cards provoked lively controversy – I remember discussing whether to include a manual code review step before merging to master, with someone who was quite convinced it was essential.

Iterative improvements to the exercise

I liked the exercise so much I decided to adapt it for a workshop I lead about Testing in Continuous Delivery. As I did this exercise with different groups I tweaked it a little each time. I also did it once with Llewellyn Falco and incorporated some of his suggestions too.

As the game gradually grew more sophisticated; I realized it would be worth getting some custom cards printed, so I started working with Phable on a layout and design. My colleagues at Praqma did some play-testing of an early version, and contributed some useful ideas. I’d like to thank everyone who played earlier versions of this game with me along the way. It’s a better game because of your feedback!

The second version of the cards came about six months after the first print run, following many successful CD workshops. Some of the ‘scenario’ cards have a lead time measured in weeks, and I wanted to make it possible to model those pipelines too. I added a few cards with more traditional release activities like a ‘Go/No Go meeting’ and ‘Decide Release Version number’. I also took on board some feedback from Steve Smith that ‘Constraint’ cards were poorly named – it reminds too much of ‘Theory of Constraints’, which has nothing to do with this game. I now call them ‘Game Rule’ cards, which I hope you’ll agree is more accurate.

I’d like to thank everyone who played earlier versions of this game with me along the way. It’s a better game because of your feedback!

Get your own card deck

These card decks are available for sale from Praqma. Head over to the original version of this article – there is a “Buy Now” link at the end of it: https://www.praqma.com/stories/pipeline-card-game/