In Progress
Unit 1, Lesson 1
In Progress

Cucumber Expressions with Matt Wynne

You might have heard it said that sharing executable specification documents between developers and business domain experts is a fool’s quest. The truth is, this kind of sharing is used to great effect in many teams. But pulling it off successfully requires careful attention to how those specs are written.In this episode, guest chef Matt Wynne of the Cucumber project will show you how to write declarative, maintainable executable specifications with the help of a new Cucumber feature.

Video transcript & code

The ability to share executable feature specifications between technical and non-technical members of a project can be a powerful accelerator for collaboration. I've seen this technique used to great effect; but only when teams write their executable specifications in a meaningful way.

How can you use executable specifications productively? Today, Matt Wynne is here to help answer that question, within the context of the Cucumber acceptance-testing tool. Matt got involved in the Cucumber project back in 2008, wrote a book about it in 2011 and co-founded Cucumber Limited in 2013. He lives on the west coast of Scotland on an old farm with his wife Anna, two kids, a dog, a couple of cats and some chickens.

Here's Matt, with some guidance for writing meaningful Cucumber specifications, with the help of a newer Cucumber feature you might not yet be aware of. Enjoy!


Introduction

Cucumber is a tool for running business-readable automated acceptance tests. So they're automated tests that business people can read and tell you whether or not the code you're building will be acceptable to them. When you get this collaboration right, they reflect your whole team's shared understanding of the problems you're solving together.

A single source of truth.

The interesting thing about making software this way is that when the whole team try to express their different perspectives on the problem in a single place, we start to see the assumptions lurking beneath the surface and gain new insights from each other about exactly what we need to build.

Imperative vs Declarative styles

Probably the most outfacing thing about starting to use Cucumber is deciding how to express yourselves.

For example, let's suppose we're building a TODO list app. We want to describe a new feature that lets us list all the tasks tagged for a particular context, like, say, being offline on an aeroplane:

 Feature: Listing tasks

  Scenario: Filter tasks by tag
    Given a task "write script for RubyTapas" tagged with "write,offline"
    And a task "book hire car" tagged with "online,admin"
    When I list the tasks tagged with "offline"
    Then I should see 1 task
    And the visible tasks should be:
      | write script for RubyTapas |

Now there's a little bit of structure here:

The Feature, Scenario, Given, And, When and Then keywords are known as the Gherkin syntax that enables Cucumber to parse this example as a test, whilst still keeping it human-readable. When we show this kind of document to the product owner or manual tester for the project, they'll be able to read it and give us their feedback.

Now consider another way of expressing the exact same feature, but written much more from a testing perspective:


Feature: Listing tasks

  Scenario: Filter tasks by tag
    When I click "Create task"
    And I fill in "Name" with "write script for RubyTapas"
    And I fill in "Tags" with "write,offline"
    And I click "Submit"
    And I click "Create task"
    And I fill in "Name" with "book hire car"
    And I fill in "Tags" with "online,admin"
    And I click "Submit"
    And I follow "list tasks"
    And I click "offline"
    Then I should see only "write script for RubyTapas" within ".tasks"

This one has a lot more detail. reads a few lines, sounding bored.

If you manage to pay attention, you can see that this same scenario is describing exactly the same system behaviour, but at a much more nuts-and-bolts kind of level.

It's very much focussed on the how of what's happening in the test, and not so much on what the user wants to achieve. It's much less expressive.

These two contrasting levels of detail have come to be referred to within the Cucumber community as the "imperative" and "declarative" styles. The shorter example, which reads more like a specification, and elides a lot of the details, is knows as the declarative style. The more detailed example, which specifies the details of every last click on the user interface but misses the bigger picture, is known as the imperative style.

Most people's Cucumber scenarios sit on a spectrum somewhere between these two extremes. There are quite a few trade-offs in choosing the most appropriate level of detail which sadly we don't have time for in this episode. What we'll focus on here is the trade-off around maintainability.

The maintainability trade-off

To frame this discussion about maintainability, let's think about the two main reasons why your Cucumber features and their underlying automation code would need to change.

On the one hand, you might get a new requirement. You'll need to add a new scenario or two, or go back and modify existing ones to reflect the new way you want the system to behave. You might also be catching up on your test coverage, and be writing new scenarios to test existing behaviour.

Alternatively, you might make a change to the way existing behaviour is delivered. The underlying business rules haven't changed at all, but you might change the labels on the form fields or buttons, or change the user experience workflow.

The imperative style of scenario, with lots of UI interaction details, has the disadvantage of being brittle to this last type of change. The UI details are duplicated all over your Gherkin scenarios, so you can make a perfectly valid change to login form, for example, everything still works with a manual test, yet suddenly all of your Cucumber tests are screaming at you.

This can be really annoying. It means you have a big search-and-replace job on your hands before you get your tests passing again. It's one of the mean reasons we see teams abandon Cucumber.

On the other hand, using a more declarative style forces you to push the how down into the code behind the Gherkin that automates your app. This allows you to use good software engineering practices like method re-use to eliminate the duplication.

So when the UI does change, you only have to make a change in one place in your tests.

From this perspective, it looks like a declarative style is the obvious choice. So why doesn't everyone write their scenarios this way?

It turns out there's a catch. Let's look at what goes on under the covers when Cucumber tests your scenarios.

 

Step Definitions

As it runs through each scenario to test it, Cucumber searches for step definitions that match the phrases or steps in your scenario. Each step definition is just a Ruby Proc tied to a search pattern that Cucumber uses to match it to steps in your scenarios. The job of the Proc is to do something to your app that matches the intention of what was expressed in the Gherkin step.

If Cucumber finds a step definition with a matching pattern, and the Proc executes without raising an exception, then Cucumber marks that step as passed, and moves on to the next one. If all the steps in the scenario pass, the scenario passes.

Hooray!

So the step definitions are the glue that binds your plain-language scenarios to your app. They do the actual work of poking and querying it to test that the behaviour in each scenario has been correctly implemented.

Until recently, the way to express the pattern for a step definition was using regular expressions. For example, the step Given a task "write script for RubyTapas" tagged with "write,offline" might have had a step definition like this:


Given(/^a task "([^"]*)" tagged with "([^"]*)"$/) do |name, tags|
  create_task name: name, tags: tags.split(',')
end

This step definition simply calls out to a create_task helper method that's implemented elsewhere. We don't know exactly how it's doing it - it might be clicking buttons on the UI, or it might be reaching behind the screen and going direct to our database to insert the data. That's not what we're here to talk about though.

What we're here to talk about is that regular expression.

This particular regex captures two arguments from the text in the Gherkin step:

The name of the task and a comma-delimited string of tags. In order to do that, we've had to use a feature of regular expressions called a capture group with some gnarly open-parens, open-square-braket, caret, quote-mark, close-square-bracket, asterisk, close-parents type of nonsense going on.

What's wrong with using Regular Expressions?

At Cucumber HQ we had feedback from our users that lots and lots of people found those regular expressions intimidating. They certainly look ugly!

And this is where we get into that maintainability trade-off, because if you find regular expressions intimidating, then you're going to be feeling unconfident about crafting new patterns like this each time your team want to use a phrase in your scenarios.

So what are you going to be inclined to do when you write those specifications? You're naturally going to try and shoe-horn your scenarios into using existing step definitions whenever possible, just so you don't have to touch those regular expressions.

Teams like this want a small number of re-usable step definitions that they can compose into lots of different scenarios. Those step defintions will tend to be low-level, like the ones we saw in our imperative example.

This is a kind of a trap. It looks like the right thing to do because you're keeping the amount of step definition code to a minimum, but then when you need to change the UI, you end up with pain.

Now remeber Cucumber's goal is to engage effective collaboration between techincal and non-technical members of a team. Guess what's one of the best ways of putting off the non-technical members of your team from collaborating on your Gherkin specifications? Re-phrasing everything they want to say into something that reads like a boring, repetitive computer program.

So now we have a tension. We want to be free to express ourselves in our Gherkin scenarios, but at the same time we don't want a maintenance burden with crazy regular expressions down in our step definitions. This is where cucumber expressions, a new feature in Cucumber, are here to help.

 

Introducing Cucumber Expressions

Instead of that gnarly regular expression, we can simply pass Cucumber a string.


Given("a task {string} tagged with {string}") do |name, tags|
  create_task name: name, tags: tags.split(',')
end

We can tell Cucumber that we want to capture a string at this point, and another at this point.

Cucumber will automatically convert to a set of primitive types, so for example we can match the assertion step, which contains the expected number of tasks, like this.


Then("I should see {int} task") do |expected_num|
  assert_equal expected_num, visible_tasks.length
end

Again, we're calling an imaginary visible_tasks method here that will return the list of tasks the user would be able to see. The expected_num argument is automatically turned into an integer for us.

 

Flexibility

We'll probably want to be able to use that step when there's more than one task visible. To do that, we put the optional "s" in parentheses, like this:


Then("I should see {int} task(s)") do |expected_num|
  assert_equal expected_num, visible_tasks.length
end

In Cucumber expressions, parenthesis don't mean capture groups, they mean what people usually mean with parenthesis when they write prose: optional text.

Although it’s generally a good idea to encourage people to be precise with their use of domain language, there are some words that don’t matter all that much, and it’s nice to make your step definitions flexible for some of the different ways you might want to say the same thing. For example, there's not really any difference between "with" or "as" in this expression, so let's allow people to use either:


Given("a task {string} tagged with/as {string}") do |name, tags|
  create_task name: name, tags: tags.split(',')
end

Adding the slash tells Cucumber expression that either of those words would be fine to match on.

 

Custom Parameter Types

Looking at this step definition, it would be nice if the tags argument was already an array of strings - or even tag objects - so we don't have to convert it ourselves; similar to how the expected_num was automatically turned into an integer for us.

Luckily Cucumber lets you define your own parameter type for this sort of thing:

From our step definition or support code, we call the ParameterType DSL method, passing it the name of the parameter, a regular expression pattern - in this case we'll just match any combination of lower-case letters or commas, and a transformer block that will take the string matched by the regular expression and split it by comma into an Array.


ParameterType(
  name: 'tags',
  regexp: /[a-z,]/,
  transformer: -> (match) { match.split(',') }
)

Now we can change the expression to use our new custom parameter type.

From now on, tags will be passed into our step definition as an Array of String, so we can get rid of that call to split.


Given("a task {string} tagged with/as {tags}") do |task, tags|
  create_task name: task, tags: tags
end

Now suppose you're writing a new Gherkin step that doesn't have a step definition yet.


  Scenario: Delete some tags
    Given a task "write script for RubyTapas" tagged with write,offline
    When the tags write,admin are deleted
    And I list the tasks tagged with offline
    Then I should see 1 task
    And the visible tasks should be:
      | write script for RubyTapas |

When we run Cucumber, Cucumber will recognize that write,offline, and write,admin matches your tags parameter type and suggest a step definition snippet that you can paste right in:


When("the tags {tags} are deleted") do |tags|
  # TODO
end

This makes it a lot easier for you and your team to reuse parameter types. This also helps you grow your domain language - each "thing" in your domain might get its own parameter type.

Another nice thing about Cucumber Expressions is that you no longer have to put all of your arguments between double quotes anymore, which makes your Gherkin documents easier to read.

So that’s about it. We’ll be continuing to support regular expressions in Cucumber for the foreseeable future, so you don’t need to worry about upgrading your existing test suite in a hurry. Still, hopefully this episode of RubyTapas has given you a taste for why you might want to.

And if you weren’t even using Cucumber yet, hopefully I’ve whetted your appetite to give it a try.

Responses