1. Discovering BDD
The script for this chapter is currently in Google Docs
2. Your first Scenario
The video and audio assets for this chapter are here.
2.1. An introduction to Shouty
Shouty is a social network that allows people who are physically close to communicate, just like people have always communicated with their voices. In the real world you can talk to someone in the same room, or across the street. Or even 100 m away from you in a park - if you shout.
That’s Shouty. What you say on this social network can only be “heard” by people who are nearby.
2.2. Choose the first scenario
Let’s start with a very basic example of Shouty’s behaviour. Something we might have discussed in a three amigos meeting:
Sean the shouter shouts "free bagels at Sean’s" and Lucy the listener who happens to be stood across the street from his store, 15 metres away, hears him. She walks into Sean’s Coffee and takes advantage of the offer.
🎬 1 We can translate this into a Gherkin scenario so that Cucumber can run it. Here’s how that would look.
Scenario: Listener is within range
Given Lucy is located 15m from Sean
When Sean shouts "free bagels at Sean's"
Then Lucy hears Sean’s message
You can see there are four special keywords being used here. Scenario
just tells Cucumber we’re about to describe an example that it can execute. Then you see the lines beginning with Given, When and Then.
🎬 2: Highlight line 2
Given
is the context for the scenario. We’re putting the system into a specific state, ready for the scenario to unfold.
🎬 3
When
is an action. Something that happens to the system that will cause something else to happen: an outcome.
🎬 4
Then
is the outcome. It’s the behaviour we expect from the system when this action happens in this context.
You’ll notice we’ve omitted from our outcome anything about Lucy walking into Sean’s store and making a purchase. Remember, Gherkin is supposed to describe the behaviour of the system, so it would be distracting to have it in our scenario.
Each scenario has these three ingredients: 🎬 5 a context, 🎬 6 an action, 🎬 7 and one or more outcomes.
Together, they describe one single aspect of the behaviour of the system. An example.
Now that we’ve translated our example into Gherkin, we can automate it!
2.2.1. Lesson 2 - Questions
What’s an advantage of using Gherkin to express our examples in BDD? (choose one) ::
-
We can get Cucumber to test whether the code does what the scenario describes.
-
We can easily automate tests even if we don’t know much about programming.
-
We can use tools to generate the scenarios.
Explanation: Gherkin is just one way of expressing examples of how you want your system to behave. The advantage of using this particular format is that you can use Cucumber to test them for you, making them into Living Documentation.
Which of these are Gherkin keywords? (choose multiple)::
-
Scenario
-
Story
-
Given
-
Only
-
If
-
When
-
Before
-
Then
-
While
-
Check
Explanation:
We’ve introduced four Gherkin keywords so far:
* Scenario
tells Cucumber we’re about to describe an example that it can execute.
* Given
, When
and Then
identify the steps of the scenario.
There are a few other keywords which will be introduced later in the course.
The Gherkin keywords Given, When and Then, allow us to express three different components of a scenario. Which of these statements correctly describes how each of these keywords should be used? (Choose multiple)::
-
Given describes something that has already happened before the interesting part of the scenario starts. (Correct)
-
Then describes an action you want to take.
-
When explains what should happen at the end of the scenario.
-
Then explains what should happen at the end of the scenario. (Correct)
-
When expresses an action that changes the state of the system. (Correct)
-
Given describes the context in which the scenario occurs. (Correct)
-
Explanation:
-
Given is the context for the scenario. We’re putting the system into a specific state, ready for the scenario to unfold.
-
When is an action. Something that happens to the system that will cause something else to happen: an outcome.
-
Then is the outcome. It’s the behaviour we expect from the system when this action happens in this context.
Explanation: Given is the context for the scenario. We’re putting the system into a specific state, ready for the scenario to unfold.
When is an action. Something that happens to the system that will cause something else to happen: an outcome.
Then is the outcome. It’s the behaviour we expect from the system when this action happens in this context.
Why did our scenario not mention anything about Lucy walking into Sean’s store and making a purchase?
-
It’s a business goal which does not belong in a Gherkin document.
-
As BDD practitioners, we’re focussed on the behaviour of the system, so we don’t care about the people who use the software.
-
Including details about these two people would be distracting from the main point of our scenario.
-
Executable scenarios need to stay focussed on the behaviour of the system itself. We can document business goals elsewhere in our Gherkin to provide context. - TRUE
Explanation: Behaviour-Driven Development practitioners definitely do care about business goals, but when we’re writing the Scenario part of our Gherkin, we need to focus on the observable, testable behaviour of the system we’re building.
Later in the course we’ll show you how you can use other parts of Gherkin documents to add other relevant details, like business goals, to make great executable specifications.
2.3. Install Cucumber
Before we get started make sure you have a modern version of Ruby installed, and the Bundler gem. Open a command-prompt and check the Ruby version, 🎬 1 and the version of Bundler, Ruby’s package manager: 🎬 2
$ ruby -v
$ bundle -v
If you see an error message when you run these commands, you’ll need to fix your Ruby installation.
🎬 3 Go back to the command prompt and create a new directory for our project:
$ mkdir shouty
Use cd
to go into that directory. 🎬 4 If you want to, you can open the directory up in your favourite text editor at this point.
$ cd shouty
🎬 5 🎬 6 First we’ll create a Gemfile that describes the Ruby gems we need for our project. 🎬 7 We’ll add Cucumber and 🎬 8 RSpec-Expectations.
source "https://rubygems.org"
gem 'cucumber'
gem 'rspec'
Now go back to the command-line 🎬 9 and run bundle install
to install those gems.
$ bundle install
Now we’re ready! If we run cucumber
at this point 🎬 10, we’ll see it’s telling us to create a features directory.
$ bundle exec cucumber
No such file or directory - features. You can use `cucumber --init` to get started.
Good, we’ve installed Cucumber.
🎬 11
As instructed, we can use the cucumber --init
command to create the conventional folder structure for our Gherkin specifications and the code that will let Cucumber test them:
$ cucumber --init
create features
create features/step_definitions
create features/support
create features/support/env.rb
Now we’re ready to create our first feature file.
$ bundle exec cucumber
0 scenarios
0 steps
0m0.000s
2.4. Add a scenario, wire it up
Let’s create our first feature file. Call the file hear_shout.feature
🎬 1
$ touch features/hear_shout.feature
🎬 2
All feature files start with the keyword Feature:
🎬 3
followed by a name.
It’s a good convention to give it a name that matches the file name.
🎬 4 Now let’s write out our first scenario.
Feature: Hear shout
Scenario: Listener is within range
Given Lucy is located 15m from Sean
When Sean shouts "free bagels at Sean's"
Then Lucy hears Sean’s message
You’ll see Cucumber has found our feature file and read it back to us.🎬 7 We can see a summary of the test results below the scenario: 🎬 8 one scenario, 🎬 9 three steps - all undefined. 🎬 10
$ bundle exec cucumber
Feature: Hear shout
Scenario: Listener is within range # features/hear_shout.feature:2
Given Lucy is located 15m from Sean # features/hear_shout.feature:3
When Sean shouts "free bagels at Sean's" # features/hear_shout.feature:4
Then Lucy hears Sean's message # features/hear_shout.feature:5
1 scenario (1 undefined)
3 steps (3 undefined)
0m0.051s
You can implement step definitions for undefined steps with these snippets:
Given("Lucy is located {int}m from Sean") do |int|
pending # Write code here that turns the phrase above into concrete actions
end
When("Sean shouts {string}") do |string|
pending # Write code here that turns the phrase above into concrete actions
end
Then("Lucy hears Sean's message") do
pending # Write code here that turns the phrase above into concrete actions
end
🎬 11 Undefined means Cucumber doesn’t know what to do for any of the three steps we wrote in our Gherkin scenario. It needs us to provide some step definitions.
Step definitions translate from the plain language you use in Gherkin into Ruby code.
When Cucumber runs a step, it looks for a step definition that matches the text in the Gherkin step. If it finds one, then it executes the code in the step definition.
If it doesn’t find one… well, you’ve just seen what happens. Cucumber helpfully prints out some code snippets that we can use as a basis for new step definitions.
🎬 12 Let’s copy those to create our first step definitions.
🎬 13 🎬 14
We’ll paste them into a Ruby file under the step_definitions
directory, inside the features
directory. We’ll just call it steps.rb
.
Given("Lucy is located {int}m from Sean") do |int|
pending # Write code here that turns the phrase above into concrete actions
end
When("Sean shouts {string}") do |string|
pending # Write code here that turns the phrase above into concrete actions
end
Then("Lucy hears Sean's message") do
pending # Write code here that turns the phrase above into concrete actions
end
🎬 15 Now run Cucumber again.
This time the output is a little different. None of the steps are undefined anymore. We now have a pending step 🎬 16 and two skipped ones.🎬 17 This means Cucumber found all our step definitions, and executed the first one.
🎬 18
But that first step definition throws a PendingException
, which causes Cucumber to stop, skip the rest of the steps, and mark the scenario as pending.
$ bundle exec cucumber
Feature: Hear shout
Scenario: Listener is within range # features/hear_shout.feature:2
Given Lucy is located 15m from Sean # features/step_definitions/steps.rb:1
TODO (Cucumber::Pending)
./features/step_definitions/steps.rb:2:in `"Lucy is located {int}m from Sean"'
features/hear_shout.feature:3:in `Given Lucy is located 15m from Sean'
When Sean shouts "free bagels at Sean's" # features/step_definitions/steps.rb:5
Then Lucy hears Sean's message # features/step_definitions/steps.rb:9
1 scenario (1 pending)
3 steps (2 skipped, 1 pending)
0m0.008s
Now that we’ve wired up our step definitions to the Gherkin steps, it’s almost time to start working on our solution. First though, let’s tidy up the generated code.
🎬 19
We’ll rename the int
parameter to something that better reflects its meaning. We’ll call it distance
.
🎬 20 We can print it to the terminal to see what’s happening.
Given("Lucy is located {int}m from Sean") do |distance|
puts distance
pending # Write code here that turns the phrase above into concrete actions
end
When("Sean shouts {string}") do |string|
pending # Write code here that turns the phrase above into concrete actions
end
Then("Lucy hears Sean's message") do
pending # Write code here that turns the phrase above into concrete actions
end
$ bundle exec cucumber
Feature: Hear shout
Scenario: Listener is within range # features/hear_shout.feature:2
Given Lucy is located 15m from Sean # features/step_definitions/steps.rb:1
15
TODO (Cucumber::Pending)
./features/step_definitions/steps.rb:3:in `"Lucy is located {int}m from Sean"'
features/hear_shout.feature:3:in `Given Lucy is located 15m from Sean'
When Sean shouts "free bagels at Sean's" # features/step_definitions/steps.rb:6
Then Lucy hears Sean's message # features/step_definitions/steps.rb:10
1 scenario (1 pending)
3 steps (2 skipped, 1 pending)
0m0.005s
Notice that the number 15 does not appear anywhere in our Ruby code. The value is automatically passed from the Gherkin step to the step definition. If you’re curious, that’s the 🎬 23{int}
in the step definition pattern or cucumber expression. We’ll explain these patterns in detail in a future lesson.
2.4.1. Lesson 4 - Questions
What do step definitions do? (choose one) ::
-
Provide a glossary of domain terms for your stakeholders
-
Give Cucumber/SpecFlow a way to automate your gherkin steps - TRUE
-
Add extra meaning to our Gherkin steps
-
Generate code from gherkin documents
Explanation: <java> Step definitions are Java methods that actually do what’s described in each step of a Gherkin scenario.
When it tries to run each step of a scenario, Cucumber will search for a step definition that matches. If there’s a matching step definition, it will call the method to run it. </java>
<js> Step definitions are JavaScript functions that actually do what’s described in each step of a Gherkin scenario.
When it tries to run each step of a scenario, Cucumber will search for a step definition that matches. If there’s a matching step definition, it will call the function. </js>
<ruby> Step definitions are Ruby blocks that actually do what’s described in each step of a Gherkin scenario.
When it tries to run each step of a scenario, Cucumber will search for a step definition that matches. If there’s a matching step definition, it will execute the code in the block. </ruby>
<C#> Step definitions are C# methods that actually do what’s described in each step of a Gherkin scenario.
When it tries to run each step of a scenario, SpecFlow will search for a step definition that matches. If there’s a matching step definition, it will call the method to run it. </C#>
What does it mean when Cucumber/SpecFlow says a step is Pending? (choose one) ::
-
The step took too long to execute and was terminated <java> * The step threw a
PendingException
, meaning we’re still working on implementing that step.</java> <js> * The step returned pending, meaning we’re still working on implementing that step.</js> <ruby> * The step definition threw a Pending error, meaning we’re still working on implementing that step.</ruby> <C#> ASK GASPAR </C#> -
Cucumber/SpecFlow was unable to find the step definitions
-
The scenario is passing
-
The scenario is failing
Explanation:
<java> Cucumber tells us that a step (and by inference the Scenario that contains it) is Pending when the automation code throws a PendingException.
The PendingException is a special type of exception provided by Cucumber to allow the development team to signal that automation for a step is a work in progress. This makes it possible to tell the difference between steps that aren’t finished yet and steps that are failing due to a defect in the system.
For example, when we run our tests in a Continuous Integration (CI) environment, we can choose to ignore pending scenarios. </java>
<js> Cucumber tells us that a step (and by inference the Scenario that contains it) is Pending when the automation code throws a Pending error.
This allows the development team to signal that automation for a step is a work in progress. This makes it possible to tell the difference between steps that are still being worked on and steps that are failing due to a defect in the system.
For example, when we run our tests in a Continuous Integration (CI) environment, we can choose to ignore pending scenarios. </js>
<ruby> Cucumber tells us that a step (and by inference the Scenario that contains it) is Pending when the automation code throws a Pending error.
This allows the development team to signal that automation for a step is a work in progress. This makes it possible to tell the difference between steps that are still being worked on and steps that are failing due to a defect in the system.
For example, when we run our tests in a Continuous Integration (CI) environment, we can choose to ignore pending scenarios. </ruby>
<C#> ASK GASPAR </C#>
Which of the following might you want to consider when using a snippet generated by Cucumber/SpecFlow?
-
Does the name of the method correctly describe the intent of the step? - TRUE
-
Do the parameter names correctly describe the meaning of the arguments? - TRUE
-
Does the snippet correctly automate the gherkin step as described? - FALSE
Explanation: When Cucumber/SpecFlow generates a snippet, it has no idea of the business context of the undefined step. The implementation that Cucumber/SpecFlow generates will definitely not automate what’s been written in your Gherkin - that’s up to you! Also, the name of the method and the parameters are just placeholders. It’s the job of the person writing the code to rename the method and parameters to reflect the business domain.
What’s the next step in BDD after we’ve pasted in the step definition snippet and seen it fail with a pending
status?
-
Check with our project manager about the requirement
-
Implement some code that does what the Gherkin step describes - TRUE
-
Create a test framework for modelling our application
-
Run a manual test to check what the system does
Explanation: If you read the comment in the generated snippet, Cucumber/SpecFlow is telling you to "turn the phrase above into concrete actions".
You need your step definition to call your application and do whatever the Gherkin step describes. In the case of our first step here, we want to tell the system that there are two people in certain locations.
We can use the act of fleshing out the body of our step definition as an opportunity to do some software design. We can think about what we want the interface to our system to look like, from the point of view of someone who needs to interact with it. Should we interact with it through the User Interface, or make a call to the programmer API directly? How would we like that interface to work?
We can do all of this without writing any implementation yet.
This is known as "outside-in" development. It helps us to ensure that when we do come to implementing our solution, we’re implementing it based on real needs.
2.5. Sketch out the solution
🎬 1 Now that we have the step definitions matching, we can start working on our solution. We like to use our scenarios to guide our development, so we’ll start designing the objects we’ll need by sketching out some code in our step definitions.
The scenario will be failing while we do this, but we should see the error messages gradually progressing as we drive out the interface to our object model.
Our next goal is for the scenario to fail because we need to implement the actual business logic. Then we can work on changing the business logic inside our objects to make it pass.
Given("Lucy is located {int}m from Sean") do |distance|
puts distance
pending # Write code here that turns the phrase above into concrete actions
end
When("Sean shouts {string}") do |string|
pending # Write code here that turns the phrase above into concrete actions
end
Then("Lucy hears Sean's message") do
pending # Write code here that turns the phrase above into concrete actions
end
🎬 2
To implement the first step, we need to create a couple of Person
objects, with the specified distance between them. 🎬 3 We can remove the pending
status now, and this puts
statement,🎬 4 and write the implementation for the first step like this:
Given("Lucy is located {int}m from Sean") do |distance|
@lucy = Shouty::Person.new
@sean = Shouty::Person.new
@lucy.move_to(distance)
end
When("Sean shouts “free bagels at Sean’s”") do
pending # Write code here that turns the phrase above into concrete actions
end
Then("Lucy hears Sean’s message") do
pending # Write code here that turns the phrase above into concrete actions
end
We have two instances of person, one representing Lucy, 🎬 5and one representing Sean. 🎬 6Then we call a method to move Lucy to the position specified in the scenario.🎬 7
To keep things simple, we’re going to assume all people are situated on a line: a one-dimensional co-ordinate system. We can always introduce proper geo-locations later. We’ll place Sean in the centre, and Lucy 15 metres away from Sean.
This might not be the design we’ll end up with once this is all working, but it’s a decent place to start.
🎬 8
If we run Cucumber, we’ll see a compilation error from Ruby. we need to define our Shouty
module.
$ bundle exec cucumber
Feature: Hear shout
Scenario: Listener is within range # features/hear_shout.feature:2
Given Lucy is located 15m from Sean # features/step_definitions/steps.rb:1
uninitialized constant Shouty (NameError)
./features/step_definitions/steps.rb:2:in `"Lucy is located {int}m from Sean"'
features/hear_shout.feature:3:in `Given Lucy is located 15m from Sean'
When Sean shouts "free bagels at Sean's" # features/step_definitions/steps.rb:7
Then Lucy hears Sean's message # features/step_definitions/steps.rb:11
Failing Scenarios:
cucumber features/hear_shout.feature:2 # Scenario: Listener is within range
1 scenario (1 failed)
3 steps (1 failed, 2 skipped)
0m0.004s
🎬 9
Let’s give our solution a home by creating a lib
directory.
🎬 10
We’ll put our Shouty application in a Ruby file called shouty.rb
in that directory.
$ mkdir lib
$ touch lib/shouty.rb
module Shouty
class Person
end
end
🎬 13
For now, we’ll just require the shouty application from our steps.rb
file. In a later lesson we’ll talk more about how to organise this code a bit better.
require 'shouty'
Given("Lucy is located {int}m from Sean") do |distance|
# ...
end
🎬 14
When we run the scenario this time, Cucumber tells us that we’re missing the move_to
method on the Person
class. 🎬 15
$ bundle exec cucumber
Feature: Hear shout
Scenario: Listener is within range # features/hear_shout.feature:2
Given Lucy is located 15m from Sean # features/step_definitions/steps.rb:3
undefined method `move_to' for #<Shouty::Person:0x00007fed0da12b68> (NoMethodError)
./features/step_definitions/steps.rb:6:in `"Lucy is located {int}m from Sean"'
features/hear_shout.feature:3:in `Given Lucy is located 15m from Sean'
When Sean shouts "free bagels at Sean's" # features/step_definitions/steps.rb:9
Then Lucy hears Sean's message # features/step_definitions/steps.rb:13
Failing Scenarios:
cucumber features/hear_shout.feature:2 # Scenario: Listener is within range
1 scenario (1 failed)
3 steps (1 failed, 2 skipped)
0m0.004s
🎬 16 We’ll add the method definition without implementing it yet.
module Shouty
class Person
def move_to(distance)
end
end
end
🎬 17 When we run the scenario again, the first step is green!
bundle exec cucumber
Feature: Hear shout
Scenario: Listener is within range # features/hear_shout.feature:2
Given Lucy is located 15m from Sean # features/step_definitions/steps.rb:3
When Sean shouts "free bagels at Sean's" # features/step_definitions/steps.rb:9
TODO (Cucumber::Pending)
./features/step_definitions/steps.rb:10:in `"Sean shouts {string}"'
features/hear_shout.feature:4:in `When Sean shouts "free bagels at Sean's"'
Then Lucy hears Sean's message # features/step_definitions/steps.rb:13
1 scenario (1 pending)
3 steps (1 skipped, 1 pending, 1 passed)
0m0.006s
We’re making progress!
We’ll keep working like this until we see the scenario failing for the right reasons.
In the second step definition, we want to tell Sean to shout something.
🎬 18 In order to send instructions to Sean from the second step, we’ve stored him in an instance variable, so that he’ll be accessible from all of our step definitions.
🎬 19
In the When
step, we’re capturing Sean’s message using the {string}
pattern, so let’s give that argument a more meaningful name.🎬 20
🎬 21 And now we can now tell him to shout the message:
When("Sean shouts {string}") do |message|
@sean.shout(message)
end
🎬 22
If we run Cucumber, it’s telling is that our Person
class needs a shout
method.
bundle exec cucumber
Feature: Hear shout
Scenario: Listener is within range # features/hear_shout.feature:2
Given Lucy is located 15m from Sean # features/step_definitions/steps.rb:4
When Sean shouts "free bagels at Sean's" # features/step_definitions/steps.rb:10
undefined method `shout' for #<Shouty::Person:0x0000558e1ee78428> (NoMethodError)
./features/step_definitions/steps.rb:11:in `"Sean shouts {string}"'
features/hear_shout.feature:4:in `When Sean shouts "free bagels at Sean's"'
Then Lucy hears Sean’s message # features/step_definitions/steps.rb:14
Failing Scenarios:
cucumber features/hear_shout.feature:2 # Scenario: Listener is within range
1 scenario (1 failed)
3 steps (1 failed, 1 skipped, 1 passed)
0m0.003s
🎬 23 Let’s not worry about the implementation yet. The most important thing right now is to discover the shape of our domain model.
module Shouty
class Person
def move_to(distance)
end
def shout(message)
end
end
end
🎬 24 The last step definition is where we implement a check, or assertion. We’ll verify that what Lucy has heard is exactly the same as what Sean shouted.
🎬 25 Once again we’re going to write the code we wish we had.
Then("Lucy hears Sean's message") do
expect(@lucy.messages_heard).to include @message_from_sean
end
So we need a way to ask Lucy what messages she’s heard, and we also need to know what it was that Sean shouted.
We can record what Sean shouts by storing it in an instance variable as the When
step runs.🎬 26 This is a common pattern to use in Cucumber step definitions when you don’t want to repeat the same test data in different parts of a scenario. Now we can use that in the assertion check.🎬 27
When("Sean shouts {string}") do |message|
@sean.shout(message)
@message_from_sean = message
end
We also need to add a messages_heard
method to our Person class.🎬 28 Let’s do that now, we’ll just return an empty array for now.🎬 29
module Shouty
class Person
def move_to(distance)
end
def shout(message)
end
def messages_heard
[]
end
end
end
…and watch Cucumber run the tests again.🎬 30
$ bundle exec cucumber
Feature: Hear shout
Scenario: Listener is within range # features/hear_shout.feature:2
Given Lucy is located 15m from Sean # features/step_definitions/steps.rb:4
When Sean shouts "free bagels at Sean's" # features/step_definitions/steps.rb:10
Then Lucy hears Sean’s message # features/step_definitions/steps.rb:15
expected [] to include "free bagels at Sean's" (RSpec::Expectations::ExpectationNotMetError)
./features/step_definitions/steps.rb:16:in `"Lucy hears Sean’s message"'
features/hear_shout.feature:5:in `Then Lucy hears Sean’s message'
Failing Scenarios:
cucumber features/hear_shout.feature:2 # Scenario: Listener is within range
1 scenario (1 failed)
3 steps (1 failed, 2 passed)
0m0.032s
This is great! Whenever we do BDD, getting to our first failing test is a milestone. Seeing the test fail proves that it is capable of detecting errors in our code!
Never trust an automated test that you haven’t seen fail!
Now all we have to do is write the code to make it do what it’s supposed to.
2.5.1. Lesson 5 - Questions
How does the practice writing a failing test before implementing the solution help us?
-
Until you see a scenario fail, you can’t be sure that it can ever fail [true]
-
There’s no need to always see a scenario fail [false]
-
BDD practitioners use failing scenarios to guide their development [true]
-
A passing scenario implies the functionality it describes has already been implemented, so it may be a duplicate of an existing scenario [true]
-
BDD practitioners believe in learning from failure [false]
Explanation: Behaviour-Driven Development comes from Test-Driven Development, where we always start with a failing test, then use that to guide our development. This sometimes described as red-green-refactor.
red - write a scenario/test and see it fail green - make it pass (as simply as possible) refactor - improve your code, while keeping all the tests/scenarios green
It’s surprisingly easy to write scenarios and step definitions that don’t do anything. It’s the transition from red to green that gives us confidence that the scenario and the implementation actually do what we expect.
If a scenario passes as soon as we write it, that means that either it’s not doing what we think it should or the behaviour that it describes has already been implemented. In that case, we’re not developing using behaviour-driven development.
Why did we change to use an instance variable for storing each Person? (select one) ::
-
It ensures we can interact with the same object from different steps. [true]
-
It’s a better way to organise the code
-
It’s more efficient for performance
-
Cucumber/SpecFlow requires us to store our objects as instance variables.
Explanation: In Cucumber/SpecFlow, one of the ways to access the same instance of an object from different step definition methods, is to store it on an instance variable.
How did we avoid having to mention the detail of the text Sean had shouted in our When and Then steps? (select one) ::
-
We duplicated the text inside our Person class
-
We used an instance variable to store the text that was shouted [true]
-
We called a method on the Person class to retrieve the messages heard
-
We passed the message text in from our Gherkin scenarios
Explanation: When you need to assert for a specific value coming out of your system in a Then step, you can use an instance variable to store it where it goes into the system (in a Given or When) step. This means you can avoid duplicating the value in multiple places in your code.
Which flow should we follow when making a Scenario pass? (select one) ::
-
Domain modelling → Write some code → Make it compile → Run the scenario & watch it fail
-
Write some code → Domain modelling → Make it compile → Run the scenario
-
Write some code → Make it compile → Domain modelling → Run the scenario - TRUE
-
Domain modelling → Run the scenario → Write some code → Make it compile
Explanation: Our goal at this stage is to get to a failing test, where the only thing left to do to make it pass is make changes to the implementation of the app itself.
On an existing system, we might not need to create so much new code to get to this goal, but we might need to make some changes to how we call the system. This gives us an opportunity to do some lightweight domain modelling.
It may not compile first-time, so we implement the bare-bones of our solution until it does.
We use the scenarios to guide us in our implementation.
2.6. Make the scenario pass
So we have our failing scenario: 🎬 1
bundle exec cucumber
Feature: Hear shout
Scenario: Listener is within range # features/hear_shout.feature:2
Given Lucy is located 15m from Sean # features/step_definitions/steps.rb:4
When Sean shouts "free bagels at Sean's" # features/step_definitions/steps.rb:10
Then Lucy hears Sean’s message # features/step_definitions/steps.rb:15
expected [] to include "free bagels at Sean's" (RSpec::Expectations::ExpectationNotMetError)
./features/step_definitions/steps.rb:16:in `"Lucy hears Sean’s message"'
features/hear_shout.feature:5:in `Then Lucy hears Sean’s message'
Failing Scenarios:
cucumber features/hear_shout.feature:2 # Scenario: Listener is within range
1 scenario (1 failed)
3 steps (1 failed, 2 passed)
0m0.032s
Lucy is expected to hear Sean’s message, but she hasn’t heard anything: we got an empty Array back from the messages_heard
method.🎬 2
🎬 3 In this case, we’re going to cheat. We have a one-line fix that will make this scenario pass, but it’s not a particularly future-proof implementation. Can you guess what it is?🎬 4
module Shouty
class Person
def move_to(distance)
end
def shout(message)
end
def messages_heard
["free bagels at Sean's"]
end
end
end
I told you it wasn’t very future proof!
$ bundle e xec cucumber
Feature: Hear shout
Scenario: Listener is within range # features/hear_shout.feature:2
Given Lucy is located 15m from Sean # features/step_definitions/steps.rb:4
When Sean shouts "free bagels at Sean's" # features/step_definitions/steps.rb:10
Then Lucy hears Sean’s message # features/step_definitions/steps.rb:15
1 scenario (1 passed)
3 steps (3 passed)
0m0.028s
Woohoo! Our scenario is passing for the first time. As long as this is the only message anyone ever shouts, we’re good to ship this thing!
Now, the fact that such a poor implementation can pass our tests shows us that we need to work on our tests. A more comprehensive set of scenarios would guide us towards a better implementation.
It’s also a good habit to look for the most simple solution. We can trust that, as our scenarios evolve, so will our solution.
This is the essence of Behaviour-Driven Development. Examples of behaviour drive the development. We do just enough to make the next scenario pass, and no more.
Instead of writing a note on our TODO list, let’s write another scenario that shouts a different message. 🎬 6
Feature: Hear shout
Scenario: Listener is within range
Given Lucy is located 15m from Sean
When Sean shouts "free bagels at Sean's"
Then Lucy hears Sean’s message
Scenario: Listener hears a different message
Given Lucy is located 15m from Sean
When Sean shouts "Free coffee!"
Then Lucy hears Sean's message
It fails, reminding us we need to find a solution that doesn’t rely on hard-coding the message. 🎬 7 Now when we come back to this code, we can just run the tests and Cucumber will remind us what we need to do next. We’re done for today!
$ bundle exec cucumber
Feature: Hear shout
Scenario: Listener is within range # features/hear_shout.feature:2
Given Lucy is located 15m from Sean # features/step_definitions/steps.rb:3
When Sean shouts "free bagels at Sean's" # features/step_definitions/steps.rb:9
Then Lucy hears Sean's message # features/step_definitions/steps.rb:14
Scenario: Listener hears a different message # features/hear_shout.feature:7
Given Lucy is located 15m from Sean # features/step_definitions/steps.rb:3
When Sean shouts "Free coffee!" # features/step_definitions/steps.rb:9
Then Lucy hears Sean's message # features/step_definitions/steps.rb:14
expected ["free bagels at Sean's"] to include "Free coffee!" (RSpec::Expectations::ExpectationNotMetError)
./features/step_definitions/steps.rb:15:in `"Lucy hears Sean's message"'
features/hear_shout.feature:10:in `Then Lucy hears Sean's message'
Failing Scenarios:
cucumber features/hear_shout.feature:7 # Scenario: Listener hears a different message
2 scenarios (1 failed, 1 passed)
6 steps (1 failed, 5 passed)
0m0.039s
Of course, if you’re in the mood, you can always try to implement a solution yourself that makes both scenarios pass. Have fun!
2.6.1. Questions
-
Why should we always make sure that we see a scenario fail before we make it pass? (select multiple)
-
Until you see a scenario fail, you can’t be sure that it can ever fail [true]
-
There’s no need to always see a scenario fail [false]
-
BDD practitioners use failing scenarios to drive their development [true]
-
A passing scenario implies the functionality it describes has already been implemented, so it may be a duplicate of an existing scenario [true]
-
BDD practitioners believe in learning from failure [false]
-
-
Why did we change to use an instance variable for storing each Person?
-
It ensures we can interact with the same object from different steps. [true]
-
It’s a better way to organise the code
-
It’s more efficient for performance
-
Cucumber requires us to store our objects as instance variables.
-
-
How did we avoid having to mention the detail of the text Sean had shouted in our When and Then steps?
-
We duplicated the text inside our Person class
-
We used an instance variable to store the text that was shouted [true]
-
We called a method on the Person class to retrieve the messages heard
-
We passed the message text in from our Gherkin scenarios
-
-
Which flow should we follow when making a Scenario pass?
-
Domain modelling → Write some code → Make it compile → Run the scenario & watch it fail
-
Write some code → Domain modelling → Make it compile → Run the scenario
-
Write some code → Make it compile → Domain modelling → Run the scenario
-
Domain modelling → Run the scenario → Write some code → Make it compile
-
-
Why is our naive implementation of Person.getMessagesHeard, with a hard-coded message OK in BDD? (select multiple)
-
It shows us that we need better examples to pin down the behaviour we really want from the code. [correct]
-
We know we will iterate on our solution, when we come up with more examples of what we want it to do. [correct]
-
Nobody is using our solution yet [incorrect]
-
We have to do a bad implementation so we can see our test fail. [incorrect]
-
-
Look at this diagram (1) Write a scenario, 2) Automate it and watch it fail, 3) Write just enough code to make it pass). Which stage are we at as the video ends?
-
1
-
2
-
3
-
3. Expressing yourself
3.1. Cucumber expressions not regular expressions
In the previous chapter we explored the fundamental components of a Cucumber test suite, and how we use Cucumber to drive out a solution, test-first.
First we specified the behaviour we wanted, using a Gherkin scenario in a feature file. Then we wrote step definitions to translate the plain english from our scenario into concrete actions in code. Finally, we used the step definitions to guide us in building out our very basic domain model for the Shouty application.
We tend to think of the code that actually pokes around with the system as distinct from the step definitions, so we’ve drawn an extra box labelled "automation code" for this.
Automation code can do almost anything to your application: it can drive a web browser around your site, make HTTP requests to a REST API, or — as you’ve already seen — drive a domain model directly.
Automation code is a big topic that we’ll come back to. First, we want to concentrate on step definitions. Good step definitions are important because they enable the readability of your scenarios. The better you are at matching plain language phrases from Gherkin, the more expressive you can be when writing scenarios. Teams who do this well refer to their features as living documentation - a specification document that never goes out of date.
When Cucumber first started, we used to use regular expressions to match plain language phrases from Gherkin steps.
Regular expressions have quite an intimidating reputation.
So we replaced them with something simpler, something we call Cucumber expressions. Cucumber is backwards compatible so you can still use the power of regular expressions if that’s your thing.
This chapter is all about Cucumber Expressions.
3.1.1. Lesson 1 - Questions (Ruby, Java, JS)
Which of the following statements are true?
-
Step definitions translate human-readable scenarios into concrete actions in code - TRUE
-
BDD practitioners think of "step definitions" and "automation code" as distinct concepts - TRUE
-
Cucumber only supports automation through the user interface - FALSE
Answer: A step definition is a piece of code that is called by Cucumber in response to a step in a scenario. You can write any code you like inside a step definition, but we’ve found it easier to maintain if we keep them short. This leads to step definitions calling dedicated automation code to perform concrete actions against the system under construction. That automation code can manipulate the user interface, make a REST call, or drive the domain model directly.
Which of the following statements are true?
-
Regular Expressions are exactly the same as Cucumber Expressions - FALSE
-
Modern versions of Cucumber only support both Cucumber Expressions and Rregular Expressions - TRUE
-
Cucumber Expressions are more intimidating than Regular Expressions - FALSE
Answer: Regular Expressions are a powerful tool that have been in use in computer science for many decades. They can be hard understand and maintain, so the Cucumber team created a simplified mechanism, called Cucumber Expressions. However, Cucumber remains backwards compatible, so you can use both Regular Expressions and Cucumber Expressions with modern releases of Cucumber.
3.1.2. Lesson 1 - Questions (SpecFlow/C#/Dotnet)
Which of the following statements are true?
-
Step definitions translate human-readable scenarios into concrete actions in code - TRUE
-
BDD practitioners think of "step definitions" and "automation code" as distinct concepts - TRUE
-
SpecFlow only supports automation through the user interface - FALSE
Answer: A step definition is a piece of code that is called by SpecFlow in response to a step in a scenario. You can write any code you like inside a step definition, but we’ve found it easier to maintain if we keep them short. This leads to step definitions calling dedicated automation code to perform concrete actions against the system under construction. That automation code can manipulate the user interface, make a REST call, or drive the domain model directly.
Which of the following statements are true?
-
Regular Expressions are exactly the same as Cucumber Expressions - FALSE
-
In modern versions of SpecFlow steps can be defined using Cucumber Expressions but this feature has to be enabled first - TRUE
-
Cucumber Expressions are more intimidating than Regular Expressions - FALSE
-
Regular expressions can still be used even in modern versions of SpecFlow even if Cucumber Expressions are enabled - TRUE
Answer: Regular Expressions are a powerful tool that have been in use in computer science for many decades. They can be hard understand and maintain, so the Cucumber team created a simplified mechanism, called Cucumber Expressions that is now also available for SpecFlow. SpecFlow remains backwards compatible, so you can use both Regular Expressions and Cucumber Expressions with modern releases of SpecFlow.
3.2. Literal expressions
🎬 1 Let’s look at the Shouty scenario from the last chapter.
Feature: Hear shout
Scenario: Listener is within range
Given Lucy is located 15 metres from Sean
When Sean shouts "free bagels at Sean's"
Then Lucy hears Sean's message
As Cucumber starts to execute this feature, it will come to the first step of the scenario Given Lucy is located 15 metres from Sean
🎬 2 and say to itself "now - do I have any step definitions that match the phrase Lucy is located 15 metres from Sean
?”"
🎬 3 The most simple cucumber expression that would match that step is this one:
Lucy is located 15 metres from Sean
That’s pretty simple isn’t it? Cucumber expressions are just string patterns, and the most simple pattern you can use is a perfect match.
In Ruby, we can use this pattern to make a step definition like this: 🎬 4
Given("Lucy is located 15 metres from Sean") do
# TODO: automation code to place Lucy and Sean goes here
pending "matched!"
end
We use a normal Ruby string to pass the cucumber expression to Cucumber.
3.2.1. Lesson 2 - Questions
Which of the following Cucumber Expressions will match the step "Given Lucy is 15 metres from Sean"?
-
"lucy is 15 metres from sean" - FALSE
-
"Given Lucy is 15 metres from Sean" - FALSE
-
"Lucy is 15 metres from Sean" - TRUE
-
"Lucy is 15 metres from Sean Smith" - FALSE
Answer: Cucumber Expressions look for a match of the whole step text EXCLUDING the Gherkin keyword (Given/When/Then/And/But). The match is case sensitive and matches whitespace as well.
3.3. Capturing parameters
🎬 1 Sometimes, we want to write step definitions that allow us to use different values in our Gherkin scenarios. For example, we might want to have other scenarios that place Lucy a different distance away from Sean. 🎬 2
Feature: Hear shout
Scenario: Listener is within range
Given Lucy is located 100 metres from Sean
When Sean shouts "free bagels at Sean's"
Then Lucy hears Sean's message
To capture interesting values from our step definitions, we can use a feature of Cucumber Expressions called parameters.
For example, to capture the number of metres, we can use the {int}
parameter: 🎬 3 which is passed as an argument to our step definition: 🎬 4
Given("Lucy is located {int} metres from Sean") do |distance|
Now we’re capturing that value as an argument. The value 100
will be passed to our code automatically by Cucumber.
Because we’ve used Cucumber Expressions' built-in {int}
parameter type, the value has been cast to an Integer
data type for us automatically, so we can do maths with it if we want.🎬 5
Given("Lucy is located {int} metres from Sean") do |distance|
# TODO: automation code to place Lucy and Sean goes here
pending "Lucy is #{distance * 100} centimetres from Sean"
end
Cucumber has a bunch of built-in parameter types: {int}
, {float}
, {word}
and {string}
. You can also define your own, as we’ll see later.
3.3.1. Lesson 3 )Ruby, Java, JS)
Which of the following is NOT a built in Cucumber Expression parameter type?
-
float - FALSE
-
integer - TRUE
-
string - FALSE
-
word - FALSE
Answer: The Cucumber Expression parameter type that matches an integer is {int}
, not {integer}
Which of the following statements is true?
-
You cannot create your own Cucumber Expression parameter types - FALSE
-
Cucumber discards the value that matches a Cucumber Expression parameter type - FALSE
-
Your step definition code will be passed the value that matched the Cucumber Expression parameter type - TRUE
-
Cucumber always passes the matched parameter as a string - FALSE
Answer: Cucumber will pass the step definition a parameter for each Cucumber Expression parameter type. Cucumber will attempt to convert the text that matched into a suitable format. Using the {int}
parameter type will result in a number being passed to the step definition. You can extend the predefined Cucumber Expression parameter types, by creating your own.
3.3.2. Lesson 3 - Questions (SpecFlow/C#/Dotnet)
Which of the following is NOT a built in Cucumber Expression parameter type?
-
float - FALSE
-
integer - TRUE
-
string - FALSE
-
word - FALSE
Answer: The Cucumber Expression parameter type that matches an integer is {int}
, not {integer}
Which of the following statements is true?
-
You cannot create your own Cucumber Expression parameter types - FALSE
-
Cucumber discards the value that matches a Cucumber Expression parameter type - FALSE
-
Your step definition code will be passed the value that matched the Cucumber Expression parameter type - TRUE
-
SpecFlow always passes the matched parameter as a string - FALSE
Answer: SpecFlow will pass the step definition a parameter for each Cucumber Expression parameter type. SpecFlow will attempt to convert the text that matched into a suitable format. Using the {int}
parameter type will result in a number being passed to the step definition. You can extend the predefined Cucumber Expression parameter types, by creating your own.
3.4. Flexibility
Although it’s important to try to use consistent terminology in our Gherkin scenarios to help develop the ubiquitous language of your domain, we also want scenarios to read naturally, which sometimes means allowing a bit of flexibility.
Ideally, the language used in scenarios should never be constrained by your step definitions. Otherwise they’ll end up sounding like they were written by robots. Or worse, they read like code.
🎬 1 One common example is the problem of plurals. Suppose we want to place Lucy and Sean just 1 metre apart:
Given Lucy is located 1 metre from Sean
🎬 2
Because we’ve used the singular metre
instead of the plural metres
we don’t get a match:
$ bundle exec cucumber
Feature: Hear shout
Scenario: Listener is within range # features/hear_shout.feature:2
Given Lucy is located 1 metre from Sean # features/hear_shout.feature:3
When Sean shouts "free bagels at Sean's" # features/step_definitions/steps.rb:8
Then Lucy hears Sean's message # features/step_definitions/steps.rb:13
1 scenario (1 undefined)
3 steps (2 skipped, 1 undefined)
0m0.017s
You can implement step definitions for undefined steps with these snippets:
Given("Lucy is located {int} metre from Sean") do |int|
pending # Write code here that turns the phrase above into concrete actions
end
What a pain!
Fear not. We can just surround the s
in parentheses to make it optional, like this: 🎬 3
Given("Lucy is located {int} metre(s) from Sean") do |distance|
🎬 4 Now our step matches:
$ bundle exec cucumber
Feature: Hear shout
Scenario: Listener is within range # features/hear_shout.feature:2
Given Lucy is located 1 metre from Sean # features/step_definitions/steps.rb:3
Lucy is 100 centimetres from Sean (Cucumber::Pending)
./features/step_definitions/steps.rb:5:in `"Lucy is located {int} metre(s) from Sean"'
features/hear_shout.feature:3:in `Given Lucy is located 1 metre from Sean'
When Sean shouts "free bagels at Sean's" # features/step_definitions/steps.rb:8
Then Lucy hears Sean's message # features/step_definitions/steps.rb:13
1 scenario (1 pending)
3 steps (2 skipped, 1 pending)
This is one way to smooth off some of the rough edges in your cucumber expressions, and allow your scenarios to be as expressive as possible.
Another is to allow alternates - different ways of saying the same thing. For example, to accept this step: 🎬 5
Given Lucy is standing 1 metre from Sean
…we can use this Cucumber Expression: 🎬 6
Given("Lucy is located/standing {int} metre(s) from Sean") do |distance|
🎬 7 Now we can use either 'standing' or 'located' in our scenarios, and both will match just fine:
$ bundle exec cucumber
Feature: Hear shout
Scenario: Listener is within range # features/hear_shout.feature:2
Given Lucy is standing 1 metre from Sean # features/step_definitions/steps.rb:3
Lucy is 100 centimetres from Sean (Cucumber::Pending)
./features/step_definitions/steps.rb:5:in `"Lucy is located/standing {int} metre(s) from Sean"'
features/hear_shout.feature:3:in `Given Lucy is standing 1 metre from Sean'
When Sean shouts "free bagels at Sean's" # features/step_definitions/steps.rb:8
Then Lucy hears Sean's message # features/step_definitions/steps.rb:13
1 scenario (1 pending)
3 steps (2 skipped, 1 pending)
3.4.1. Lesson 4
How can you express in a Cucumber Expression that matching some text is optional?
-
Enclose it in square brackets: [] - FALSE
-
Enclose it in parentheses: () - TRUE
-
Place a question mark after it: ? - FALSE
-
Precede it with a slash: / - FALSE
Answer: Any text in a Cucumber Expression that is surrounded by parentheses ()
is considered optional.
What does a slash /
separating words mean in a Cucumber Expression?
-
The words are considered alternatives - the Cucumber Expression will match any of them - TRUE
-
It doesn’t mean anything special - the Cucumber Expression will match the slash as a literal character- FALSE
-
The word that follows the slash is considered optional - FALSE
Answer: Words in a Cucumber Expression that are separated by a slash /
are considered alternates. There must be no whitespace between the word and the slash.
Which of the following Cucumber Expressions would match both "it weighed 3 grammes" and "it weighed 1 gramme"?
-
"it weighed {int} gramme(s)" - TRUE
-
"it weighed 1/3 gramme/s" - FALSE
-
"it weighed 1/3 gramme(s)" - TRUE
-
"it weighed 1 / 3 gramme(s)" - FALSE
-
"it weighed 1/2/3 gramme/grammes" - TRUE
Answer: Any text surrounded by parentheses ()
is considered optional. Any words separated by a slash /
are considered to be alternates. You can find full documentation about Cucumber Expressions at https://cucumber.io/docs/cucumber/cucumber-expressions/
3.5. Custom parameters
Although you can get a long way with Cucumber Expressions' built-in parameter types, you get real power when you define your own custom parameter types. This allows you to transform the text captured from the Gherkin into any object you like before it’s passed into your step definition.
For example, let’s define our own {person}
custom parameter type that will convert the string Lucy
into an instance of Person
automatically.
🎬 1 We can start with the step definition, which would look something like this:
Given("{person} is located/standing {int} metre(s) from Sean") do |person, distance|
person.move_to(distance)
end
🎬 2
If we run Cucumber at this point we’ll see an error, because we haven’t defined the {person}
parameter type yet.
$ bundle exec cucumber Undefined parameter type {person} (Cucumber::CucumberExpressions::UndefinedParameterTypeError)
Here’s how we define one.
require 'shouty'
We use the ParameterType
DSL method from Cucumber to define our new parameter type. 🎬 5
We need to give it a name
which is the name we’ll use inside the curly brackets in our step definition expressions. 🎬 6
We also need to define — gasp! — a regular expression. 🎬 7 This is necessary to tell cucumber expressions what text to match when searching for this parameter in a Gherkin step. We won’t go into the details of regular expressions in this video, but in this case we’re just matching on either of the names of the people we’re using in our scenario. 🎬 8
Finally, there’s a transformer block, 🎬 9 which takes the text captured from the Gherkin step that matched the regular expression pattern, and runs some code. The return value of this block is what will be passed to the step definition. In this case, the block is passed the name of the person (as a string) 🎬 10 which we can then pass to the Person
class’s constructor. 🎬 11
require 'shouty'
ParameterType(
name: 'person',
regexp: /Lucy|Sean/,
transformer: -> (name) { Shouty::Person.new(name) }
)
🎬 12
All of this means that when we run our step, we’ll be passed an instance of Person
into our step definition automatically.
$ bundle exec cucumber
Feature: Hear shout
Scenario: Listener is within range # features/hear_shout.feature:2
Given Lucy is standing 1 metre from Sean # features/step_definitions/steps.rb:3
When Sean shouts "free bagels at Sean's" # features/step_definitions/steps.rb:8
Then Lucy hears Sean's message # features/step_definitions/steps.rb:14
1 scenario (1 passed)
3 steps (3 passed)
Custom parameters allow you to bring your domain model - the names of the classes and objects in your solution - and your domain language - the words you use in your scenarios and step definitions - closer together.
3.5.1. Lesson 5 - Questions (Java)
What role do Regular Expressions play in Cucumber Expressions?
-
None
-
Cucumber Expressions provide a subset of Regular Expression syntax
-
Cucumber Expressions are exactly the same as Regular Expressions
-
A Regular Expression is used to define the text to be matched when using a custom Parameter Type - TRUE
Answer: We use a Regular Expression to specify the text that should be matched when a custom Parameter Type is used in a Cucumber Expression.
How would you use the custom Parameter Type defined by the following code?
@ParameterType("activated") public Status state(String activationState) { return new Status(activationState); }
-
{activated}
-
{activationState}
-
{state} - TRUE
-
{Status}
Answer: The name of a custom Parameter Type is defined by the name of the method that is decorated with the @ParameterType
annotation.
3.5.2. Lesson 5 - Questions (Javascript)
What role do Regular Expressions play in Cucumber Expressions?
-
None
-
Cucumber Expressions provide a subset of Regular Expression syntax
-
Cucumber Expressions are exactly the same as Regular Expressions
-
A Regular Expression is used to define the text to be matched when using a custom Parameter Type - TRUE
Answer: We use a Regular Expression to specify the text that should be matched when a custom Parameter Type is used in a Cucumber Expression.
How would you use the custom Parameter Type defined by the following code?
defineParameterType({ name: 'state', regexp: /activated/, transformer: activationState ⇒ new Status(activationState) })
-
{activated}
-
{activationState}
-
{state} - TRUE
-
{Status}
Answer: The name of a custom Parameter Type is defined by the name
parameter passed to the defineParameterType
method.
3.5.3. Lesson 5 - Questions (Ruby)
What role do Regular Expressions play in Cucumber Expressions?
-
None
-
Cucumber Expressions provide a subset of Regular Expression syntax
-
Cucumber Expressions are exactly the same as Regular Expressions
-
A Regular Expression is used to define the text to be matched when using a custom Parameter Type - TRUE
Answer: We use a Regular Expression to specify the text that should be matched when a custom Parameter Type is used in a Cucumber Expression.
How would you use the custom Parameter Type defined by the following code?
ParameterType( name: 'state', regexp: /activated/, transformer: → (activationState) { Status.new(activationState) } )
-
{activated}
-
{activationState}
-
{state} - TRUE
-
{Status}
Answer: The name of a custom Parameter Type is defined by the name
parameter passed to the ParameterType
method.
3.5.4. Lesson 5 - Questions (SpecFlow/C#/Dotnet)
What role do Regular Expressions play in Cucumber Expressions?
-
None
-
Cucumber Expressions provide a subset of Regular Expression syntax
-
Cucumber Expressions are exactly the same as Regular Expressions
-
A Regular Expression is used to restrict the text to be matched when using a custom parameter type (StepArgumentTransformation) - TRUE
Answer: We use a Regular Expression to restrict the text that should be matched when a custom parameter type (StepArgumentTransformation) is used in a Cucumber Expression. You can find more examples of how to use StepArgumentTransformation
in the SpecFlow documentation.
How would you use the custom Parameter Type defined by the following code?
public Status ConvertState(string activationState) { return new Status(activationState); }
-
{activated} or {deactivated}
-
{activationState}
-
{Status} - TRUE
-
{ConvertState}
Answer: The name of a custom Parameter Type is defined by the name of the return type in the method that is decorated with the [StepArgumentTransformation]
annotation.
4. Cleaning up
4.1. The importance of readability
In the previous chapter, we talked about the importance of having readable scenarios, and you learned some new skills with Cucumber Expressions to help you achieve that goal. Those skills will give you the confidence to write scenarios exactly the way you want, knowing you’ll be able to match the Gherkin steps easily from your step definition code.
We emphasise readability because from our experience, writing Gherkin scenarios is a software design activity. Cucumber was created to bridge the communication gap between business domain experts and development teams. When you collaborate with domain experts to describe behaviour in Gherkin, you’re expressing the group’s shared understanding of the problem you need to solve. The words you use in your scenarios can have a deep impact on the way the software is designed, as we’ll see in later chapters.
The more fluent you become in writing Gherkin, the more useful a tool it becomes to help you facilitate this communication. Keeping your scenarios readable means you can get feedback at any time about whether you’re building the right thing. Over time, your features become living documentation about your system. We can’t emphasize enough how important it is to see your scenarios as more than just tests.
Maintaining a living document works both ways: the scenarios will guide your solution design, but you may also have to update your Gherkin to reflect the things you learn as you build the solution. This dance back and forth between features and solution code is an important part of BDD.
In this chapter, we’ll learn about feature descriptions, the Background keyword, and about keeping scenarios and code up-to-date with your current understanding of the project.
First, let’s catch up with what’s been happening on the Shouty project.
4.1.1. Continuity Annoucement
Before we start, I need to explain about a continuity error between the previous chapter and this next one.
In the last chapter we showed you how to use parameter types to automatically create an instance of our Person
class whenever we used it in a step defintion.
Now the first version of video series was first created many years ago, before we had added parameter types to Cucumber. Although we updated the previous chapter to demonstrate parameter types to you, we haven’t yet updated this one. So you’ll notice as you follow along here that there’s no mention of parameter types anymore.
Some of the things we’ll be doing to clean up the code in this chapter would be even cleaner if we used parameter types, and we hope to update this video someday to incorporate them into the story. In the meantime we’ll leave it as an exercise for you to think about how you would change the work we do in this episode to make the most of them.
Have fun, and don’t forget to come on the #school community Slack channel to ask if you need any guidance!
4.1.2. Lesson 1 - Questions (Ruby, Java, JS)
Which aspects of Cucumber help bridge the communication gap between business domain experts and development teams?
-
The readability of Gherkin scenarios - TRUE
-
Cucumber’s availability for different programming languages - FALSE
-
Being able to express scenarios using your own domain language - TRUE
Answer: The feature files that Cucumber understands are written using Gherkin, so that you can create scenarios that utilise your own domain language, so that they can be read and understood by everyone involved in specifying and delivering your software.
How do Cucumber feature files differ from more traditional automated tests?
-
The purpose of feature files is to create readable specifications that can be understood by the whole team, not to provide test coverage
-
Business-readable specifications make it easier to obtain feedback about what you’re building while you’re building it, rather than waiting for a later test cycle
-
Feature files become "living documentation" when they are automated, providing a single source of truth for the whole team
-
Feature files should be written collaboratively by business and delivery, not in isolation by testers
-
There is no difference - FALSE
Answer: BDD is the collaborative approach to developing software that Cucumber was created to support. Although Cucumber scenarios do act as tests when they are automated, this is not their primary purpose. Their primary purpose is to provide a single, shared specification, written in the domain language of your business — facilitating collaboration, feedback, and reliable documentation. The primary purpose of traditional automated tests, on the other hand, is to check that the software behaves as expected.
4.1.3. Lesson 1 - Questions (SpecFlow/C#/Dotnet)
Which aspects of SpecFlow help bridge the communication gap between business domain experts and development teams?
-
The readability of Gherkin scenarios - TRUE
-
Gherkin scenarios can be automated in different programming languages - FALSE
-
Being able to express scenarios using your own domain language - TRUE
Answer: The feature files that SpecFlow understands are written using Gherkin, so that you can create scenarios that utilise your own domain language, so that they can be read and understood by everyone involved in specifying and delivering your software.
How do SpecFlow feature files differ from more traditional automated tests?
-
The purpose of feature files is to create readable specifications that can be understood by the whole team, not to provide test coverage
-
Business-readable specifications make it easier to obtain feedback about what you’re building while you’re building it, rather than waiting for a later test cycle
-
Feature files become "living documentation" when they are automated, providing a single source of truth for the whole team
-
Feature files should be written collaboratively by business and delivery, not in isolation by testers
-
There is no difference - FALSE
Answer: BDD is the collaborative approach to developing software that SpecFlow was created to support. Although SpecFlow scenarios do act as tests when they are automated, this is not their primary purpose. Their primary purpose is to provide a single, shared specification, written in the domain language of your business — facilitating collaboration, feedback, and reliable documentation. The primary purpose of traditional automated tests, on the other hand, is to check that the software behaves as expected.
4.2. Review changes that happened while we were away
While we were away, the developers of Shouty have been busy working on the code. Let’s have a look at what they’ve been up to.
🎬 1: bundle exec cucumber We’ll start out by running Cucumber.
Great! It looks like both these scenarios are working now - both the different messages that Sean shouts are being heard by Lucy.
🎬 2: open stepdef file Let’s dig into the code and see how these steps have been automated.
require 'shouty'
Given("Lucy is {int} metres from Sean") do |distance|
@network = Shouty::Network.new
@lucy = Shouty::Person.new(@network)
@sean = Shouty::Person.new(@network)
@lucy.move_to(distance)
end
When("Sean shouts {string}") do |message|
@sean.shout(message)
@message_from_sean = message
end
Then("Lucy should hear Sean's message") do
expect(@lucy.messages_heard).to eq [@message_from_sean]
end
In the step definition layer, we can see that a new class has been defined, the Network.🎬 3 We’re creating an instance of the network here. Then we pass that network instance to each of the Person instances we create here.🎬 4 So both instances of Person depend on the same instance of network. The Network is what allows people to send messages to one another.
There are also a couple of new unit test classes in the spec
directory,🎬 5 one for the Network class,🎬 6 and another one for the Person class.🎬 7
Unit tests are fine-grained tests that define the precise behaviour of each of those classes. We’ll talk more about this in a future lesson, but feel free to have a poke around in there in the meantime.
The rspec
command will run the unit tests. 🎬 8: run rspec
The first thing I notice coming back to the code is that the feature file is still talking about the distance between Lucy and Sean, but we haven’t actually implemented any behaviour around that yet.
Feature: Shout
Scenario: Listener within range
Given Lucy is 15 metres from Sean
When Sean shouts "free bagels at Sean's"
Then Lucy should hear Sean's message
Scenario: Listener hears a different mesage
Given Lucy is 15 metres from Sean
When Sean shouts "Free coffee!"
Then Lucy should hear Sean's message
This happens to us all the time - we have an idea for a new feature, but then we find the problem is too complex to solve all at once, so we break it down into simpler steps. If we’re not careful, little bits of that original idea can be left around like clutter, in the scenarios and in the code. That clutter can get in the way, especially if plans change.
We’re definitely going to develop this behaviour, but we’ve decided to defer it to our next iteration. Our current solution is just focussed on broadcasting messages between the people on the network.
Let’s clean up the feature to reflect that current understanding.
4.2.1. Lesson 2 - Questions (Ruby, Java, JS)
Why have the Shouty developers created unit tests for the Person and Network classes?
-
They don’t understand how to do BDD - FALSE
-
Unit tests are fine-grained tests that define the precise behaviour of each of those classes - TRUE
-
Unit tests run faster than Cucumber scenarios - FALSE
Answer: Unit tests (also known as programmer tests) are used to define precise behaviour of units of code that may not be interesting to the business — and so should not be written in a feature file. Writing unit tests is entirely compatible with BDD.
There is no reason for Cucumber scenarios to run significantly slower than unit tests. The Shouty step definitions that we’ve seen so far interact directly with the domain layer and run extremely fast.
Why is the distance between Sean and Lucy not being used by Shouty?
-
The team has decided to defer implementing range functionality until a later iteration - TRUE
-
The developers have misunderstood the specification
-
The specification has changed since the scenarios were written
-
The distance between Sean and Lucy is being used to decide if the shout is "in range"
Answer: Teams often find that the problem is too big to solve all at once, so we split it into thinner slices. Working in smaller steps is a faster, safer way of delivering software. In this case the team has decided that broadcasting messages and calculating if a person is in-range are different problems that they will address separately.
4.2.2. Lesson 2 - Questions (SpecFlow)
Why have the Shouty developers created unit tests for the Person and Network classes?
-
They don’t understand how to do BDD - FALSE
-
Unit tests are fine-grained tests that define the precise behaviour of each of those classes - TRUE
-
Unit tests run faster than Cucumber scenarios - FALSE
Answer: Unit tests (also known as programmer tests) are used to define precise behaviour of units of code that may not be interesting to the business — and so should not be written in a feature file. Writing unit tests is entirely compatible with BDD.
There is no reason for SpecFlow scenarios to run significantly slower than unit tests. The Shouty step definitions that we’ve seen so far interact directly with the domain layer and run extremely fast.
Why is the distance between Sean and Lucy not being used by Shouty?
-
The team has decided to defer implementing range functionality until a later iteration - TRUE
-
The developers have misunderstood the specification
-
The specification has changed since the scenarios were written
-
The distance between Sean and Lucy is being used to decide if the shout is "in range"
Answer: Teams often find that the problem is too big to solve all at once, so we split it into thinner slices. Working in smaller steps is a faster, safer way of delivering software. In this case the team has decided that broadcasting messages and calculating if a person is in-range are different problems that they will address separately.
4.3. Description field
After the feature keyword, we have space in a Gherkin document to write any arbitrary text that we like.🎬 1: feature file We call this the feature’s description. This is a great place to write up any notes or other details that can’t easily be expressed in examples. You might have links to wiki pages or issue trackers, or to wireframes. You can put anything you like in here, as long as you don’t start a line with a Gherkin keyword, like “Rule:” or “Scenario:”.
In this case, we can add a high level description of the Shouty application.🎬 2 Because Shouty doesn’t yet filter by proximity, we can also write a todo list here so it’s clear that we do intend to get to that soon.🎬 3
Feature: Shout
Shouty allows users to "hear" other users "shouts" as long as they are close enough to each other.
To do:
- only shout to people within a certain distance
Changing the description doesn’t change anything about how Cucumber will run this feature. It just helps the human beings reading this document to understand more about the system you’re building.
Our two scenarios are examples of how Shouty can broadcast a shout to other users. This is one of the main business rules, which we can document using the Rules keyword.🎬 4: add Rules keyword and indent scenarios We’ll learn more about this in a later chapter.
Rule: Shouts can be heard by other users
Scenario: Listener within range
Given Lucy is 15 metres from Sean
When Sean shouts "free bagels at Sean's"
Then Lucy should hear Sean's message
Scenario: Listener hears a different mesage
Given Lucy is 15 metres from Sean
When Sean shouts "Free coffee!"
Then Lucy should hear Sean's message
🎬 5: shows 'Lucy is 15 metres from Sean' step definition The step “Given Lucy is 15 metres from Sean” is misleading, since the distance between the two people is not yet relevant in our current model.
Rule: Shouts can be heard by other users
Scenario: Listener within range
Given Lucy is 15 metres from Sean
When Sean shouts "free bagels at Sean's"
Then Lucy should hear Sean's message
Scenario: Listener hears a different mesage
Given Lucy is 15 metres from Sean
When Sean shouts "Free coffee!"
Then Lucy should hear Sean's message
The step definition calls the moveTo method on Person, 🎬 6
🎬 7: shows move_to method in Person class but the moveTo method doesn’t actually do anything.
def move_to(distance)
end
🎬 8 Let’s simplify this code to do just what it needs to do right now, and no more. We can start from the scenario by changing this single step to express what’s actually going on.🎬 9 We’ll work on one scenario at a time, and update the other one once we’re happy with this one.
Rule: Shouts can be heard by other users
Scenario: Listener hears a message
Given a person named Lucy
And a person named Sean
When Sean shouts "free bagels at Sean's"
Then Lucy should hear Sean's message
Scenario: Listener hears a different mesage
Given Lucy is 15 metres from Sean
When Sean shouts "Free coffee!"
Then Lucy should hear Sean's message
Now the scenario names make sense, and we have two steps, each creating a person. Notice we’re starting to reveal some more of our domain language here: we’ve introduced the words Person
and name
. Person is already a part of our domain language, so it’s nice to have that revealed in the language of the scenario. Name may well become an attribute of our person soon, so it’s also useful to have that surfaced so we can get feedback about it from the team.
One thing we’ve lost by doing this is the idea that, eventually, the two people will need to be close to each other for the message to be transmitted. We definitely wouldn’t remove detail like that unilaterally, without discussing it with the other people who were involved in writing and reviewing this scenario.
🎬 10: create new Rule and writes out two new empty scenarios for in / out of range In this case, as well as adding it to the TODO list above, we’ve decided to document the range rule, and write a couple of new empty scenarios to remind us to implement that behaviour later.
Rule: Shouts should only be heard if listener is within range
Scenario: Listener is within range
Scenario: Listener is out of range
🎬 11: bundle exec cucumber Let’s press on. We can run Cucumber to generate new step definition snippets for the new steps and paste them into our steps file. 🎬 12
Given('a person named Lucy') do
pending # Write code here that turns the phrase above into concrete actions
end
Given('a person named Sean') do
pending # Write code here that turns the phrase above into concrete actions
In the next lesson we’ll look at a couple of ways that we can implement these new step definitions.
4.3.1. Lesson 3 - Questions
What is a feature file description?
-
Any lines of text between the feature name and the first rule or scenario - TRUE
-
A line of text that starts with the
#
character -
A block of text introduced by the
Description:
keyword
Answer: You can add a free text description of a feature file after the Feature:
line that defines the feature’s name. The description can be any number of lines long. The description continues until the first rule, scenario, or scenario outline is encountered.
What is the purpose of writing an empty scenario?
-
It is not valid Gherkin syntax to write an empty scenario
-
Empty scenarios act as a reminder that we have more work to do - TRUE
-
Empty scenarios are a way of pretending that we have done more work than we actually have
Answer: Cucumber treats empty scenarios as work that needs to be done and reports them as pending.
4.4. The "Before" hook
🎬 1: broken implementation We now have two step definitions to implement, and that presents us with a bit of a problem. We need the same instance of Network available in both. We could just assume that the Lucy step will always run first and create it there, but that seems fragile. If someone wrote a new scenario that didn’t create people in the right order, they’d end up with no Network instance, and weird bugs. We want our steps to be as independent as possible, so they can be easily composed into new scenarios.
Given('a person named Lucy') do
@lucy = Shouty::Person.new(@network)
end
Given('a person named Sean') do
@sean = Shouty::Person.new(@network)
The most straightforward way to create this network instance in Ruby is to use a hook.
We need an instance of Network in every scenario, so we can declare a Before Hook that creates one before each scenario starts, like this:🎬 2 Now we can use that Network instance as we create Lucy and Sean in these two new steps.🎬 3
Before do
@network = Shouty::Network.new
end
Given('a person named Lucy') do
@lucy = Shouty::Person.new(@network)
end
Given('a person named Sean') do
@sean = Shouty::Person.new(@network)
🎬 4: bundle exec cucumber It should be working again now. Let’s run Cucumber to check.
🎬 5: Copy and paste in feature file Good. Let’s do the same with the other scenario.
Scenario: Listener hears a different mesage
Given a person named Lucy
And a person named Sean
When Sean shouts "Free coffee!"
Then Lucy should hear Sean's message
🎬 7: Delete lucy_is_located_m_from_Sean Now we can remove this old step definition. We know we’ll need something like this in the future when we implement the proximity rule, but we don’t want to second-guess what that code will look like, so let’s clean it out for now.
require 'shouty'
Before do
@network = Shouty::Network.new
end
Given("Lucy is {int} metres from Sean") do |distance|
@network = Shouty::Network.new
@lucy = Shouty::Person.new(@network)
@sean = Shouty::Person.new(@network)
@lucy.move_to(distance)
end
Given('a person named Lucy') do
@lucy = Shouty::Person.new(@network)
end
Given('a person named Sean') do
@sean = Shouty::Person.new(@network)
end
When("Sean shouts {string}") do |message|
@sean.shout(message)
@message_from_sean = message
end
Then("Lucy should hear Sean's message") do
expect(@lucy.messages_heard).to eq [@message_from_sean]
end
🎬 9: delete moveTo Now we have one last bit of dead code left, the move_to method on Person. Let’s clean that up too.
module Shouty
class Person
attr_reader :messages_heard
def initialize(network)
@messages_heard = []
@network = network
@network.subscribe(self)
end
def shout(message)
@network.broadcast(message);
end
def hear(message)
@messages_heard << message
end
end
class Network
def initialize
@listeners = []
end
def subscribe(person)
@listeners << person
end
def broadcast(message)
@listeners.each do |listener|
listener.hear(message)
end
end
end
end
4.4.1. Lesson 4 - Questions
When does a Before hook run?
-
Before every run of Cucumber
-
Before the first scenario in each feature file
-
Before each scenario - TRUE
-
Before each step in a scenario
Answer: A Before hook runs before each scenario. Since there is no way to tell if a hook exists by looking at the feature file, you should only use hooks for performing actions that you don’t expect the business to provide feedback on.
You can read more about hooks at https://cucumber.io/docs/cucumber/api/#hooks
Why isn’t it a good idea to create a Network instance in the same step definition where we create Lucy?
-
It is a good idea
-
Steps should be independent and composable. If the Network is only created when Lucy is created, future scenarios will be forced to create Lucy - TRUE
-
We’ll need to create another Network instance when we create Sean
Answer: Every person needs to share the same Network instance, which means we need to create the Network before we create any people. By creating the Network instance in the same step definition that we create Lucy, we are forcing people to: * create Lucy — even if the scenario doesn’t need Lucy * create Lucy before any other person — because otherwise Network will not have been created yet
4.5. Create Person in a generic stepdef
OK, so we’ve cleaned things up a bit, to bring the scenarios, the code and our current understanding of the problem all into sync. What’s nice to see is how well those new steps that create Lucy and Sean match the code inside the step definition.
When step definitions have to make a big leap to translate between our plain-language description of the domain in the Gherkin scenario, and the code, that’s usually a sign that something is wrong. We like to see step definitions that are only one or two lines long, because that usually indicates our scenarios are doing a good job of reflecting the domain model in the code, and vice-versa.
One problem that we still have with these scenarios is that we’re very fixed to only being able to use these two characters, Lucy and Sean. If we want to introduce anyone else into the scenario, we’re going to be creating quite a lot of duplicate code. In fact, the two steps for creating Lucy and Sean are almost identical, apart from those instance variables.
On a real project we wouldn’t bother about such a tiny amount of duplication at this early stage, but this isn’t a real project! Let’s play with the skills we learned in the last chapter to make a single step definition that can create Lucy or Sean.
The first problem we’ll need to tackle is these hard-coded instance variable names.🎬 2
We can use a Hash to store all the people involved in the scenario.
Let’s try replacing Lucy first.
🎬 3: Edit Stepdefs.java, adding private instance field people, creating in @Before hook, and using for Lucy throughout We’ll start by creating a new Hash in the before hook, like this.
Before do
@network = Shouty::Network.new
@people = {}
end
Now we can store Lucy in the hash. We’ll use her name as the key, hard-coding it for now.🎬 4
Given('a person named Lucy') do
@people['Lucy'] = Shouty::Person.new(@network)
end
Finally, where we check Lucy’s messages heard here in the assertion, we need to fetch her out of the hash. 🎬 5
expect(@people['Lucy'].messages_heard).to eq [@message_from_sean]
end
With that little refactoring done, we can now try and make this first step generic for any name.
Using your new found Cucumber expression skills from the last chapter, you’ll know that if we replace the word Lucy here with a 'word' expression,🎬 7 we’ll have the name passed into our step definition as an argument, here.🎬 8 Now we can use that as the key in the hash.🎬 9
Given('a person named {word}') do |name|
@people[name] = Shouty::Person.new(@network)
end
🎬 10: bundle exec cucumber FAILS If we try and run Cucumber now, we get an error about an ambiguous match. Our generic step definition is now matching the step “a person named Sean”, but so is the original one. In bigger projects, this can be a real issue, so this warning is important.
Let’s remove the old step definition,🎬 11 and fetch Sean from the hash here where he shouts his message.🎬 12
When("Sean shouts {string}") do |message|
@people['Sean'].shout(message)
@message_from_sean = message
end
Great, we’re green again.
4.5.1. Lesson 5 - Questions (Ruby, Java, JS)
Why should a step definition be short?
-
Because the plain-language description of the domain in the Gherkin step should be close to the domain model in the code - TRUE
-
Step definitions don’t need to be short
-
Cucumber limits the length of step definitions to five lines of code
Answer: Step definitions are a thin glue between the plain-language description in a scenario and the software that we’re building. If the business domain and the solution domain are aligned, then there should be little translation to do in the step definition.
What does it mean when Cucumber complains about an ambiguous step?
-
Cucumber couldn’t find a step definition that matches a step
-
Cucumber only found one step definition that matches a step
-
Cucumber found more than one step definition that matches a step - TRUE
Answer: If more than one step definition matches a step, then Cucumber doesn’t know which one to call. When this ambiguity occurs, Cucumber issues an error, rather than try to choose between the matching step definitions.
4.6. Backgrounds
🎬 1: Feature file Let’s switch back to the feature to show you one more technique for improving the readability of your scenarios.
Rule: Shouts can be heard by other users
Scenario: Listener hears a message
Given a person named Lucy
And a person named Sean
When Sean shouts "free bagels at Sean's"
Then Lucy should hear Sean's message
Scenario: Listener hears a different mesage
Given a person named Lucy
And a person named Sean
When Sean shouts "Free coffee!"
Then Lucy should hear Sean's message
When we have common context steps - the Givens - in all the scenarios in our feature, it can sometimes be useful to get those out of the way.
🎬 2: move Given steps into background We can literally move them into the background, using a background keyword, like this:
Background:
Given a person named Lucy
And a person named Sean
🎬 3: bundle exec cucumber As far as Cucumber is concerned, these scenarios haven’t changed. It will still create both Lucy and Sean as the first things it does when running each of these scenarios.
🎬 4: Feature file, showing Background and shortened scenarios But from a readability point of view, we can now see more clearly what’s important and interesting about these two scenarios - in this case, the message being shouted.
Rule: Shouts can be heard by other users
Scenario: Listener hears a message
When Sean shouts "free bagels at Sean's"
Then Lucy should hear Sean's message
Scenario: Listener hears a different mesage
When Sean shouts "Free coffee!"
Then Lucy should hear Sean's message
Notice we just went straight into When steps in our scenarios.🎬 5 That’s absolutely fine. We still have a context for the scenario, but we’ve chosen to push it off into the background.
Again, it’s debatable whether we’d bother to use a Background to do this on a real project, but this at least illustrates the technique. We rarely use Backgrounds in our projects, because although they can improve readability by removing the duplication of repeated contexts, they also harm readability by requiring people to read the Background in conjunction with each Scenario.
To maintain trust in the BDD process, it’s important to keep your features fresh. Even when you drive the development from BDD scenarios, you’ll still learn lessons from the implementation that might need to be fed back into your Gherkin documentation.
In this case, we discovered that we could find a smaller slice of this story, and defer the business rule about proximity until our next iteration. Splitting stories like this is a powerful agile technique, and one that BDD can help you to master. Now we have a clean codebase and a suite of scenarios that reflects the current state of the system’s development.
We’re ready to start the next iteration.
4.6.1. Lesson 6 - Questions (Ruby, Java, JS)
What does the Gherkin keyword Background do?
-
It provides a place to write a description of why the feature is valuable
-
It is treated exactly like a scenario, but is run as soon as Cucumber starts
-
It is treated exactly like a scenario, but is run once before any other scenario in the feature file
-
The steps from the background are run as if they were inserted at the beginning of every scenario in the feature file - TRUE
Answer: The background is used to reduce duplication in scenarios by moving steps that are common to all scenarios into a single location. The steps in the background are run before every scenario in the feature file.
There can be a maximum of one Background per feature file. A Background only affects scenarios that are in the same feature file as the Background.
How might Backgrounds decrease the readability or maintainability of a feature file?
-
Backgrounds always improve readability
-
Readability can decrease because the reader must remember the contents of the background even when reading scenarios at the end of the feature file
-
Maintainability can decrease because the maintainer must be aware that there is a background even when adding scenarios to the end of the feature file
-
Maintainability can decrease because the maintainer must be aware of the background when moving a scenario to a different feature file
Answer: Backgrounds were created to aid readability, by reducing duplication in the scenarios. Unfortunately, moving important information out of a scenario means that anyone reading or modifying a feature file must be fully aware that of the existence and content of a background. Since feature files typically contain several scenarios, that means holding two sections of the feature file in your mind at the same time, making a feature file harder to read or maintain.
5. Loops
5.1. Removing redundant scenarios
Welcome back to Cucumber School.
Feature: Shout
Shouty allows users to "hear" other users "shouts" as long as they are close enough to each other.
To do:
- only shout to people within a certain distance
Rule: Shouts can be heard by other users
Scenario: Listener hears a message
Given a person named Lucy
And a person named Sean
When Sean shouts "free bagels at Sean's"
Then Lucy should hear Sean's message
Scenario: Listener hears a different mesage
Given a person named Lucy
And a person named Sean
When Sean shouts "Free coffee!"
Then Lucy should hear Sean's message
Rule: Shouts should only be heard if listener is within range
Scenario: Listener is within range
Scenario: Listener is out of range
Last time we worked on cleaning up the Shouty features to keep them in sync with the current status of the project. We stripped the scenarios back to only specify the behaviour of passing messages between people. We made it clear that the proximity rule had not yet been implemented.
You’ll already remember from the Cucumber expressions chapter how important it is to be expressive in your scenarios, and keep them readable. In this chapter we’re going to learn some new tricks with Gherkin that will give you even more flexibility about how you write scenarios.
Once again the Shouty developers — have been hard at work implementing that proximity rule. Let’s have a look at how they got on. 🎬 2: Show chapter 5 initial checkin feature file
Right, so those two scenarios we just left as placeholders: the one where the listener is within range,🎬 3: show in feature file and the one where the listener is out of range 🎬 4: show in feature file are passing. 🎬 5: Show output from Cucumber showing the in-range/out-of-range results Fantastic! If we look at our step definitions, we can see how they have been implemented. 🎬 6: show the in-range/out-of-range scenarios in the step definitions
Scenario: Listener is within range
Given the range is 100
And a person named Sean is located at 0
And a person named Lucy is located at 50
When Sean shouts "free bagels at Sean's"
Then Lucy should hear Sean's message
Scenario: Listener is out of range
Given the range is 100
And a person named Sean is located at 0
And a person named Larry is located at 150
When Sean shouts "free bagels at Sean's"
Then Larry should not hear Sean's message
Let’s review the changes to the feature file in more detail.
We now have four scenarios:🎬 7 our original two from the last time we looked at the code,🎬 8 and the two placeholders we wrote as reminders.🎬 9
Feature: Hear shout
Shouty allows users to "hear" other users "shouts" as long as they are close enough to each other.
Rule: Shouts can be heard by other users
Scenario: Listener hears a message
Given the range is 100
And a person named Sean is located at 0
And a person named Lucy is located at 50
When Sean shouts "free bagels at Sean's"
Then Lucy should hear Sean's message
Scenario: Listener hears a different message
Given the range is 100
And a person named Sean is located at 0
And a person named Lucy is located at 50
When Sean shouts "Free coffee!"
Then Lucy should hear Sean's message
Rule: Shouts should only be heard if listener is within range
Scenario: Listener is within range
Given the range is 100
And a person named Sean is located at 0
And a person named Lucy is located at 50
When Sean shouts "free bagels at Sean's"
Then Lucy should hear Sean's message
Scenario: Listener is out of range
Given the range is 100
And a person named Sean is located at 0
And a person named Larry is located at 150
When Sean shouts "free bagels at Sean's"
Then Larry should not hear Sean's message
We used the second scenario - Listener hears a different message 🎬 10: Highlight second scenario - to triangulate and force us to replace the hard-coded message output with a proper implementation. Now we have a domain model that uses a variable for the message, there’s an insignificant chance of this behaviour regressing, so we can safely remove the second scenario. 🎬 11: removes second scenario: Listener hears a different message
Scenario: Listener hears a different message
Given the range is 100
And a person named Sean is located at 0
And a person named Lucy is located at 50
When Sean shouts "Free coffee!"
Then Lucy should hear Sean's message
Keeping excess scenarios is wasteful: they clutter up your feature files, distracting your readers. When you run your features as tests, excess scenarios make them take longer to run than necessary. The one where a "listener hears a "message" is a perfectly good way of checking that the message has been sent correctly.
5.1.1. Lesson 1 - Questions
Why was it a good idea to delete the scenario?
-
It doesn’t help illustrate the rule "Shouts can be heard by other users" — TRUE
-
No one should give away free coffee
-
There should only be one scenario per rule
Explanation: We created the scenario "Listener hears a different message" to force us to replace our hard-coded implementation. Now we have a domain model that uses a variable for the message, there’s an insignificant chance of this behaviour regressing, so we can safely remove the second scenario.
Keeping excess scenarios is wasteful: they clutter up your feature files and slow down feedback.
5.2. Incidental details
The first scenario has changed since we last looked at it 🎬 1: hear_shout.feature - it now specifies the range of a shout and the location of Sean and Lucy.🎬 2: Highlight the first three line of the first scenario This scenario exists to illustrate that a listener hears the message exactly as the shouter shouted it. All the additional details are incidental and make the scenario harder to read.
Scenario: Listener hears a message
Given the range is 100
And a person named Sean is located at 0
And a person named Lucy is located at 50
When Sean shouts "free bagels at Sean's"
Then Lucy should hear Sean's message
🎬 3: Edit context of first scenario Let’s ensure that this scenario includes only essential information for the reader and remove all references to location and range.
Scenario: Listener hears a message
Given a person named Sean
And a person named Lucy
When Sean shouts "free bagels at Sean's"
Then Lucy should hear Sean's message
🎬 4: Add Network creation We’ll need to make changes to the step definitions to make sure that a Network class is always created - which we can do using an instance variable.
@network = Shouty::Network.new(DEFAULT_RANGE)
🎬 5: Highlight DEFAULT_RANGE We’ve defaulted the range to 100. If a scenario needs to document specific range, that can still be done by explicitly including a "Given the range is …" step.
DEFAULT_RANGE = 100
We’ll also need to add a step definition that can create a person without the scenario needing to specify where they are located.🎬 6: Create new step definition The step definition gives each person 🎬 7 created this way a default location of 0.🎬 8
Given "a person named {word}" do |name|
@people[name] = Shouty::Person.new(@network, 0)
end
🎬 9: run Cucumber Let’s run Cucumber to check we haven’t broken anything… and we’re good!
🎬 10 Looking at the two new scenarios - Listener is within range 🎬 11 & Listener is out of range 🎬 12 - we can see that they also contain incidental details. Since their purpose is to illustrate the "Shouts should only be heard if listener is within range" rule, there’s no need to actually document the content of the shout. 🎬 13: "highlight free bagels at seans
Scenario: Listener is within range
Given the range is 100
And a person named Sean is located at 0
And a person named Lucy is located at 50
When Sean shouts "free bagels at Sean's"
Then Lucy should hear Sean's message
Scenario: Listener is out of range
Given the range is 100
And a person named Sean is located at 0
And a person named Larry is located at 150
When Sean shouts "free bagels at Sean's"
Then Larry should not hear Sean's message
🎬 14: Delete message text Let’s remove the details that aren’t relevant to the range rule.
Scenario: Listener is within range
Given the range is 100
And a person named Sean is located at 0
And a person named Lucy is located at 50
When Sean shouts
Then Lucy should hear a shout
Scenario: Listener is out of range
Given the range is 100
And a person named Sean is located at 0
And a person named Larry is located at 150
When Sean shouts
Then Larry should not hear a shout
🎬 15: Add new step definition Next we add a step definition that allows Sean to shout, without needing us to specify the exact message.
When "Sean shouts" do
@people["Sean"].shout("Hello, world")
end
🎬 16 One that allows us to check that Lucy has heard exactly one shout - because she’s in range of the shouter.
Then "Lucy should hear a shout" do
expect(@people['Lucy'].messages_heard.count).to eq 1
end
🎬 17 And one that allows us to check that Larry hasn’t heard any messages at all - because he’s out-of-range.
Then "Larry should not hear a shout" do
expect(@people['Larry'].messages_heard.count).to eq 0
end
🎬 18: Run Cucumber And finally run Cucumber - and we’re still green.
🎬 19: Show hear_shout.feature That’s better. We’ve removed inessential details, so that each scenario contains only the information needed to illustrate its business rule.
The scenarios would still run green if we removed the steps that set the range of a shout 🎬 20: Highlight in last two scenarios🎬 20b , because the range already has a default value. We’re not going to, because since those scenarios are illustrating the rule that deals with the range of a shout, it’s an essential part of context for anyone reading them.
A happy side-effect is that, in order to set the range from our scenario, we’ve had to make it a configurable property of the system 🎬 21: highlight Network constructor. So if our business stakeholders ever change their minds about the range, we won’t have to go hunting around in the code for where we’d hard-coded it.
5.2.1. Lesson 2 - Questions
Why did we remove any reference to range or location from the first scenario "Listener hears a message"?
-
They are essential to system behaviour
-
They are incidental to the rule being illustrated — TRUE
-
They were only needed to triangulate the implementation
-
They made the scenario too long
Explanation: The first scenario exists to illustrate the rule "Listener hears a message." Since the behaviour is not affected by the range of a shout, neither the range nor the distance between the shouter and the listener is relevant. The information is therefore incidental and should be omitted from the scenario.
Why do we need to know the names of people using Shouty?
-
It’s important that every person in the system has a real name
-
It’s necessary to use persona, where the shouter is called Sean and the listeners are called Lucy or Larry
-
It doesn’t matter what we call them — but the automation code does need to be able to tell them apart — TRUE
-
The automation code has been written to recognise the names Sean, Lucy, and Larry
Explanation: It’s necessary to be able to distinguish the people that are involved in the scenario. We have called them Sean, Lucy, and Larry, but we could have called them Shouter, Listener1, and Listener2 (or even User1, User2, and User3).
We find that using persona (where the name gives an indication of the person’s purpose in the scenario) can be a useful way of conveying information, without cluttering up the scenario. If the names conveyed no information at all, they would not contribute to the readability of the scenario, and could be considered incidental.
Which pieces of information are incidental in this scenario?
Rule: Offer is only valid for Shouty users Scenario: Customer is not a Shouty user Given Nora is not a Shouty user And Sean shouted "free bagels until midday" When Nora orders a bagel and a coffee at 11:00am Then she should be charged 75¢ for the bagel
-
Nora is not a Shouty user
-
Sean is offering "free bagels!"
-
Sean’s offer is only valid until midday — TRUE
-
Nora orders a bagel
-
Nora orders a coffee — TRUE
-
Nora places her order at 11:00am — TRUE
-
Nora gets charged for the bagel
-
Nora get charged 75¢ for the bagel — TRUE
Explanation: This scenario is illustrating the rule that the "offer is only valid for Shouty users". It’s therefore essential to know that Nora is not a Shouty user, because this means that she is not eligible for the offer.
We don’t need to know that Nora orders a coffee, because that has no relevance to the rule. Nor do we need to know when the offer expires, when Nora places the order, or how much she will be charged — there will be other rules (and other scenarios) that illustrate that behaviour.
Although it’s incidental that the offer is for bagels, it is necessary to illustrate that Nora has ordered the item that is on offer to Shouty users — and that she will be charged for that item. We use "bagels" as an example to make the scenario easier to read, not because there’s something inherently special about bagels!
5.3. Refactoring to Data Tables
Let’s look at the two scenarios that illustrate the rule about range again 🎬 1: show the range rule & scenarios. Notice how the steps that create the Sean,🎬 2 Lucy,🎬 3 and Larry 🎬 4 are very similar.
Rule: Shouts should only be heard if listener is within range
Scenario: Listener is within range
Given the range is 100
And a person named Sean is located at 0
And a person named Lucy is located at 50
When Sean shouts
Then Lucy should hear a shout
Scenario: Listener is out of range
Given the range is 100
And a person named Sean is located at 0
And a person named Larry is located at 150
When Sean shouts
Then Larry should not hear a shout
When we see steps like this, Gherkin’s Given When Then syntax starts to feel a bit clunky. Imagine if we could just write out a table, like this:
And people are located at
| name | location |
| Sean | 0 |
| Lucy | 50 |
Well, we’re in luck. You can!
Gherkin has a special syntax called Data Tables, that allows you to specify tabular data for a step, using pipe characters to mark the boundary between cells.🎬 6: highlight data table
Given "people are located at" do |table|
# table is a Cucumber::MultilineArgument::DataTable
pending # Write code here that turns the phrase above into concrete actions
end
As you can see, the step definition implicitly takes a single argument 🎬 9: highlight stepdef parameters, which as this comment explains is a DataTable.🎬 10: highlight comment This object has a rich API for using the tabular data.
At its most basic, the table is just a two-dimensional array. So, Lucy’s location can be accessed 🎬 11: Print out a single cell value by getting the value from array cell (2, 1)
Given "people are located at" do |table|
p table.raw[2, 1]
end
You can also turn the table into an array of hashes 🎬 13: Print out data table, where the first row is used for the hash keys,🎬 14: split IntelliJ screen horizontally, showing feature file at top and stepdefs below. Highlight first row and each following row is used for the hash values.🎬 15: highlight second and third rows
Given "people are located at" do |table|
p table.symbolic_hashes
end
Now we can easily iterate 🎬 17: write the loop over these hashes and turn them into instances of Person: 🎬 18
Given "people are located at" do |table|
table.symbolic_hashes.each do |name: , location: |
@people[name] = Shouty::Person.new(@network, location.to_i)
end
end
With that done, we can update the other scenario 🎬 20: update other scenario …
Scenario: Listener is out of range
Given the range is 100
And people are located at
| name | location |
| Sean | 0 |
| Larry | 150 |
When Sean shouts
Then Larry should not hear a shout
Now we can check that everything is still green. 🎬 21
-
and delete our old step definition 🎬 22: delete unused step def, which is now unused.
Given "a person named {word} is located at {int}" do |name, location|
@people[name] = Shouty::Person.new(@network, location)
end
Cucumber strips all the white space surrounding each cell 🎬 24: show lining up of pipe characters, so we can have a nice neat table in the Gherkin but still get clean values in the step definition underneath.
Notice we’ve still had to convert the location from a string to an integer 🎬 25: Conversion in step def, because Cucumber can’t know that’s the type of value in our table.
Given "people are located at" do |table|
table.symbolic_hashes.each do |name: , location: |
@people[name] = Shouty::Person.new(@network, location.to_i)
end
end
That looks much nicer - 🎬 26: hear_shout.feature data tables people positioned using a table in the feature file and 🎬 27: people_are_located_at() really clean code that creates and positions people according to the data.
Scenario: Listener is out of range
Given the range is 100
And people are located at
| name | location |
| Sean | 0 |
| Larry | 150 |
When Sean shouts
Then Larry should not hear a shout
5.3.1. Lesson 3 - Questions
What is the name of the Gherkin syntax that allows you to specify pipe-separated, tabular data for a step?
-
Array
-
Data Matrix
-
Data Table — TRUE
-
Example Table
-
Table
Explanation:
The Gherkin syntax is called a Data Table. It represents a 2-dimensional array, with cell boundaries indicated by pipe characters |
What value would be retieved from cell (1, 2) in the following table?
| 0 | 1 | 2 | | 3 | 4 | 5 | | 6 | 7 | 8 |
-
0
-
1
-
2
-
3
-
4
-
5 — TRUE
-
6
-
7
-
8
Explanation: The data table is indexed, starting from 0. The first coordinate indicates which row to access, the second indicates the column.
JAVA ONLY
Which of the answers correctly shows how Cucumber will convert the following data table into a list of maps: List< Map<String, String> > ?
| A | B | C | | D | 0 | 1 | | E | 2 | 3 |
-
[{A=A, B=B, C=C}, {A=D, B=0, C=1}, {A=E, B=2, C=3}]
-
[{A=D, B=2, C=1}, {A=E, B=0, C=3}]
-
[{A=B, D=0, E=2}, {A=C, D=1, E=3}]
-
[{A=D, B=0, C=1}, {A=E, B=2, C=3}] — TRUE
Explanation: When Cucumber converts a Data Table into a list of maps, it treats the first row as the labels (or indexes), and each subsequent row provides the values for the next map within the list.
So, in this example, since there are three rows, we end up with two maps in the list — the first row providing the indexes into each map, the next two rows providing the values for the two maps that are added to the list.
Which of the following Data Tables will this method process successfully?
@DataTableType public OrderLine createOrderLine(Map<String, String> orderItem) { return new OrderLine(entry.get("Item Name"), Integer.parseInt(entry.get("Quantity"))); }
-
| Item Name | Quantity | — TRUE | Cheese & tomato | 1 |
-
| name | quantity | | Cheese & tomato | 1 |
-
| Item Name | Quantity | — TRUE | Cheese & tomato | 1 | | Pepperoni | 1 |
-
| Item Name | Quantity | — TRUE
-
| Item Name | Quantity | Notes | — TRUE | Cheese & tomato | 1 | Extra cheese |
-
| Cheese & tomato | 1 | | Pepperoni | 1 |
-
| Quantity | Item Name | — TRUE | 1 | Cheese & tomato |
Explanation: Cucumber attempts to convert each non-header row in the data table into an instance of OrderLine by calling this method. For the conversion to succeed, the header cell text must match the hard coded index strings used in the method exactly. The order of the columns is not significant and any extra columns are ignored. A Data Table with only a header row would be considered an empty table and createOrderLine() would not be called and the relavent step definition would be passed an empty list of OrderLine objects.
There is more documentation about Data Table conversions at https://github.com/cucumber/cucumber/tree/master/datatable
RUBY ONLY
Which of the answers correctly shows how Cucumber will convert the following data table into symbolic hashes?
| A | B | C | | D | 0 | 1 | | E | 2 | 3 |
-
[{:A="A", :B="B", :C="C"}, {:A="D", :B="0", :C="1"}, {:A="E", :B="2", :C="3"}]
-
[{:A="D", :B="2", :C="1"}, {:A="E", :B="0", :C="3"}]
-
[{:A="B", :D="0", :E="2"}, {:A="C", :D="1", :E="3"}]
-
[{:A="D", :B="0", :C="1"}, {:A="E", :B="2", :C="3"}] — TRUE
Explanation: DataTable.symbolic_hashes returns an array of hashes. The first row is treated as the symbolic labels (or keys), and each subsequent row provides the values for the next hash within the array.
So, in this example, since there are three rows, we end up with two hashes in the list — the first row provides the keys into each hash, the next two rows providing the values for the two hashes that are added to the list.
Which of the following Data Tables will this step definition process successfully?
When "Sean orders" do |order| order.symbolic_hashes.each do |name:, quantity: | p "Item #{name} x #{quantity}" end end
-
| Name | Quantity | — TRUE | Cheese & tomato | 1 |
-
| Item name | quantity | | Cheese & tomato | 1 |
-
| Name | Quantity | — TRUE | Cheese & tomato | 1 | | Pepperoni | 1 |
-
| Name | Quantity | — TRUE
-
| Name | Quantity | Notes | — TRUE | Cheese & tomato | 1 | Extra cheese |
-
| Cheese & tomato | 1 | | Pepperoni | 1 |
-
| Quantity | Name | — TRUE | 1 | Cheese & tomato |
Explanation: For this method to execute successfully, the header cell text must match the hard coded index strings used in the method. The order of the columns is not significant and any extra columns are ignored. A Data Table with only a header row would be considered an empty table and symbolic_hashes would return an empty array.
There is more documentation about Data Table conversions at https://github.com/cucumber/cucumber/tree/master/datatable
5.4. Deeper into Data Tables
The way that we’ve specified this data is OK,
Rule: Shouts should only be heard if listener is within range
Scenario: Listener is within range
Given the range is 100
And people are located at
| name | location |
| Sean | 0 |
| Lucy | 50 |
When Sean shouts
Then Lucy should hear a shout
Scenario: Listener is out of range
Given the range is 100
And people are located at
| name | location |
| Sean | 0 |
| Larry | 150 |
When Sean shouts
Then Larry should not hear a shout
but your product owner would prefer you to express it like this instead: 🎬 2: edit data table to have headings in the first column
Scenario: Listener is within range
Given the range is 100
And people are located at
| name | Sean | Lucy |
| location | 0 | 50 |
When Sean shouts
Then Lucy should hear a shout
Scenario: Listener is out of range
Given the range is 100
And people are located at
| name | Sean | Larry |
| location | 0 | 150 |
When Sean shouts
Then Larry should not hear a shout
It’s always good to please the product owner, but you’re worried how we’ll handle it in our step definition? Fear not. Cucumber has you covered.
If you call the transpose
method on the table 🎬 3: stepdef, Cucumber will turn each row into a column before passing it to the step definition.
table.transpose.symbolic_hashes.each do |name: , location: |
🎬 5: show data tables in feature file Data tables are very useful for setting up data in Given steps, but you can also use them for specifying outcomes.
One rule that we’ve been implying but have never actually explored with an example is that people can hear more than one shout. So far we’ve only specified a single message, so let’s try writing a scenario where Sean shouts more than once: 🎬 6: write two-shouts scenario
Rule: Listener should be able to hear multiple shouts
Scenario: Two shouts
Given a person named Sean
And a person named Lucy
When Sean shouts "Free bagels!"
And Sean shouts "Free toast!"
Then Lucy hears the following messages:
| Free bagels |
| Free toast |
See how natural it is to use a Data Table here? We also haven’t used any column headers in this case, since the data is all in a single column anyway.
So how do we implement this step definition? Well, the DataTable has a really handy method called diff that we can use to compare two Data Tables. diff will pass if the tables are the same, and fail if they’re different.
Let’s run Cucumber to generate the new step definition 🎬 7: run Cucumber and paste it into our step definitions file 🎬 8: paste snippet 🎬 9
We’ll need the actual messages that Lucy’s heard to be stored in an object that looks like a DataTable, so we can compare it to the ones we expect.
An array of arrays will do 🎬 10: write code to create list-of-list-of-strings, so we can just iterate over Lucy’s messages and create a new single-item array for each row.🎬 11
actual_messages = @people['Lucy'].messages_heard.map { |message| [ message ] }
Now we can pass that array to the diff method on the table of expected messages passed in from the Gherkin. 🎬 12: write code to call diff
expected_messages.diff!(actual_messages)
Oops! It looks like we made a typo in our scenario. We should have included exclamation marks on the expected messages. 🎬 14 Well, at least this gives you a chance to see the nice diff output from Cucumber when the tables are different. We see the expected values prefixed with a minus, and the actual values prefixed with a plus.
Let’s fix just one of these 🎬 15: add exclamation mark so you can see how the diff output changes.
| Free bagels! |
🎬 16: run cucumber The matching bagels! line no longer has a minus, and for the mismatched row, the expected value still has a minus, and the actual value has a plus.
Let’s fix this last typo 🎬 17: add another exclamation mark, and we should be green again.
| Free toast! |
Great.
5.4.1. Lesson 4 - Questions
JAVA ONLY ===== Why might you use the @Transpose annotation?
-
Transpose has been deprecated and you should not use it
-
Use it if it is more readable to have the headers in the first column — TRUE
-
Use it to ensure that every row in the data table has the same number of columns
-
Use it to reverse the order of the data rows in the table
Explanation: The @Transpose annotation (and the transpose method on DataTable) allows you to use tables that have their headers in the first column of a table.
What method on DataTable compares two DataTables and produces a textual output showing their differences?
-
compare
-
difference
-
contrast
-
diff — TRUE
-
covariance
Explanation: The method that compares two data tables is called diff
5.5. DocString
When writing scenarios, occasionally we want to use a really long piece of data. 🎬 1: Show feature file
For example, let’s introduce a new rule about the maximum length of a message 🎬 2: New rule
Rule: Maximum length of message is 180 characters
…and add a scenario to illustrate it 🎬 3: New scenario, making the string just over the boundary of the rule:
Scenario: Message is too long
Given a person named Sean
And a person named Lucy
When Sean shouts "123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890x"
Then Lucy should not hear a shout
That’s pretty ugly isn’t it!
Still, we’ll press on and get it to green, then we’ll show you how to clean it up.
🎬 4: run Cucumber Our existing step definition handles that ugly step with the long message just fine, but the last outcome step is undefined. We could either add a new step definition, or parameterise "Larry should not hear a shout". Let’s modify the existing step definition 🎬 5: Modify stepdef
Then "{word} should not hear a shout" do |name|
expect(@people[name].messages_heard.count).to eq 0
end
OK, so we have a failing acceptance test.🎬 7 Let’s dive down into our solution and implement this new rule. It seems like the Network should be responsible for implementing this rule, so let’s go to its unit tests 🎬 8: show NetworkTest As we explain in more detail in the next lesson, we start by adding a new example to specify this extra responsibility. 🎬 9: start creating new unit test
We’ll create a 181-character message like this 🎬 10: finish implementing unit test and then assert that the message should not be heard when it’s broadcast.🎬 11
it "does not broadcast a message over 180 characters even if listener is in range" do
sean_location = 0
long_message = 'x' * 181
laura = spy(Shouty::Person, location: 10)
network.subscribe(laura)
network.broadcast(long_message, sean_location)
expect(laura.messages_heard).not_to have_received(:hear).with(long_message)
end
Let’s run that test 🎬 12: run Cucumber - show the unit test results🎬 13. Good, it fails. Lucy’s still getting the message at the moment. Now how are we going to implement this?
It looks like we’re already implementing the proximity rule here 🎬 14: Network proximity logic in the broadcast method. Let’s add another if statement here about the message length. 🎬 15: add message length logic
if message.size <= 180
listener.hear(message)
end
Run the unit test again… and it’s passing.🎬 16
And cucumber is green as well 🎬 17: run cucumber
Great.
The code here has got a little bit messy and hard to read. 🎬 18: show the modified logic in the Network class One very basic move we could make to improve it would be to just extract a couple of temporary variables, one for the range rule 🎬 19: extract range temporary variable and one for the length rule. 🎬 20: extract length temporary variable
@listeners.each do |listener|
within_range = (listener.location - shouter_location).abs <= @range
short_enough = message.size <= 180
if within_range
if short_enough
listener.hear(message)
end
end
end
That’s better. This code could be improved even further of course, but at least we haven’t made it any worse.
Let’s just run the tests to check. 🎬 21: run cucumber Great - everything’s still green.
Now we have everything passing again, we can tidy up the Gherkin to use a new piece of syntax we’ve been wanting to tell you about: a DocString. 🎬 22: show message length scenario in feature file
DocStrings allow you to specify a text argument for a step that spans over multiple lines. We could change our step to look like this instead: 🎬 23: convert message to DocString
Scenario: Message is too long
Given a person named Sean
And a person named Lucy
When Sean shouts the following message
"""
This is a really long message
so long in fact that I am not going to
be allowed to send it, at least if I keep
typing like this until the length is over
the limit of 180 characters.
"""
Then Lucy should not hear a shout
Now the scenario is much more readable. 🎬 24: show existing stepdef code
🎬 25: show existing stepdef code We have to add a new step definition too 🎬 26: sean_shouts_the_following_message. It doesn’t need a parameter in the Cucumber Expression 🎬 27 — the DocString gets passed as a string argument to the step definition automatically.🎬 28
Now we can fill out the rest of our new step definition. 🎬 29
When 'Sean shouts the following message' do |message|
@people["Sean"].shout(message)
@message_from_sean = message
end
Let’s check that we’re still green 🎬 30: run cucumber🎬 31 — and we are!
We don’t use DocStrings very often - having such a lot of data in a test can often make it quite brittle. But when you do need it, it’s useful to know about.
5.5.1. Lesson 5 - Questions
We start implementing the maximummessage length rule by writing a scenario and seeing it fail. What did we do next?
-
Write another scenario to triangulate the new behaviour of the Network class
-
Implement the changed behaviour in the Network class
-
Add a new unit test to NetworkTest that specifies the change in behaviour of the Network class — TRUE
Explanation: We wrote a new unit test in NetworkTest. We’ll talk more about this in the next lesson.
Why would we use a DocString?
-
It’s the only way to include multi-line strings in a scenarios
-
It’s a readable way to include long strings in a scenario — TRUE
-
DocStrings support multiple languages
-
Cucumber provides a DocString type that provides useful string manipulation features
Explanation: The DocString is Gherkin syntax that allows long strings to be readably represented in a scenario.
All the magic happens when the DocString is read from the Gherkin. The content of the DocString gets passed to the step definition as a normal String — there’s no corresponding Cucumber type.
Which of the following snippets of code are correct for the step below?
Then Simone says """ Now on that limb there was a branch A rare branch and a rattlin' branch And the branch on the limb And the limb on the tree And the tree in the bog And the bog down in the valley-o """
-
@Then("Simone says") public void simone_says() { }
-
@Then("Simone says") public void simone_says(String lyrics) { } — TRUE
-
@Then("Simone says {string}") public void simone_says(String lyrics) { }
-
@Then("Simone says {docstring}") public void simone_says(String lyrics) { }
-
@Then("Simone says {docstring}") public void simone_says(DocString lyrics) { }
Explanation: When using a DocString in a scenario, you do not add any parameter to the matching Cucumber Expression. You do need to provide a String parameter to the step definition to receive the contents of the DocString.
5.6. TDD Loops
You might have noticed that we’ve followed a pattern when we added behaviour to the system during this episode.
First we expressed the behaviour we wanted in a Gherkin scenario, wired up the step definitions, then ran Cucumber to watch it fail.
Then, we found the first class in our domain model that needed to change in order to support that new behaviour. In this case, the Network class. We used a unit test to describe how we wanted instances of that class to behave. Then we ran the unit test and watched it fail.
We focused in and made changes to the class until its unit tests were passing. When the unit tests were passing, we then made some minor changes to clean up the code and make it more readable. This is the basic test-driven-development cycle: red, green, clean.
The technical name for this last clean-up step is refactoring. Refactoring is an ugly name for an extremely valuable activity: improving the design of existing code without changing its behaviour. You can think about it like cleaning up and washing the dishes after you’ve prepared a meal: basic housekeeping. But imagine the state of your kitchen if you never made time to do the dishes.
Go on, imagine it for a second.
Yuck!
Well, that’s how many, many codebase end up. The good thing about taking this course is that we’re teaching you how to write solid automated tests, and the good thing about having solid automated tests is you can refactor with confidence, knowing that if you accidentally change the system’s behaviour, your tests will tell you.
Once we’re done refactoring, what do we do next? Run Cucumber, of course! In this case, our scenario was passing with a single trip round the inner TDD loop, but sometimes you can spend several hours working through all the unit tests you need to get a single scenario to green.
Once the acceptance test is passing, we figure out the next most valuable scenario on our todo list, and start the whole thing all over again!
Together, these two loops make the BDD cycle. The outer loop, which starts with an acceptance test, keeps us focussed on what the business needs us to do next. The inner loop, where we continuously test, implement then refactor small units of code, is where decide how we’ll implement that behaviour.
Both of these levels of feedback are important. It’s sometimes said that your acceptance tests ensure you’re building the right thing, and your unit tests ensure you’re building the thing right.
That’s all for this chapter of Cucumber School. See you next time!
5.6.1. Lesson 6 - Questions
Which of the following is the best definition of the term "refactoring"?
-
Improving the efficiency of the code without changing its behaviour
-
Adding new functionality to the application
-
Changing the behaviour of the code
-
Tidying up the code without changing its behaviour — TRUE
-
Rearchitecting the code to get ready for adding new functionality
Explanation: The definition of refactoring is: improve the design (of some code) without changing its behaviour.
When can refactoring happen?
-
When a refactoring story gets prioritised by the Product Owner
-
Whenever all tests are green — TRUE
-
When at least one test is failing
-
First thing in the morning
-
Before committing code to source control
-
At the end of an iteration
Explanation: Refactoring is part of the day-to-day work of every software developer. It’s when they tidy up the code once they’ve got it working.
Since part of the definition of refactoring is that it shouldn’t change the behaviour of the code, they will run the tests to make sure nothing was broken. Which means that the tests MUST be passing BEFORE they start refactoring. Otherwise, how can they be sure that the behaviour hasn’t changed?
How are the acceptance test and unit test related?
-
Acceptance tests ensure we "build the right thing"; unit tests ensure we "build the thing right" — TRUE
-
An acceptance test should be passing before we start writing unit tests
-
We may write many unit tests before the currently failing acceptance test passes — TRUE
-
All unit tests should be passing before we write an acceptance test
-
BDD consists of two loops: an outer acceptance test loop and an inner unit test loop — TRUE
Explanation: Once we have a scenario, we automate it — and we expect it to fail, because we haven’t added the functionality it specifies to the system yet. This is the beginning of the outer, acceptance test loop, that ensures we’re building what the Product Owner wants: "build the right thing."
We then enter the inner, unit test loop. It’s unit tests that define the precise behaviour of small units of code — and ensure that we "build the thing right." They give us the safety to improve the code’s design (refactor), because they will fail if we accidentally change the code’s behaviour while refactoring. We may have to go round the inner loop a number of times, adding several unit tests, before we’ve added enought functionality to make the outer acceptance test pass.
And then we write the next failing acceptance test…
6. Working with Cucumber
6.1. Basic Filtering
🎬 1: Animation Hello, and welcome back to Cucumber School.
Last time we learned about two very different kinds of loops. First, we used DataTables to loop over data in your scenarios.
Then we learned about TDD cycles. We saw how the outer loop of TDD helps you to build the right thing while the inner loop helps you build the thing right.
In this lesson, we’re going to teach you all about how to configure Cucumber.
When we start working on a new scenario we often take a dive down to the inner TDD loop where we use a unit testing tool to drive out new classes or modify the behaviour of existing ones. When our unit tests are green and the new code is implemented we return to the Cucumber scenarios to verify whether we have made overall progress or not.
bundle exec cucumber
If we have lots of Cucumber scenarios, it can be distracting to run all of them each time we do this. We often want to focus on a single scenario - or perhaps just a couple - to get feedback on what we’re currently working on.
There are several ways to do this in Cucumber - and your IDE may offer other options. We’ll start by showing some basic filtering on the command line, before showing you some alternatives.
Probably the easiest way to filter is to tell Cucumber to run only a scenario with a particular name.
🎬 4: type bundle exec command in console, show output and highlight that it has only run the one scenario 'Message too long' 🎬 5
We can pass arguments to Cucumber using the --name
option, which tells Cucumber to only run scenarios with a name matching "Message is too long".
bundle exec cucumber --name "Message is too long"
The value of the --name
option is actually a regular expression, so you can use your imagination here to run more than one scenario. Let’s use it to run all scenarios with the text "range" in their name. 🎬 6: type bundle exec command in console, show output and highlight that it has run the two scenarios containing the word 'range'🎬 7
bundle exec cucumber --name "range"
We can also use the abbreviated form of --name
, -n
🎬 8
bundle exec cucumber -n "range"
We can accomplish the same thing by adding some configuration on the cucumber.yml
file, but we’ll talk about that at the end of the lesson.
Another way to tell Cucumber to run a specific scenario is to specify the line number of the scenario within a feature file. The 'Message too long' scenario starts on line 44 in the feature file. 🎬 9: in feature file show that 'Message too long' starts on line 44
We can use that line number when we run Cucumber. 🎬 10: use line number on comand line 🎬 11 🎬 12
bundle exec cucumber features/hear_shout.feature:44
You can even specify multiple line numbers for each file. Let’s run 'Two shouts' as well. [🎬 13: in feature file, show 'Two shouts' starts on line 33
Let’s add that line number when we run Cucumber. 🎬 14: type bundle exec command in console, show output and highlight that it has run two scenarios - 'Two shouts' and 'Message too long'🎬 15
bundle exec cucumber features/hear_shout.feature:33:44
You can list several files and lines together like this if you have a specific set of scenarios you want to run.
The examples that you’ve just seen set a Ruby property on the command line. If you’re working in an IDE this is less than ideal, but fortunately there are other ways to achieve exactly the same outcome.
Let’s look at using a property file to set the system properties. Create a property file called cucumber.yml
in the root path of the project 🎬 16 and add default: --name range
🎬 17
When we run Cucumber, it picks up the property setting from the file and only runs the "range" scenarios. 🎬 18: type bundle exec cucumber
in console, show output and highlight that it has run the two scenarios containing the word 'range'🎬 19
bundle exec cucumber
The benefit of a properties file is that it gets checked in with your code, which means that any settings can be shared by all team members.
In this lesson you’ve learnt how to filter the set of scenarios to run using scenario names and line numbers from the command line and using property files.
6.1.1. Lesson 1 - Questions (Ruby)
Which command-line option sets a regular expression to filter the scenarios run by name?
-
--regex
-
--name (Correct)
-
--filter
-
--name-regex
Explanation:
You can find a list of supported Cucumber options by running cucumber --help
What are the benefits of using a cucumber.yml
config file rather than using the command line to set Cucumber properties? (MULTIPLE-CHOICE)
-
You can check
cucumber.yml
into source control to share with your team (Correct) -
You can set sensible defaults and avoid having to type them in each time (Correct)
-
Using
cucumber.yml
is more performant. (Incorrect)
Explanation:
Because cucumber.yml
is a file that lives within the project, it can be checked into source control and it can be accessed by all of your team.
You can still override these settings with what you specify on the command-line, but this enables to you set sensible defaults.
6.2. Filtering With Tags
🎬 1 In the previous lesson, we modified Cucumber’s behavior using scenario names and line numbers. In this lesson I’m going to show you how to filter scenarios using tags.
First, let’s delete the contents of the cucumber.yml
file. 🎬 2. We’ll leave default --quiet
for now. We’ll explain what’s it about in a later lesson.
If we run Cucumber, it will run all scenarios. 🎬 3: run Cucumber
We’ll put a focus tag right here, above this scenario. 🎬 4: add a @focus tag to the 'Listener is out of range' scenario Tags start with an at-sign and are case sensitive.
Now, after re-creating the `cucumber.yml`file, let’s add a tag expression to the default key, which Cucumber will use to filter the scenarios run 🎬 5
default: --quiet --tags @focus
Now Cucumber will run only the scenarios tagged with focus - there should be only one… 🎬 6: run Cucumber, then highlight that only 'Listener is out of range' scenario was run🎬 7
Yep.
It’s entirely up to you what you name your tags. When we’re working on a particular area of the application it is common to use a temporary tag like this - we’ll remove it before we check our code into source control.
Tags can be used for other purposes as well. If you have lots of scenarios it can be time-consuming to run them all every time. For example, you can tag a few of them with @smoke and run only those before you check in code to source control. 🎬 8: tag first and third scenario with @smoke🎬 9
default: --quiet --tags @smoke
Running just the smoke tests will give you a certain level of confidence that nothing is broken without having to run them all. 🎬 10: run Cucumber, then highlight that only first and third scenarios were run🎬 11
If you’re running Cucumber on a Continuous Integration Server as well, you could run all the scenarios there, detecting any regressions you might have missed by only running the smoke tests.
Tags give you a way to organise your scenarios that cut across feature files. You can think of them like sticky labels you might put into a book to mark interesting pages that you want to refer back to.
Some teams also use tags to reference external documents, for example, tickets in an issue tracker or planning tool. Let’s pretend we are using an issue tracker while working on Shouty and all the behaviour we built so far is related to the issue number 11. We could tag the whole feature file with this single line at the top. 🎬 12: tag feature with @SHOUTY-11 All the scenarios within that file now inherit that tag, so if we change the tag expression in cucumber.yml 🎬 13: edit YML file,
default: --quiet --tags @SHOUTY-11
Cucumber will run all the scenarios in the feature file. 🎬 14: run Cucumber, then show that all scenarios were run🎬 15
You can use more complex tag expressions to select the scenarios you want to run. For example, you could use a tag expression to exclude all the scenarios tagged as @slow. 🎬 16: tag last two scenarios with @slow Then rewrite the tag expression in cucumber.yml using the not
keyword 🎬 17: "modify Cucumber.yml Now when you run Cucumber, the "@slow" scenarios won’t be run. 🎬 18: run Cucumber, then show that the slow scenarios were not run
default: --quiet --tags "not @slow"
Note the need to add quotes around "not @slow"
.🎬 19 If they’re not added, running bundle exec cucumber
will result in an error.
You can read about how to build more complicated tag expressions on the Cucumber website 🎬 20: open the link and show the Cucumber website
There’s one more thing to learn about tags. They can be combined with hooks, so that you can be selective about which hooks to run when. We’ll cover that in a future chapter.
6.2.1. Lesson 2 - Questions (Ruby)
Look at this feature file, containing three scenarios, and several tags. Which of the tag expressions below would cause the scenario "Two" to be executed?: (MULTIPLE_CHOICE)
@mvp Feature: My feature Rule: rule A Scenario: One @smoke @slow @regression-pack Scenario: Two @regression-pack Scenario: Three
-
@SLOW
-
@regression-pack (Correct)
-
@mvp (Correct)
-
@regression-pack and not @slow
-
@MVP or @smoke (Correct)
Explanation:
Tags are inherited from the enclosing scope, so a Scenario inherits tags from the Feature. At present Rules cannot be tagged, although we expect this to be fixed in the near future, at which point tags will be inherited like this: Feature→Rule→Scenario.
Tags are case-sensitive, so @SLOW does not match @slow.
Tags can be on the same line and on consecutive lines.
Tag expressions implement the boolean operators: and, not, or.
6.3. More Control
Cucumber is first and foremost a tool that facilitates a common understanding between people on a project. Imagine our customers were cats. We could write our features in English, but the cats would obviously not understand that, so I’ll show you how to write a scenario in LOLCAT instead.
You can get a list of all the supported languages with --i18n-languages. 🎬 1
bundle exec cucumber --i18n-languages
Cucumber supports over 70 different languages, thanks to contributions from people from all over the world.
To see the translation of the Gherkin keywords for a particular language, just use the '--i18n' option with the language code. 🎬 2
bundle exec cucumber --i18n-keywords en-lol
Now create a new feature file in the features
directory called cat.feature
🎬 3
The first line tells Cucumber which language the feature file is written in. 🎬 4
# language: en-lol
Cucumber then expects the Gherkin keywords to be in LOLCAT 🎬 5
# language: en-lol
OH HAI: HEAR SHOUT
MISHUN: MESAGE IZ 2 LONG
I CAN HAZ A KAT CALLD SHOUTR
The step is undefined, but we can quickly generate it by running Cucumber 🎬 6: run new scenario Cucumber has generated a step definition.
We could copy and use this step definition in our steps.rb
file the same way we do with regular English. But let’s remove the cat.feature
file and continue.🎬 7
Notice that when we run Cucumber, the scenarios run in the order that the occur in the feature file 🎬 8: expand/highlight the test results That’s fine, but there’s a chance that since they always run in the same order we might accidentally make one scenario dependent on some other scenario.
Each scenario should be isolated - it’s result should not depend on the outcome of any other scenario. To help you catch any dependencies between your scenarios, Cucumber can be told to run your scenarios in a random order.
To do this, we can use the random flag in the profile file. 🎬 9
default: --quiet --order random
Now when we run Cucumber, the scenarios are run in a random order 🎬 10: run Cucumber & show new order of scenario execution. Now there’s almost no chance of a dependency between scenarios slipping through without being noticed.
🎬 11: show 'List configuration options' section of
A full list of Cucumber’s configuration properties can be found by passing the --help
flag on the command line. 🎬 12: highlight top of output The same can be accomblished by using the abbreviated version -h
🎬 13
bundle exec cucumber --help
bundle exec cucumber -h
That’s quite a lot to digest, but to make Cucumber really useful to your team, it’s good to spend some time learning the details of how to configure it. In this lesson, we showcased two of Cucumber’s configuration options and you learned how to write your scenarios in different spoken languages.
6.3.1. Lesson 3 - Questions
Which of the following first lines changes the language of a feature file?
-
# language: en-lol ----TRUE
-
! language: en-lol
-
language: en-lol
-
# i18n: en-lol
Explanation:
Gherkin supports lots of languages on a per feature file basis. It has to be the first line in the feature file, and has to be a comment with the content language: <language_identifier>
Why would you choose to execute scenarios in a random order?
-
For fun
-
To help discover memory leaks
-
The best documentation is always sorted randomly
-
To ensure scenarios are isolated from each other (Correct)
-
Scenarios, when run in parallel, are always executed in a random order
Explanation:
Scenarios should be isolated from each other - which means that one scenario should never rely on the behaviour of any other scenario for it to behave as expected. Cucumber is implemented to facilitate the cleaning of state after each scenario, but cannot guarantee this. Random execution order can help identify unintentional dependencies between scenarios.
See http://xunitpatterns.com/Erratic%20Test.html#Interacting%20Tests for more information.
6.4. Formatter plugins
Cucumber can report results in other formats, which can be useful for generating reports. Let’s take a closer look at some of the formatter plugins that ship with Cucumber - starting with the HTML formatter.
When we use the html formatter we need to specify the filename that the report should be written to using the --out flag.🎬 1 Otherwise it will be written directly to the console.
bundle exec cucumber --format html --out report.html
Let’s take a look at the html that has been generated. 🎬 2: Open target/my-report in the browser As you can see, this generates a nicely structured HTML page that details what scenarios were run and whether the system behaved as expected.
🎬 3: run Cucumber
Next, we’ll use the JSON formatter. We can use the abbreviated version of our --format
and --out
flags -f
and -o
.🎬 4🎬 5🎬 6
bundle exec cucumber -f json -o report.json
The JSON report outputs the results in a single file. 🎬 7: open the json file You can write your own script or program to post-process this file to generate your own report. Additionally, many continuous integration servers understand the json format and can turn it into a nicely formatted report.
There is also a progress formatter,🎬 8: run Cucumber which just prints out a single character for each step.🎬 9
bundle exec cucumber -f progress
🎬 10: run cucumber with multiple formatters We can specify multiple formatters like this.
bundle exec cucumber -f html -o report.html -f json -o report.json -f progress
The one without a path appended will be written to the console. If you ask multiple formatters to output to the console, only the last one mentioned will actually produce any output.
6.4.1. Rerun formatter
One of the formatter plugins is rather special - the rerun formatter. Before we try it out, let’s make one of our scenarios fail. 🎬 12: Changes “Two shouts” scenario’s “Sean shouts "Free bagels!"” step to “Sean shouts "Free cupcakes!"
Scenario: Two shouts
Given a person named Sean
And a person named Lucy
When Sean shouts "Free cupcakes!"
And Sean shouts "Free toast!"
Then Lucy hears the following messages:
| Free bagels! |
| Free toast! |
And see it fail on the terminal. 🎬 13: run npm test and show failing output
bundle exec cucumber
We choose the rerun formatter 🎬 14 and send the output to a file. Let’s call it rerun.txt
.🎬 16 🎬 17: run cucumber using rerun formatter
bundle exec cucumber -f rerun -o rerun.txt
Let’s look at what’s in that rerun.txt file. 🎬 18: open rerun.txt It’s a list of the scenarios that failed! And the format looks familiar doesn’t it? It’s using the line number filtering format that we showed you earlier.
This is really useful when you have a few failing scenarios and you want to re-run only ones that failed. We tell Cucumber to run only the failed scenarios by pointing it at the rerun file. 🎬 19: run Cucumber using the @rerun.txt file
bundle exec cucumber @rerun.txt
Using the @
-sign in front of the filename tells Cucumber to read the rerun file’s contents as though it was arguments on the command-line, just like we showed you in lesson 1.
This is a big time saver when you’re in the middle of a refactoring where you have broken a few scenarios and are working yourself back to green.
Let’s fix the scenario 🎬 20: undo change in feature file
Scenario: Two shouts
Given a person named Sean
And a person named Lucy
When Sean shouts "Free bagels!"
And Sean shouts "Free toast!"
Then Lucy hears the following messages:
| Free bagels! |
| Free toast! |
and rerun the failing scenario again 🎬 21: run npm test from console
Great. We’re back to green again.
6.4.2. Lesson 4 - Questions (Ruby)
Which or the following formatter plugins ship with Cucumber? (MULTIPLE-CHOICE)
-
AsciiDoc
-
HTML (Correct)
-
JSON (Correct)
-
Jira
-
JUnit (Correct)
-
Pretty (Correct)
-
Progress (Correct)
-
Rerun (Correct)
-
Message (Correct)
-
TAP
Explanation:
Cucumber ships with lots of formatter plugins. If the plugin that you want does not exist yet you can create your own. You can see an example in this feature file.
A newer plugin (that is out of scope for this course) is the message formatter. This emits a stream of JSON documents describing every event that happens during the Cucumber run.
How many formatters can output to the console in any run of Cucumber? MULTIPLE-CHOICE
-
Zero (Correct)
-
One (Correct)
-
More than one
Explanation:
So that the output remains easy to read, no more than one formatter is allowed to write to the console in any given run of Cucumber. You may choose to write the output of every plugin to file.
What does the rerun formatter do?
-
It causes Cucumber to rerun each scenario multiple times
-
It causes Cucumber to rerun each failed scenario
-
It outputs a list of failed scenarios identified by feature file and line number ----TRUE
-
It outputs a running total of how many times each scenario has ever been run
Explanation:
The rerun formatter keeps track of the feature file and line number of every scenario that fails. This information is output in a <feature file>:<line number>
format that can be saved to a file. You can then use this file to easily run just the scenarios that failed.
6.5. Publishing Results
🎬 1: show HTML output In the last lesson we saw the output generated by the HTML plugin. This is a great way for your business colleagues and customers to engage with the development of the product. Since the scenarios are written in business language, they act as documentation that everyone interested in the product can understand. And, because the results are colour-coded green (if the system behaves as expected) or red (if it doesn’t) everyone can see what behaviour has been implemented.
The challenge is making sure that this documentation is easily available. Feature files are usually checked into source control along with the source code, which isn’t a great way of making it accessible to less technical team members. The HTML output can be shared, but it needs to be stored somewhere that everyone has access to - such as on your intranet or wiki. That’s something that your team has to configure themselves.
Since Cucumber 6.0.0 there has been a new feature that allows you to automatically publish the results online to a free service called Cucumber Reports. 🎬 2: go to reports.cucumber.io
Ensure that you’re using Cucumber 6.0.0 or later 🎬 3: show package-lock.json file
cucumber (6.0.0)
and remove publish-quiet from cucumber.yml
. 🎬 4: cucumber.js
default: --order random
Now when we run Cucumber, 🎬 5: npm test it helpfully prints out a banner telling you how to publish your results to Cucumber Reports 🎬 6: highlight banner.
Let’s follow the instructions in the banner and add --publish to cucumber.js 🎬 7.
default: --publish --order random
Now we can run Cucumber again and we get a different banner output 🎬 8. The unique URL in this banner is the address of your published report 🎬 9: highlight URL.
🎬 10: follow URL Open the URL and you’ll find your report published in the same format as the HTML plugin. Now you can share the report just by sharing the URL. You also get some extra information about the run that generated the report. If you publish reports from a recognised CI server, then extra information is gathered.
You might have noticed the warning in the banner that says the report will "self-destruct". 🎬 12: highlight warning To prevent a report from being deleted, it has to be associated with an authenticated account.
Follow the link in the report to authenticate your account 🎬 13🎬 14 You can then retrieve your Cucumber Reports publish token 🎬 15. Reports associated with a publish token will be kept until explicitly deleted.
🎬 16: publish quiet During development you probably won’t want to publish a report after every build… and you might prefer not to see the banner output telling you about Cucumber Reports. Let’s fix this in cucumber.js by replacing --publish with --publish-quiet:
default: --publish-quiet --order random
🎬 17: no banner Now when we run Cucumber, no report is published and no banner advertising Cucumber Reports is output.
Cucumber Reports is under active development, so expect to see many more useful features coming in the coming months.
6.5.1. Lesson 5 - Questions (Ruby)
Why did the Cucumber team create Cucumber Reports?
-
To enable the removal of the HTML formatter
-
To make it easier to share living documentation between stakeholders and team members (Correct)
-
To make money from Cucumber users
-
To get a chance to play with cloud platforms
Explanation:
The output from running Cucumber forms the basis of a system’s "living documentation". There are lots of tools that let you do publish and share this living documentation, but they require effort to set up and configure. Now it’s as simple as asking Cucumber to publish it.
How does the output from the HTML formatter differ from that displayed by Cucumber Reports?
-
Cucumber Reports has much less detail that the HTML formatter
-
Cucumber Reports uses animated GIFs and emojis to make the living documentation more exciting and fun
-
Cucumber Reports are identical to the HTML output, but include some extra information about the build environment (Correct)
Explanation:
The same underlying library generates the output for both Cucumber Reports and the HTML formatter. Cucumber Reports also displays some extra information about that run of Cucumber, summarising the execution context and scenario outcomes.
What extra information can be published by Cucumber Reports? (MULTIPLE_CHOICE)
-
Username of last committer
-
Version of Cucumber that generated the report (Correct)
-
Identifier (SHA) of the last git commit (Correct)
-
Version of operating system that Cucumber ran on (Correct)
-
Timestamp of when the report was generated
-
Cucumber configuration properties used
-
Number of scenarios that ran (Correct)
-
Percentage of scenarios that passed (Correct)
-
Version of CI tool that ran Cucumber (Correct)
-
Code coverage statistics for run
Explanation:
Cucumber Reports always publishes:
-
High level statistics of scenarios run and their status
-
Elapsed time since the report was generated
-
The time it took to run the scenarios and generate the report
-
Operating system version
-
Language version used for stepdefs
-
Cucumber version
If Cucumber Reports was run by a supported CI tool, it also publishes:
-
CI tool version
-
Identifier (SHA) of the last commit
What happens to reports that are not associated with a "publish token"
-
The report is not published
-
The report is published and cannot be deleted
-
The report is published and will be deleted automatically (Correct)
Explanation:
Reports that are not associated with a publish token will be deleted approximately 24 hours after being published. Once published, a report can be associated with a publish token, which will then preserve the report until it is explicitly deleted.
For more details on publish tokens, authentication, and association with Github repositories, see https://reports.cucumber.io
7. Details
In the last lesson we took a break from the code to sharpen up your skills with Cucumber’s command-line interface.
Now it’s time to dive right back into the code. We’re going to explore one of the hottest topics that teams come across when they start to get to grips with Cucumber and BDD: how much detail to use in your scenarios.
Many teams find they can’t easily agree on this. It can often seem like a matter of personal preference. It’s true there are no right and wrong answers, but we’re going to teach you some heuristics you can apply to help you make better decisions.
7.1. Premium Accounts! With a bug!
Once again, while we were away, the developers of Shouty have been busy. A new hot-shot ninja rock-star subcontractor, Stevie, has built a new feature called premium accounts. We don’t know much about it yet but, our tester Tamsin has reported a bug from their manual exploratory testing, and it’s up to us tofix it.
7.1.1. The bug
Tamsin has helpfully documented the bug as a failing scenario. Here is it:
$ cucumber --tags @todo
Feature: Premium accounts
@todo
Scenario: BUG #2789
Given Sean has bought 30 credits
When Sean shouts "buy, buy buy!"
Then Sean should have 25 credits
Expected: 25
Got: 15
Hum. So let’s try and figure out what this is all about. Sean starts out with some credits, 🎬 2: Highlight first step presumably that’s what gives his account premium status?
7.1.2. Reading the new feature
Let’s read the whole feature file - that should tell us some more about how the system is supposed to behave.
This is very difficult to read.
Feature: Premium account
Background:
Given the range is 100
And people are located at
| name | Sean | Lucy |
| location | 0 | 100 |
Scenario: Test premium account features
Given Sean has bought 30 credits
When Sean shouts "Come and buy a coffee"
And Sean shouts "My bagels are yummy"
And Sean shouts "Free cookie with your espresso for the next hour"
And Sean shouts the following message
"""
You need to come and visit Sean's coffee,
we have the best bagels in town.
"""
And Sean shouts "Who will buy my sweet red muffins?"
And Sean shouts the following message
"""
This morning I got up early and baked some
bagels especially for you. Then I fried some
sausages. I went out to see my chickens, they
had some delicious fresh eggs waiting for me
and I scrambled them just for you. Come on over
and let's eat breakfast!
"""
And Sean shouts "Buy my delicious sausage rolls"
And Sean shouts the following message
"""
Here are some things you will love about Sean's:
- the bagels
- the coffee
- the chickens
Come and visit us today! We'd really love to see you.
Pop round anytime, honestly it's fine.
"""
And Sean shouts "We have cakes by the dozen"
Then Lucy hears all Sean's messages
And Sean should have 11 credits
@todo
Scenario: BUG #2789
Given Sean has bought 30 credits
When Sean shouts "buy, buy buy!"
Then Sean should have 25 credits
Apart from Tamsin’s new bug report scenario at the bottom, there’s only one scenario, and it’s long! I think I can count twelve steps altogether, excluding the background steps. As a general guideline, we like our scenarios to be no longer than about five steps long, so this is big.
🎬 6
The scenario has several When
steps - nine of them in all! - which is often a sign that the scenario is trying to test more than one business rule at a time.
🎬 7
We normally like to document the business rules in the Gherkin, and add a few paragraphs of prose description, but there’s no Rule
keyword and the description section is blank, so we haven’t been left any clues there.
Let’s see if we can gleam the rules from reading the scenario carefully. Sean starts out with 30 🎬 8 credits and ends up with 11, but why?🎬 9
There’s so much detail here. I wonder which bits are important. Could it be the number of words Sean shouts that affects his credits? Or the number of messages? Based on Tamsin’s test case, maybe the word “buy” is important…
It’s really hard to tell.
It’s interesting that Lucy hears all Sean’s messages - even these ones which look to be over 180 characters and would normally be blocked. Perhaps premium accounts get to send messages over 180 characters in exchange for credits?
This is a classic example of a scenario written by someone who is using Cucumber as a testing tool rather than a documentation tool. This scenario may well work as an effective test, but it’s terrible documentation. We have no clear idea from reading it what the system is supposed to do.
Let’s try reading the code instead.
class Network
def initialize(range)
@range = range
@listeners = []
end
def subscribe(person)
@listeners << person
end
def broadcast(message, shouter)
short_enough = message.size <= 180
deduct_credits(short_enough, message, shouter)
@listeners.each do |listener|
within_range = (listener.location - shouter.location).abs <= @range
if within_range && (short_enough || shouter.credits >= 0)
listener.hear(message)
end
end
end
def deduct_credits(short_enough, message, shouter)
shouter.credits -= 2 unless short_enough
shouter.credits -= (message.scan(/buy/i) || []).size * 5
end
end
OK, so we have a deduct_credits
method here that seems to encapsulate the rules.🎬 11 It looks like over-long messages - messages that are not short enough - cost two credits,🎬 12 and each time the word “buy” is mentioned, we deduct 5 credits.🎬 13
It’s a good job we know how to read code!
One of the promises of BDD is that our feature files become the single source of truth for the whole team. Yet here, the scenario does such a poor job of documenting what the system does, we had to go hunting for the truth in the code. That’s fine for us, because we know how to read code, but it’s excluded anyone on our team who isn’t technical. How would they be able to understand the behaviour of premium accounts at the moment?
We need to fix this feature file.
7.1.3. Lesson 1 - Questions
Which do we think are the correct business rules in Shouty problem domain?
-
Mentioning the word "buy" costs 5 credits (Correct)
-
Mentioning the word "buy" costs 25 credits
-
A message that’s not short enough costs 2 credits (Correct)
-
Each message costs 11 credits
-
Each word costs 1 credit
Explanation:
In the end, we discovered that the business rules appear to be:
-
Mention the word "buy" and you lose 5 credits.
-
Long messages cost 2 credits
How did we discover the business rules?
-
They were clearly documented in the Gherkin
-
We went and talked to the stakeholders
-
We had to read the code (Correct)
Explanation:
Talking to the stakeholders would be great if we had easy access to them, but they’re often busy and certainly don’t want to have to repeat themselves over and over to the delivery team. If we’ve had them tell us the business rules once, it’s a good idea for us to document them clearly in a place that everyone can read.
Why is it a problem that the business rules could only be found in the code?
-
Not everyone can read code, so not everyone can look up the business rules when they want to (Correct)
-
Code is always changing
-
The code could be badly designed and hard to read (Correct)
-
Business rules in code can’t be validated
Explanation:
The code is the ultimate source of truth for the system’s behaviour, but it’s often difficult for even experienced developers to read code and know exactly what it will do. Certainly, less technical team memmbers are completely excluded from this source of truth.
The advantage of using Gherkin together with Cucumber to validate it, is that you get a single source of truth that everyone on the team can read.
What makes it hard to understand the business rules from Stevie’s scenario, "Test premium account features"
-
It’s long (Correct)
-
It’s trying to test multiple business rules at the same time (Correct)
-
It doesn’t document the business rules that the scenarios are supposed to illustrate (Correct)
-
It has too much detail (Correct)
-
The name of the scenario does not describe a specific aspect of the system’s behaviour (Correct)
-
It’s designed to be a test, rather than documentation (Correct)
Explanation:
We made up this example to showcase some of the classic mistakes we’ve seen people make with their scenarios. In lesson 3 we’ll show you some specific criteria you can use to judge the quality of your team’s scenarios.
7.2. Clarifying the rules
This seems like a good time to ask our domain experts for clarification about this behaviour. Ideally a three amigos meeting would have done this already, but things don’t always go according to the script. Let’s pay a visit to our product owner.
Paula the product owner tells us the rules are as follows: 🎬 2: Writes the rules up in the feature file
Feature: Premium account
Rules:
* Mention the word "buy" and you lose 5 credits.
* Long messages cost 2 credits
We’ve just scribbled these down in the feature description for now, but later, once we’ve teased this scenario apart, we’ll be able to use the Rule
keyword as we showed you in chapter 4.
OK, well that helps us a great deal, but this scenario is still doing a poor job of illustrating those rules. Let’s take a moment to understand why.
It’s all about the details.
When you’re first exploring a new domain problem, details like the exact messages people have shouted are a great way to shine light into the dark corners of your ignorance. They bring examples to life by making them vivid and real. This encourages what we call divergent thinking, which helps you discover even more examples as you explore the behaviour you need to provide.
In BDD, we call this deliberate discovery, when we try to explore the problem domain as thoroughly as we can. Details are good here.
At some point, however, we need to write a computer program. When that time comes, we need to switch to convergent thinking and try to distill down all that detail into something that clearly communicates the essence of the behaviour. Now, too much detail can be distracting, or even misleading to our future readers. We call these incidental details.
Being able to distill down the essence of a scenario with too much detail is a skill you’ll find yourself practicing time and time again as a BDD practitioner, since we almost always deliberately start with too much detail, then want to refine it afterwards as it becomes more clear which details are relevant, and which are incidental.
7.2.1. Lesson 2 - Questions
Why did we confirm the business rules with our product owner, Paula?
-
Paula will fire us if we get the business rules wrong
-
As Product Owner, it’s Paula’s responsibilty to decide which rules we should implement in our code (Correct)
-
We want to as confident as we can that they’re correct before we proceed (Correct)
-
We don’t want Paula to feel left out
Explanation:
Teams should collaborate to discover and decide on the business rules their code should implement but ultimately, on most teams, it’s the Product Owner’s decision what is "correct" or incorrect behaviour.
Getting this clarified only takes a few minuutes, and can save us potentially wasting our time further down the line building upon an earlier misunderstanding.
Why did we not use the Rule: keyword as described in chapter 4?
-
The version of Cucumber we’re using doesn’t support it
-
Paula doesn’t like that way of writing Rules in Gherkin
-
The big scenario is covering multiple rules, so we couldn’t place it under a single Rule (Correect)
Explanation:
Before we had the Rule
keyword in Gherkin, we always used to recommend to teams to write their rules down at the top of the feature file. This works great, but using the Rule
keyword to make an explicit link between each scenario and the rule it illustrates is even better. As soon as we’ve broken this scenario down into smaller ones that only illustrate one rule, we can do that.
Which of these statements are true?
-
Using real details in your scenario can help with divergent thinking (Correct)
-
Divergent thinking closes our minds to the possiblity of scenarios we haven’t thought of
-
Details are things like real names of people or products, dates, or amounts (Correct)
-
Finding the right level of detail to use in a scenario is easy
-
Incidental details are details that aren’t relevent to the purpose of the scenario (Correct)
-
If your team can’t agree on the right level of detail then you should give up on BDD
Explanation:
Finding the right level of detail, and discerning which details are incidential to your scenario is a difficult, subjective process. The key thing to do is keep trying. Each time you talk about it as a team, you will better align your perspectives (as long as you listen to each other!) and hone your own domain language.
7.3. Keeping your scenarios BRIEF
Seb Rose came up with a brilliant acronym to help you remember six heuristics or guidelines to think about when reviewing the quality of your scenarios: BRIEF.
"B" is for Business language: The words used in a scenario should be drawn from the business domain, otherwise you will not be able to engage your business colleagues painlessly. The language you use should use terms that business team members understand unambiguously.
"R" is for Real data: When you’re first discussing the details of a story, your examples should use concrete, real data. This helps bring the scenarios to life, and expose boundary conditions and underlying assumptions early in the development process. When writing scenarios, we should also use real data whenever this helps reveal intention and make them move vivid.
"I" is for Intention revealing: Scenarios should describe the intent of what the actors in the scenario are trying to achieve, rather than describing the mechanics of how they will achieve it. We should start by giving the scenario an intention revealing name, and then follow through by ensuring that every line in the scenario describes intent, not mechanics.
"E" is for Essential: The purpose of a scenario is to illustrate how a rule should behave. Any parts of a scenario that don’t directly contribute to this purpose are incidental and should be removed. If they are important to the system, then they will be covered in other scenarios that illustrate other rules. Additionally, any scenarios that do not add to the reader’s understanding of the expected behaviour have no place in the documentation.
"F" is for Focused: Most scenarios should be focused on illustrating a single rule.
You might have noticed we said "six heuristics" but we only gave you five. That’s because the word Brief itself is the sixth guideline. We suggest you try to restrict most of your scenarios to five lines or fewer. This makes them easier to read, much easier to reason about, and helps avoid that temptation to test multiple rules, or add incidental details.
7.3.1. Review our new feature using the BRIEF heuristics
Let’s have a look at this new Premium Accounts scenario and use the BRIEF heuristics to help us look for ways to improve it.
Feature: Premium account
Background:
Given the range is 100
And people are located at
| name | Sean | Lucy |
| location | 0 | 100 |
Scenario: Test premium account features
Given Sean has bought 30 credits
When Sean shouts "Come and buy a coffee"
And Sean shouts "My bagels are yummy"
And Sean shouts "Free cookie with your espresso for the next hour"
And Sean shouts the following message
"""
You need to come and visit Sean's coffee,
we have the best bagels in town.
"""
And Sean shouts "Who will buy my sweet red muffins?"
And Sean shouts the following message
"""
This morning I got up early and baked some
bagels especially for you. Then I fried some
sausages. I went out to see my chickens, they
had some delicious fresh eggs waiting for me
and I scrambled them just for you. Come on over
and let's eat breakfast!
"""
And Sean shouts "Buy my delicious sausage rolls"
And Sean shouts the following message
"""
Here are some things you will love about Sean's:
- the bagels
- the coffee
- the chickens
Come and visit us today! We'd really love to see you.
Pop round anytime, honestly it's fine.
"""
And Sean shouts "We have cakes by the dozen"
Then Lucy hears all Sean's messages
And Sean should have 11 credits
B is for Business Language
Firstly, does this scenario use business language?
Well, we can see words like "credits" and "shout" and "message" which look like terms from our problem domain’s ubiquitous language, but there are also a lot of words in these scenarios that are nothing to do with our domain - words like "cakes" or "chickens" or "muffins" for example. Granted, all these words are surrounded by quotes, but they’re still distracting.
R is for Real Data
Do these scenarios use real data? Without seeing the user research, it looks like they do. We’re not talking about "User A" or "A message", but instead we’re using realistic examples of the kinds of messages genuine users of our app might send to each other.
I is for Intention Revealing
This one really hits the nail on the head. The intention of this scenario is completely obscured by all the detail. As we said earlier, we have no idea from this Gherkin what is special about these messages, or how they contribute to the final tally of expected credits. We had to go down into the code to start to figure that out.
E is for Essential
It’s hard to know whether every step here is essential to the purpose of this scenario, because we don’t even know what it’s purpose is yet! Once we’ve made the steps more intention revealing it will be easier to see if they’re all essential.
F is for Focussed
We’re not quite clear what exactly the rules are that underpin the behaviour in this scenario (yet) but we have a hunch that there’s more than one rule being played out here. So we’re not looking good on this heuristic.
And is it Brief?
Certainly, this scenario is well over five lines long. So not Brief.
So we have some work to do!
7.3.2. Lesson 3 - Questions
What do we mean when we say that BRIEF is a set of "heuristics"?
-
We should follow them precisely or the BDD police will surely come and arrest us
-
We can use them to guide our work, but we may find times when they’re not approriate in our context (Correct)
-
They are complicated and hard to remember
Explanation:
Heuristics are mental shortcuts - general guidelines that are easy to remember and can be trusted to work in most (but not all!) contexts.
B in "BRIEF" is for…
-
Behaviour Describing - scenarios should describe behaviour
-
Big - scenarios should have as many steps as possible
-
Business Lanaguage - the language in your scenario should come from the business problem domain (Correct)
-
Basic - keep it simple, don’t add extraneous information
Explanation:
"B" is for Business language: The words used in a scenario should be drawn from the business domain, otherwise you will not be able to engage your business colleagues painlessly. The language you use should use terms that business team members understand unambiguously.
"R" in BRIEF stands for "Real Data". In the context of an ATM, which of these steps follow this heuristic?
-
When Bobby makes a withdrawl of $50 (Correct)
-
Given "User A" has opened an account
-
Given an account with an arranged overdraft of $123
-
When the user changes their PIN to 0000 (Correct)
-
Given a cheque for $19.99 was deposited on 2021-04-01 (Correct)
-
When Bobby makes a withdrawl of $1
Explanation:
"R" is for Real data: When you’re first discussing the details of a story, your examples should use concrete, real data. This helps bring the scenarios to life, and expose boundary conditions and underlying assumptions early in the development process. When writing scenarios, we should also use real data whenever this helps reveal intention and make them move vivid.
Arguably, the PIN of 0000 is not very realistic, but if this step is being used in a scenario to describe validation rules for new PIN numbers, this could be entirely appropriate real-world data for an invalid PIN.
It’s impossible to withdraw $1 from most ATMs, so this is not real data. Similarly, most banks won’t let you set an overdraft of $123. We can be more imaginative than calling our user "User A".
"I" in BRIEF stands for "Intention Revealing". Which of these steps are good at revealing their intention?
-
When Bobby makes a withdrawl of $50 (Correct)
-
When I click on ".btn-primary"
-
When I log in (Correct)
-
Then the response should match "data/test_response-4591.json"
-
Then the response should contain balance of $50 (Correct)
-
When I fill in "input[type=text].search" with "Zen Motorcycle"
-
When I search for "Zen Motorcycle" (Correct)
Explanation:
"I" is for Intention revealing: Scenarios should describe the intent of what the actors in the scenario are trying to achieve, rather than describing the mechanics of how they will achieve it. We should start by giving the scenario an intention revealing name, and then follow through by ensuring that every line in the scenario describes intent, not mechanics.
"E" in BRIEF stands for "Essential". Look at this scenario, then select the steps that are essential.
Feature: Search
Rule: Search result includes any title containing all keywords in the search
Scenario: Two keywords, only one book matches both
Given a publisher "Penguin"
And a book "Zen and the Art of Motorcycle Maintenance" is in stock
And a book "The Art of Zen Gardens" is in stock
And I already have three books on loan
When I nagivate to the search page via the main menu
And I search for "Zen Motorcycle"
Then there should be 1 result
And the results should include:
| Title |
| Zen and the Art of Motorcycle Maintenance |
-
Given a publisher "Penguin"
-
And a book "Zen and the Art of Motorcycle Maintenance" is in stock (Correct)
-
And a book "The Art of Zen Gardens" is in stock (Correct)
-
And I already have three books on loan
-
When I nagivate to the search page via the main menu
-
And I search for "Zen Motorcycle" (Correct)
-
Then there should be 1 result (Correct)
-
And the results should include: (Correct)
Explanation:
"E" is for Essential: The purpose of a scenario is to illustrate how a rule should behave. Any parts of a scenario that don’t directly contribute to this purpose are incidental and should be removed. If they are important to the system, then they will be covered in other scenarios that illustrate other rules. Additionally, any scenarios that do not add to the reader’s understanding of the expected behaviour have no place in the documentation.
The step about creating a publisher is incedental as the publisher does not appear anywhere else in the scenario. Perhaps the team added it because there’s a database relationship between books and publishers, and you can’t create a book without a publisher. That’s not a good reason to clutter up your scenario. Instead, you can use patterns like "test data builder" to create your Publisher automatically when you create a Book.
The step about having books on load is incidental because the number of books on loan, as far as we can tell, has no bearing on the behaviour of the search, which is what this scenario is focussed on.
The step about navigation is incidental. We are describing the business logic here, not the UI navigation.
"F" is for Focused: Most scenarios should be focused on illustrating a single rule. Why?
-
If a scenario fails, it’s difficult to know what’s broken. (Correct)
-
It makes it hard to develop the user interface for the feature
-
It encourages you to add behaviour in small increments (Correct)
-
It helps you to make sure you have good coverage of your business rules (Correct)
-
It keeps the code clean
Explanation:
BDD is an extension of TDD, or Test-Driven Development. In both these practices, we gradually add new behaviour, with each test demanding a little more behaviour from the system. Working like this, it makes sense to work in small increments, adding focussed scenarios so that we can iterate rapidly and keep a good flow going.
A pleasant side-effect of working this way is that each test has a single purpose, and so has fewer reasons to fail. So when a regression occurs, you tend to get a clear signal from the failing scenario about what’s broken. Compare this with having a single huge scenario that covers a lot of behaviour. When that fails, you know something’s wrong but you don’t have much idea about where to go to start fixing it.
When you’re concious of the relationship between rules and example, and you want to make sure all of your rules are covered, it’s much easier to do that and see clearly where you might have gaps if each of your examples covers a single rule.
7.4. Distilling the essence
Let’s try to distill the essence of this scenario by removing the incidental details.
Scenario: Test premium account features
Given Sean has bought 30 credits
When Sean shouts "Come and buy a coffee"
And Sean shouts "My bagels are yummy"
And Sean shouts "Free cookie with your espresso for the next hour"
And Sean shouts the following message
"""
You need to come and visit Sean's coffee,
we have the best bagels in town.
"""
And Sean shouts "Who will buy my sweet red muffins?"
And Sean shouts the following message
"""
This morning I got up early and baked some
bagels especially for you. Then I fried some
sausages. I went out to see my chickens, they
had some delicious fresh eggs waiting for me
and I scrambled them just for you. Come on over
and let's eat breakfast!
"""
And Sean shouts "Buy my delicious sausage rolls"
And Sean shouts the following message
"""
Here are some things you will love about Sean's:
- the bagels
- the coffee
- the chickens
Come and visit us today! We'd really love to see you.
Pop round anytime, honestly it's fine.
"""
And Sean shouts "We have cakes by the dozen"
Then Lucy hears all Sean's messages
And Sean should have 11 credits
We can start with this step, When Sean shouts "Come and buy a coffee".
🎬 2
From what what we know about the rules, what’s important here is that the shout contains the word “buy”. So let’s write that:
When Sean shouts a message containing the word "buy"
Better. We still have a little bit of detail - the word buy - but that’s quite important, in fact it helps to illustrate our business rule.
We’ll need a new step definition for this, of course.
bundle exec cucumber --tags "not @todo"
🎬 5 We can just copy the code for shouting a message from this step down here for now. We will deal with this duplication, but later.
When 'Sean shouts a message containing the word {string}' do |word|
message = "A message containing the word #{word}"
@people["Sean"].shout(message)
@messages_shouted_by["Sean"] << message
end
Let’s run Cucumber to check we haven’t made any mistakes…
bundle exec cucumber --tags "not @todo"
All green. Good.
Now we can use that same step everywhere else in the scenario, too, where that is the real intent of the step.
🎬 8: Changes the other two steps that mention the word “buy”. Runs cucumber --tags ~@todo
. All green.
Scenario: Test premium account features
Given Sean has bought 30 credits
When Sean shouts a message containing the word "buy"
And Sean shouts "My bagels are yummy"
And Sean shouts "Free cookie with your espresso for the next hour"
And Sean shouts the following message
"""
You need to come and visit Sean's coffee,
we have the best bagels in town.
"""
When Sean shouts a message containing the word "buy"
And Sean shouts the following message
"""
This morning I got up early and baked some
bagels especially for you. Then I fried some
sausages. I went out to see my chickens, they
had some delicious fresh eggs waiting for me
and I scrambled them just for you. Come on over
and let's eat breakfast!
"""
When Sean shouts a message containing the word "buy"
And Sean shouts the following message
"""
Here are some things you will love about Sean's:
- the bagels
- the coffee
- the chickens
Come and visit us today! We'd really love to see you.
Pop round anytime, honestly it's fine.
"""
And Sean shouts "We have cakes by the dozen"
Then Lucy hears all Sean's messages
And Sean should have 11 credits
And run cucumber again. 🎬 9
bundle exec cucumber --tags "not @todo"
7.4.1. Distill the regular shout step
We can apply this same pattern to remove the noisy incidental details from this next step too. What’s the essence of this step? 🎬 10
And Sean shouts "My bagels are yummy"
In this case, it really doesn’t matter what Sean is shouting - a regular shout doesn’t have any effect on his premium account credits. So we could just re-word it like this:
And Sean shouts a message
Again, we’ll need a new step definition, and again, we’ll just duplicate the code for shouting a message for now.
🎬 12: Runs cucumber --tags ~@todo
, copies snippet. Copies shouting code from step def below, then adds code above to create a test message.
🎬 13
When "Sean shouts a message" do
message = "A message from Sean"
@people["Sean"].shout(message)
@messages_shouted_by["Sean"] << message
end
We’ll run Cucumber again just in case…
bundle exec cucumber --tags "not @todo"
All green.
And finally, let’s use that new step everywhere we can…
Scenario: Test premium account features
Given Sean has bought 30 credits
When Sean shouts a message containing the word "buy"
And Sean shouts a message
And Sean shouts a message
And Sean shouts the following message
"""
You need to come and visit Sean's coffee,
we have the best bagels in town.
"""
When Sean shouts a message containing the word "buy"
And Sean shouts the following message
"""
This morning I got up early and baked some
bagels especially for you. Then I fried some
sausages. I went out to see my chickens, they
had some delicious fresh eggs waiting for me
and I scrambled them just for you. Come on over
and let's eat breakfast!
"""
When Sean shouts a message containing the word "buy"
And Sean shouts the following message
"""
Here are some things you will love about Sean's:
- the bagels
- the coffee
- the chickens
Come and visit us today! We'd really love to see you.
Pop round anytime, honestly it's fine.
"""
And Sean shouts a message
Then Lucy hears all Sean's messages
And Sean should have 11 credits
…and run Cucumber once again.
bundle exec cucumber --tags "not @todo"
All green.
7.4.2. Distill the long shout step
Now let’s deal with this next step.🎬 17 What’s the essence here?
And Sean shouts the following message
"""
You need to come and visit Sean's coffee,
we have the best bagels in town.
"""
Again, the exact words in the shout don’t have any significance in this scenario. What matters, if anything, is that this is a long message. It’s under 180 characters, but still longer than a regular message. It’s not clear how important this distinction is just yet, but let’s give the authors of this scenario the benefit of the doubt for now that there must be some reason it’s different. So we’ll push the details about the message content down into the step definition, following the same recipe.
And Sean shouts a long message
Again, we’ll tolerate the duplication for now, and just generate a long message here.
When "Sean shouts a long message" do
message = ["A message from Sean", "that spans multiple lines"].join("\n")
@people["Sean"].shout(message)
@messages_shouted_by["Sean"] << message
end
7.4.3. Distill the over-long shout step
Now what’s interesting about this next step? 🎬 23
And Sean shouts the following message
"""
This morning I got up early and baked some
bagels especially for you. Then I fried some
sausages. I went out to see my chickens, they
had some delicious fresh eggs waiting for me
and I scrambled them just for you. Come on over
and let's eat breakfast!
"""
This shout is over our 180-character limit. After a quick chat with Paula, we’ve confirmed she wants to call this an over-long message. Let’s update the rule we wrote up earlier to document this evolution of our domain language:
Rules:
* Mention the word "buy" and you lose 5 credits.
* Over-long messages cost 2 credits
Then we can use that term in a new, more abstract step:
And Sean shouts an over-long message
Again, we’ll make a step definition that duplicates the shouting code, and this time make a 181-character message, just long enough to be over the 180 character limit.
When "Sean shouts an over-long message" do
base_message = "A message from Sean that is 181 characters long "
message = base_message + "x" * (181 - base_message.size)
@people["Sean"].shout(message)
@messages_shouted_by["Sean"] << message
end
Again, we have another step we can replace with this more abstract one:
And Sean shouts an over-long message
7.4.4. Remove pointless steps
Right. With the incidental details removed from these steps, it’s starting to become easier to see what’s going on. It would be even easier if we re-ordered them so they’re grouped together. Let’s do that:
Scenario: Test premium account features
Given Sean has bought 30 credits
When Sean shouts a message
And Sean shouts a message
And Sean shouts a message
And Sean shouts a long message
And Sean shouts an over-long message
And Sean shouts an over-long message
And Sean shouts a message containing the word "buy"
And Sean shouts a message containing the word "buy"
And Sean shouts a message containing the word "buy"
Then Lucy hears all Sean's messages
And Sean should have 11 credits
So now we can clearly see that Sean shouts three regular messages, a long message, two over-long messages, and three messages containing the word “buy”. It’s still a lot to digest, but a clearer picture is starting to emerge.
Are there any other incidental details remaining in this scenario? In BRIEF terms, is every step here essential to the purpose of this particular scenario?
Arguably, the steps that create regular and long messages are incidental, since they have no bearing on the behaviour we’re describing with this scenario. We can remove them altogether, and the behaviour should be exactly the same. So let’s do that:
Scenario: Test premium account features
Given Sean has bought 30 credits
When Sean shouts an over-long message
And Sean shouts an over-long message
And Sean shouts a message containing the word "buy"
And Sean shouts a message containing the word "buy"
And Sean shouts a message containing the word "buy"
Then Lucy hears all Sean's messages
And Sean should have 11 credits
That’s better.
Now that we’ve removed all that incidental detail, it’s much easier to see how the figure of 11 credits has been calculated: two over-long messages at 2 credits per message, and three messages containing the word “buy” at 5 credits each makes a total of 19 credits, which subtracted from the initial 30 makes 11.
But we can make this even clearer.
7.4.5. Merge repeated steps
These repetitive steps aren’t necessary. They make this scenario look more like a computer program than a specification document. Instead of repeating the step “When Sean shouts an over-long message” over and over again, let’s just tell it like it is: 🎬 32: Replaces two steps with new step: `When Sean shouts 2 over-long messages`
When Sean shouts 2 over-long messages
We’ll need a new step definition for this one.
We can re-use the existing step definition, we just need to run a loop around it, based on the number of messages specified in the scenario.
When "Sean shouts {int} over-long messages" do |count|
count.times do
base_message = "A message from Sean that is 181 characters long "
message = base_message + "x" * (181 - base_message.size)
@people["Sean"].shout(message)
@messages_shouted_by["Sean"] << message
end
end
Let’s do the exact same thing with these other three steps.
And Sean shouts 3 messages containing the word "buy"
Again, we have the bulk of this code that we can just recycle. We just need to capture the count, then add add a loop around the outside.
When 'Sean shouts {int} messages containing the word {string}' do |count, word|
count.times do
message = "A message containing the word #{word}"
@people["Sean"].shout(message)
@messages_shouted_by["Sean"] << message
end
end
And everything is still green. 🎬 39
Let’s step back and see how BRIEF our scenario reads now:
Scenario: Test premium account features
Given Sean has bought 30 credits
When Sean shouts 2 over-long messages
And Sean shouts 3 messages containing the word "buy"
Then Lucy hears all Sean's messages
And Sean should have 11 credits
Nice!
7.4.6. Lesson 4 - Questions
What are the details in this step?
When Sean shouts "Come and buy a coffee"
-
Sean’s name (Correct)
-
the fact that Sean is shouting
-
the text "Come and buy a coffee" (Correct)
Explanation:
Sean’s name, like the text "Come and buy a coffee" are both details in this step. Note that details don’t always have to be surrounded by quotes. The action, shouts is not really what we’d call a detail.
Which words in the phrase "Come and buy a coffee" are essential to this scenario?
-
Come
-
and
-
buy (Correct)
-
a
-
coffee
Explanation:
The rule we’re illustrating is that a shout containing the word "buy" costs 5 credits. All the other words in the message are incidental details.
Why did we change the rule "Long messages cost 2 credits" to "Over-long messages cost 2 credits"?
-
It just feels better that way
-
We agreed with our product owner that it was the right term to use (Currect)
-
It’s important to capture our evolving understading of the domain lanaguge (Correct)
-
We have to, otherwise the scenarios will fail.
Explanation:
Gherkin makes a great place to capture your emerging understanding of your team’s Ubuiquitous Language (see Eric Evans’s book, Domain Driven Design). Because it’s version controlled, you can keep refining the words in your scenarios as you get a clearer understanding of the domain.
Why did we run Cucumber each time we changed the scenario?
-
We wanted to capture our latest test results
-
We wanted to catch any mistakes in our refactoring quickly (Correct)
-
We wanted to show our product owner we were making progress
-
It keeps us confident that things are working as before (Correct)
Explanation:
A great benefit of automated tests is that you can run them while you’re refactoring to ensure you haven’t made a mistake. The definition of refactoring is that you’re improving the maintainability without changing the behaviour. This applies to your test code too, and the best way to check the behaviour hasn’t changed is to run the automated tests and check they’re all still passing.
Which of these statements is true about the trade-offs in what we did when refining the essence of the scenario?
-
We had to push some complexity into the step definition code in order to get the scenarios to read how we wanted. (Correct)
-
We made our step definition code simpler and more elegant.
-
We ended up with more step definitions than before. (Correct)
-
We removed some duplication in the step definitions
-
We removed some duplication in the scenario (Correct)
-
The scenario ended up much easier to read (Correct)
-
The scenario had a lot more steps
Explanation:
When refining your scenarios like this, there is a trade-off that you need to add a little bit more complexity to your step definition code. This is why it’s important to have software engineers working on Cucumber tests, because they’ll be confident that they can do this. When Cucumber automation is left to people with less development experience, they may be afraid of pushing additional complexity into the step defintion code, and end up leaving some really ugly scenarios.
The ideal thing is to have testers and developers working together.
7.5. Expressing the rules, capturing questions
Now things are really getting clearer.
There are still a couple of problems with this scenario.🎬 1 One is the name of the scenario. It really doesn’t tell us anything at all.
Scenario: Test premium account features
A great way to name your scenarios is to think about how they named episodes of the TV sitcom series, Friends. Remember? They were all called something like The one where Ross is fine or The one where Phoebe gets married.
Instead of expressing the outcome, we find it’s best to keep that out of our scenario, and just descrbe the context and the action.
So if this scenario were an epside of Friends, what would it be called? This is the one where… well.. the one where Sean shouts some over-long messages and some containing the word “buy”.
Scenario: Sean shouts some over-long messages and some messages containing the word “buy”
Well, this is better than what we had before, but it’s s quite a complicated name isn’t it? Maybe this scenario is doing too much!
…and that’s the second problem.
It’s trying to test both business rules at once.
7.5.1. Split scenario
Normally you’ll want at least one scenario for each business rule. That way, each scenario clearly illustrates the effect of that rule on the system. Sometimes it’s interesting to also document the effect that multiple rules have when they play out at the same time, but as you’ve already experienced, that can quickly get confusing.
You’ll often need more than one scenario to illustrate each rule, but in this case, one scenario for each rule looks like it will be fine for the time being. Let’s split this scenario in two: the one where Sean shouts several over-long messages, and the one where Sean shouts several messages containing the word “buy”.
🎬 3: a shot where we duplicate the original scenario, twice 🎬 4: a shot where we change the `Scenario: ` name of each of the duplicated scenarios to match what I say in the audio, then clean up the steps to match that intent🎬 5
We can tuck each one under a Rule
keyword, 🎬 6: a shot where we add a Rule keyword above each one. removing the rules from the feature file’s free-text description:
🎬 7:
Rule: Mention the word "buy" and you lose 5 credits.
Scenario: Sean some messages containing the word “buy”
Given Sean has bought 30 credits
And Sean shouts 3 messages containing the word "buy"
Then Lucy hears all Sean's messages
And Sean should have 15 credits
Rule: Over-long messages cost 2 credits
Scenario: Sean shouts some over-long messages
Given Sean has bought 30 credits
When Sean shouts 2 over-long messages
Then Lucy hears all Sean's messages
And Sean should have 26 credits
Great. Now the effect of each of these rules is much more clearly documented.
7.5.2. Remove original scenario
We run these new scenarios past Paula and she’s delighted. She really likes how they read. She asks us - why do we still need the original scenario?
Tamsin chimes in and says she has a concern that the rules might conflict with each other in the code somehow. That’s why she likes having a scenario that covers both. As we talk about it, we realise that while we don’t need this one anymore 🎬 10: removes original long scenario🎬 11 there’s a missing scenario - where both rules apply to the same message: The one where there’s a shout that’s both over-long and with the word “buy” in it.
We’d better document that as a question for now.
Questions:
* What about the one where the same message is both over-long and contains the word "buy"
7.5.3. Ask context question
While we have Paula and Tamsin’s attention, we ask a question we know often helps to discover missing scenarios:
Is there another context that would result in a different outcome here?
They both think about this. Tamsin suggests starting Sean out with fewer credits, or shouting lots more messages, so that he runs out of credit. What would happen then?
Good question! It looks like there’s more to this feature than we’d previously thought. We’ll write that down as a question too. We still haven’t even started fixing that bug yet!
Questions:
* What about the one where the same message is both over-long and contains the word "buy"
* What happens if Sean runs out of credits?
7.5.4. Lesson 5 - Questions
We recommend you name your scenarios like which TV series?
-
Cheers
-
The Simpsons
-
Family Guy
-
Friends (Correct)
-
Brooklyn 99
Explanation:
"The one where… " is a great way to frame the name of each of your scenarios.
Which of these scenario names follows the advice we gave you in this episode?
-
Scenario: The one where the user logs in
-
Scenario: User logs in (Correct)
-
Scenario: User tries to log in with the wrong password (Correct)
-
Scenario: User tries to log in with the wrong password three times (Correct)
-
Scenario: User tries to log in with the wrong password three times and gets locked out
Explanation:
You don’t actually put the words "The one where" into the scenario title - it would appear everywhere otherwise! Just imagine that it’s there.
We suggest you avoid putting the outcome into your scenario name, and instead just describe the context and action.
How many scenarios should you normally have per business rule?
-
None
-
Exactly one
-
At least one (Correct)
Explanation:
Rules generalise examples, so each rule needs one or more examples to illustrate it.
In this episode we discovered two potentially missing scenarios. Why?
-
We’re constantly learning about the problem domain, and working together on the Gherkin helped us to discover some new scenarios we hadn’t previously considered. (Correct)
-
Tamsin the tester has been absent from work and wasn’t doing any testing.
-
We wanted to delete a scenario, and discussed the idea with our colleage who had an objection. Listening to that objection lead us to a new insight. (Correct)
-
Paula the product owner has been too busy to pay attention to what we were doing.
-
We asked the question "Is there another context that would result in a different outcome here" (Correct)
Explanation:
Software development is a constant process of discovery. When we can embrace this ignorance, instead of trying to pretend we know everything, and we listen to each other’s perspetives, we can get insights about gaps in our knowledge, helping us to see things we hadn’t considered before.
7.6. Conclusion
Well that’s been quite a session. We came in to fix one bug, and ended up having to fix our feature file first. We’re still not done with that, either. By pushing the incidental details down out of the scenario into the step definitions, we’ve made a mess of duplication in the step definitions. There’s a neat fix for that, but we’ll have to save that for next time.
Let’s reflect on what we’ve learned.
When scenarios are very heavy in detail, they can be confusing to read. We call these incidental details when they are just noise that detracts from communicating the essence of the scenario.
Scenarios that are heavy in detail are sometimes said to be written in an imperative style. They contain lots of “how”, and not much “what”. We often use this style when we’re working on a brand new domain problem and are still grasping for an understanding. We just want to write something down.
As your confidence in your domain knowledge improves, you’d expect to feel comfortable removing some of these details. Scenarios with more abstract steps like this are said to be written in a declarative style.
Using a more declarative style might mean you’ll need to find names for things, like the way we had to name an over-long message. This discovery of your ubiquitous domain language is a great side-effect of distilling the essence of your scenarios. Now you have a bigger vocabulary that you can use all the way down through your code, and in your future conversations, too.
One down-side, as you’ve seen, of pushing the “how” down from your scenarios into the step definitions is that the step definition code becomes more complex. Some teams reject this, and prefer to use a simple vocabulary of step definitions, leaving more detail in their scenarios. There are no right and wrong answers here, but if you have sufficient competence with code, you’ll easily be able to handle that extra bit of complexity and keep readability top of your priorities.
That’s what we’ll work on next time. Bye for now!
8. Example Mapping
In the last lesson we saw how easily incidental details can creep into your scenarios, talked about why they’re a problem, and showed you some techniques for massaging them back out again. But, as we pushed the details out of our scenarios, we made the step definition code more complicated. We promised to show you how to deal with that extra complexity, and we’re going to get to that in the next chapter, Chapter 9.
First though, we want to look at how we could have prevented the Premium Accounts feature from getting into such a mess in the first place.
We’re going to learn about a practice called Example Mapping, a way to structure the conversation between the Three Amigos - Tester, Developer and Product Owner - to develop shared understanding before you write any code.
8.1. Example Mapping: Why?
🎬 2: BDD circles diagram In terms of the three practices we introduced in Chapter 1 - Discovery, Formulation and Automation - what went wrong with the Premium Accounts feature?
Thinking about it, we can see that the development team 🎬 3: BDD circles diagram - (1) pointing to Automation jumped straight into Automation - writing the implementation of the feature. They did the bare minimum of 🎬 4: BDD circles diagram - (2) pointing to Formulation Formulation - just enough to automate a test for the feature, but really we did a lot of the Formulation later on as we cleaned it up. Finally, much of the 🎬 5: BDD circles diagram - (3) pointing to Discovery Discovery happened at the end once Tamsin had a chance to test the feature by hand.
🎬 6: BDD circles diagram - draw a red 'X' over the three arrows So in essence, they did everything backwards.
🎬 7: Animation ending with 'Deliberate Discovery' In software projects, it’s often the unknown unknowns that can make the biggest difference between success and failure. In BDD, we always try to assume we’re ignorant of some important information and try to deliberately discover those unknown unknowns as early as possible, so they don’t surprise us later on.
A team that invests just a little bit extra in Discovery, before they write any code, saves themselves a huge amount of wasted time further down the line.
🎬 8: Animation introducing three people, showing them talking In lesson 1, we showed you an example of the Three Amigos - Tester, Developer and Product owner - having a conversation about a new user story.
Nobody likes long meetings, so we’ve developed a simple framework for this conversation that keeps it quick and effective. We call this 🎬 9: Example Mapping Example Mapping.
An Example Mapping session takes a single User Story as input, and aims to produce four outputs:
-
Business Rules, 🎬 11: diagram: → Rules that must be satisified for the story to be implemented correctly
-
Examples 🎬 12: diagram: → Examples of those business rules playing out in real-world scenarios
-
Questions or Assumptions 🎬 13: diagram: → Questions that the group of people in the conversation need to deal with soon, but cannot resolve during the immediate conversation
-
New User Stories 🎬 14: diagram: → New User Stories sliced out from the one being discussed in order to simplify it.
We capture these, as we talk, using index cards, or another virtual equivalent.
Working with these simple artefacts rather than trying to go straight to formulating Gherkin, allows us to keep the conversation at the right level - seeing the whole picture of the user story without getting lost in details.
8.1.1. Lesson 1 - Questions
Why did we say the development team’s initial attempt at the premium accounts feature was "done backwards"?
-
They did Discovery before Automation
-
They did Discovery before Formulation
-
They started with Automation, without doing enough Discovery or Formulation first (Correct)
-
They started with Discovery, then did Formulation and finally Automation
Explanation:
The intended order is Discovery, Formulation then Automation. Each of these steps teaches us a little more about the problem.
Our observation was that the the team jumped straight into coding (Automation), retro-fitting a scenario later. The discovery only happened when Tamsin tested the feature.
What does "Deliberate Discovery" mean (Multiple choice)
-
One person is responsible for gathering the requirements
-
Discovery is something you can only do in collaboration with others
-
Having the humility to assume there are things you don’t yet understand about the problem you’re working on (Correct)
-
Embracing your ignorance about what you’re building (Correct)
-
There are no unknown unknowns on your project
Explanation:
Deliverate Discovery means we assume that there are important things we don’t yet know about the project we’re working on, and so make a deliberate effort to look for them at every opportunity.
Although we very much encouage doing that collaboratively, it’s not the main emphasis here.
Read Daniel Terhorst-North’s original blog post.
Why is it a good idea to try and slice a user story?
-
Working in smaller pieces allows us to iterate and get feedback faster (Correct)
-
We can defer behaviour that’s lower priority (Correct)
-
Smaller stories are less likely to contain unknown unknowns (Correct)
-
Doing TDD and refactoring becomes much easier when we proceed in small steps (Correct)
-
Small steps help us keep momentum, which is motivating (Correct)
Explanation:
Just like grains of sand flow through the neck of a bottle faster than pebbles, the smaller you can slice your stories, the faster they will flow through your development team.
It’s important to preserve stories as a vertical slice right through your application, that changes the behaviour of the system from the point of view of a user, even in a very simple way.
That’s why we call it slicing rather than splitting.
Why did we discourage doing Formulation as part of an Example Mapping conversation?
-
Trying to write Gherkin slows the conversation down, which means you might miss the bigger picture. (Correct)
-
It’s usually an unneccesary level of detail to go into when you’re trying to discover unknown unknowns. (Correct)
-
Formulation should be done by a separate team
-
One person should be in charge of the documentation
Explanation
This is why we’ve separated Discovery from Formulation. It’s better to stay relatively shallow and go for breadth at this stage - making sure you’ve looked over the entire user story without getting pulled into rabbit holes.
Product Owners and Domain Experts are often busy people who only have limited time with the team. Make the most of this time by keeping the conversation at the level where the team can learn the maximum amount from them.
8.2. Example Mapping: How?
We first developed example mapping in face-to-face meeting using a simple a multi-colour pack of index cards and some pens. For teams that are working remotely, there are many virtual equivalents of that nowadays.
We use the four different coloured cards to represent the four main kinds of information that arise in the conversation.
We can start with the easy stuff: Take a yellow card and write down the name of the story.
Now, do we already know any rules or acceptance criteria about this story?
Write each rule down on a blue card:
They look pretty straightforward, but let’s explore them a bit by coming up with some examples.
Darren the developer comes up with a simple scenario to check he understands the basics of the “buy” rule: "I start with 10 credits, I shout buy my muffins and then I want to buy some socks, then I have zero credits. Correct?"
"Yes", says Paula. 🎬 5: Write up Darren’s example on a green card
Darren writes this example up on a green card, and places it underneath the rule that it illustrates.
🎬 6: Show female character Tammy the tester chimes in: "How about the one where you shout a word that contains buy, like buyer for example? 🎬 7: Show female character with text appearing If you were to shout I need a buyer for my house. Would that lose credits too?"
🎬 8: Show second female character in foreground Paula thinks about it for a minute, and 🎬 9: Second female character opens arms and smiles decides that no, only the whole word buy counts. They’ve discovered a new rule! 🎬 10: Modify text on blue card, add new green card They write that up on the rule card, and place the example card underneath it.
🎬 11: Show male character in foreground, talking Darren asks: "How do the users get these credits? Are we building that functionality as part of this story too?"
🎬 12: Show female character in foreground, talking Paula tells him that’s part of another story, and they can assume the user can already purchase credits. They write that down as a rule too.
This isn’t a behaviour rule - it’s a rule about the scope of the story. It’s still useful to write it down since we’ve agreed on it. But it won’t need any examples. We could also have chosen to use a red card her to write down our assumption.
🎬 14: Female character speaking, blank background; writing appears Still focussed on the “buy” rule, Tammy asks: "What if they run out of credit? Say you start with 10 credits and shout buy three times. What’s the outcome?"
Paula looks puzzled. "I don’t know". She says. I’ll need to give that some thought.
🎬 15: Write out red card: 'What should happen when one runs out of credits? Darren takes a red card and writes this up as a question.
🎬 16: More (empty) cards appearing They apply the same technique to the other rule about long messages, and pretty soon the table is covered in cards, reflecting the rules, examples and questions that have come up in their conversation. Now they have a picture in front of them that reflects back what they know, and still don’t know, about this story.
8.2.1. Lesson 2 - Questions
What do the Green cards represent in an example map?
-
Stories
-
Rules
-
Examples (Correct)
-
Questions or assumptions
Explanation:
We use the green card to represent examples because when we turn them into tests we want them to go green and pass!
What do the Blue cards represent in an example map?
-
Stories
-
Rules (Correct)
-
Examples
-
Questions or assumptions
Explanation:
We use the blue cards to represent rules because they’re fixed, or frozen, like blue ice.
What do the Red cards represent in an example map?
-
Stories
-
Rules
-
Examples
-
Questions or assumptions (Correct)
Explanation:
We use the red cards to represent questions and assumptions because it indicates danger! There’s still some uncertainty to be resolved here.
What do the Yellow cards represent in an example map?
-
Story (Correct)
-
Rule
-
Example
-
Question or assumption
Explanation:
We chose the yellow cards to represent stories in our example mapping sessions, mostly because that was the last colour left over in the pack!
Look at the following example map. Do you think the team is ready to start coding yet?
-
No. There are still a lot of questions to resolve.
-
No. They probably haven’t explored the story enough yet. More conversation needed. (Correct)
-
No. There are too many rules. They should try to slice the story first.
-
Yes. There’s a good number of examples for each rule, and no questions.
Explanation:
When an example map shows only a few cards, and some rules with no examples at all, it suggests that either the story is very simple, or the discussion hasn’t gone deep enough yet.
Look at the following example map. Do you think the team is ready to start coding yet?
-
No. There are still a lot of questions to resolve.
-
No. They probably haven’t explored the story enough yet. More conversation needed.
-
No. There are too many rules. They should try to slice the story first.
-
Yes. There’s a good number of examples for each rule, and no questions. (Correct)
Explanation:
This example map shows a good number of examples for each rule, and no questions. If the team feel like the conversation is finished, then they’re probably ready to start hacking on this story.
Look at the following example map. Do you think the team is ready to start coding yet?
-
No. There are still a lot of questions to resolve. (Correct)
-
No. They probably haven’t explored the story enough yet. More conversation needed.
-
No. There are too many rules. They should try to slice the story first.
-
Yes. There’s a good number of examples for each rule, and no questions.
Explanation:
The large number of red cards here suggests that the team have encountered a number of questions that they couldn’t resolve themselves. Often this is an indication that there’s someone missing from the conversation. It would probably be irresponsible to start coding until at least some of those questions have been resolved.
Look at the following example map. Do you think the team is ready to start coding yet?
-
No. There are still a lot of questions to resolve.
-
No. They probably haven’t explored the story enough yet. More conversation needed.
-
No. There are too many rules. They should try to slice the story first. (Correct)
-
Yes. There’s a good number of examples for each rule, and no questions.
Explanation:
When an example map is wide like this, with a lot of different rules, it’s often a signal that there’s an opportunity to slice the story up by de-scoping some of those rules from the first iteration. Even if it’s not something that would be high enough quality to ship to a customer, you can often defer some of the rules into another story that you can implement later.
8.3. Example Mapping: Conclusions
As you’ve just seen, an example mapping session should go right across the breadth of the story, trying to get a complete picture of the behaviour. Inviting all three amigos - product owner, tester and developer - is important because each perspective adds something to the conversation.
Although the apparent purpose of an example mapping session is to take a user story, and try to produce rules and examples that illustrate the behaviour, the underlying goal is to achieve a 🎬 3: Show shared understanding appearing shared understanding and agreement about the precise scope of a user story. Some people tell us that example mapping has helped to build empathy within their team!
With this goal in mind, make sure the session isn’t just a rubber-stamping exercise, where one person does all the talking. Notice how in our example, everyone in the group was asking questions and writing new cards.
In the conversation, we often end up 🎬 4: show feedback arrow going back to 'User Story' refining, or even slicing out new user stories 🎬 5: show New user Stories to make the current one smaller. Deciding what a story is not - and maximising the amount of work not done - is one of the most useful things you can do in a three amigos session. Small stories are the secret of a successful agile team.
Each time you come up with an example, try to understand what the underlying rule or rules are. If you discover an example that doesn’t fit your rules, you’ll need to reconsider your rules. In this way, the scope of the story is refined by the group.
Although there’s no doubt of the power of examples for exploring and talking through requirements, it’s the rules that will go into the code. If you understand the rules, you’ll be able to build an elegant solution.
🎬 6: David West / Object Thinking quote As Dr David West says in his excellent book "Object Thinking", If you problem the solution well enough, the solution will take care of itself.
Sometimes, you’ll come across questions that nobody can answer. Instead of getting stuck trying to come up with an answer, just write down the question. 🎬 8: show questions / assumptions as output
Congratulations! You’ve just turned an unknown unknown into a known unknown. That’s progress.
Many people think they need to produce formal Gherkin scenarios from their three amigos conversations, but in our experience that’s only occasionally necessary. In fact, it can often slow the discussion down.
The point of an example mapping session is to do the discovery work. You can do formulation as a separate activity, next.
One last tip is to run your example mapping sessions in a timebox. When you’re practiced at it, you should be able to analyse a story within 25 minutes. If you can’t, it’s either too big, or you don’t understand it well enough yet. Either way, it’s not ready to play.
At the end of the 25 minutes, you can check whether everyone thinks the story is ready to start work on. If lots of questions remain, it would be risky to start work, but people might be comfortable taking on a story with only a few minor questions to clear up. Check this with a quick thumb-vote.
8.3.1. Lesson 3 - Questions
Which of the following are direct outcomes you could expect if your team starts practcing Example Mapping?
-
Less rework due to bugs found in your stories (Correct)
-
Greater empathy and mutual respect between team members (Correct)
-
Amazing Gherkin that reads really well
-
Smaller user stories (Correct)
-
A shared understanding of what you’re going to build for the story (Correct)
-
More predicatable delivery pace (Correct)
-
A quick sense of whether a story is about the right size and ready to start writing code. (Correct)
Explanation:
We don’t write Gherkin during an example mapping session, so that’s not one of the direct outcomes, though a good example mapping session should leave the team ready to write their best Gherkin.
Which of the following presents the most risk to your project?
-
Unknown unknowns (Correct)
-
Known unknowns
-
Known knowns
Explanation:
In project management, there are famously "uknown unknowns", "known unknowns" and "known knowns". The most dangerous are the "unknown unknows" because not only do we not know the answer to them, we have not even realised yet that there’s a question!
9. Support Code
In Chapter 7 we refined the Gherkin of the Premium Accounts feature, turning what had started out as nothing more than an automated test into some valuable documentation.
As we did that, we pushed the "how" down, making the scenarios themselves more declarative of the desired behaviour, pushing the implementation details of the testing into the code in the step defintions below.
In doing this, we got more readable, maintainable and useful scenarios in exchange for more complex automation code. In this chapter we’ll show you how to organise your automation support code so that you won’t be afraid of making this trade-off.
9.1. Problem and Solution Domains
When we build software, we’re always working across two domains.
There’s the problem domain, where our customers and business stakeholders live, and there’s a solution domain, where we solve those business problems using technology.
Each domain has its own jargon, it’s own dialect. That’s fine: specialized language helps domain experts to communicate. Often though, this jargon can prevent us from understanding one another across the two domains.
As BDD practitioners, we’re focussed on trying to grow this area in the middle, where we have a common or ubiquitous language. We know that the bigger this shared vocabulary is, the quicker the team can communicate ideas between the business and technology-facing sides of the team.
We’ve also heard it said that if you model the problem well enough, the solution will take care of itself.
Certainly, we believe that the better an understanding you have of the problem domain, the better a solution you can build. That’s why Cucumber is so powerful, because it helps you to stay rooted in the problem domain.
9.1.1. Lesson 1 - Questions
What are the two domains we described in this chapter?
-
Problem domain (Correct)
-
Test domain
-
Solution domain (Correct)
-
Data domain
-
User interface domain
-
Core domain
Explanation:
While the word "domain" is applicable in lots of ways when talking about software, the two we’re focussed on in this lesson are the problem domain, where our users and customer live, and the solution domain where our development teams and their technical tools live.
Imagine we’re building a system for a healthcare provider. Which of these concepts are likely from the solution domain?
-
Java (Correct)
-
Patient
-
Database (Correct)
-
Doctor
-
Message broker (Correct)
-
Flight
-
Medicine
-
CSS class (Correct)
-
Checkup
-
JavaScript (Correct)
-
Airport
-
Blood pressure
-
XML (Correct)
-
Bed
-
JSON (Correct)
-
Air miles
-
Availability
-
REST endpoint (Correct)
-
Waiting list
Explanation:
Words matter. You might have found this difficult because you’re someone who works in the solution domain, and you’re also probably someone who’s been to a hospital, so you can speak the language of both of these domains.
Remember that for people who come from the problem domain, these words from the solution domain can be confusing.
Imagine we’re building a system for a healthcare provider. Which of these concepts are likely from the problem domain?
-
Java
-
Patient (Correct)
-
Database
-
Doctor (Correct)
-
Flight
-
Message broker
-
Medicine (Correct)
-
CSS class
-
Checkup (Correct)
-
JavaScript
-
Airport
-
Blood pressure (Correct)
-
XML
-
Bed (Correct)
-
JSON
-
Air miles
-
Availability (Correct)
-
REST endpoint
-
Waiting list (Correct)
Explanation:
Problem domain language is the language you’ll hear people say when they talk about the system from a user’s perspective.
What’s the right term for words that have the same meaning in both the problem and solution domains?
-
Unique language
-
Ubiquitous language (Correct)
-
Programming language
-
Proper language
-
Enterprise domain
-
General language
Explanation:
The term 'ubuquitous language' comes from Eric Evans' brilliant book, Domain Driven Design.
As a consulting who worked on multiple projects, Eric noticed that a common factor between the projects that went well, was that people used the same words for stuff.
So if the business people referred to "Blood pressure" in their conversations, the database table where we stored records of those measurements would also be called BloodPressure
and not something that the engineers made up, like tbl_user_metric_items
9.2. Layers of a Cucumber Test Suite
So where do your feature files sit on this diagram of problem and solution domains?
Well, we hope they sit right in the middle here, and act as the place where the problem and solution domains come together. Someone from either domain should be able to read a feature file and it will make sense to them.
And how about step definitions?
Step definitions are right on the boundary, here, translating between the problem-domain language we use in our Gherkin scenarios and the concrete actions we take in code to pull and poke at our solution.
We want to prevent solution-domain concepts and language from leaking into our Gherkin scenarios to keep them readable. As we saw in the last lesson, when we remove details from scenarios, we trade-off for increased complexity in our step definitions.
So how do we manage this complexity?
A mature Cucumber test suite will have a layer of support code sitting between the step definitions and the system being automated.
This layer of support code literally supports the step definitions by providing an API for automating your application.
We can extract this API from our step definitions. Let’s pick up the shouty codebase from Chapter 8 and show you what we mean.
9.2.1. Lesson 2 - Questions
What language should be used in Feature files?
-
Problem domain language (Correct)
-
Solution domain language
-
Language from both problem and solution domains
Explanation:
Feature files should try to avoid using language from the solution domain, because that will exclude anyone who doesn’t write code from being able to read them and give you feedback.
It can sometimes be hard, because we have to really learn the way our users and customers talk about their problems. But in the end that’s one of the most beneficial things you can do for your project.
Examine the following scenarios. Which is using more problem domain language?
Scenario A:
Given database table "tblTasks" has data "Pay rent,Fetch milk"
When I visit "/"
And I click "ui.todos input[type='checkbox' value='Pay rent']"
Then there should be 1 item in "ul.done input[type='checkbox' value='Pay rent']"
Then there should be 1 item in "ul.not-done input[type='checkbox' value='Fetch milk']"
Scenario B:
Given a task "Pay rent"
And a task "Fetch milk"
When I complete the task "Pay rent"
Then the task "Pay rent" should be complete
Then the task "Fetch milk" should be incomplete
-
Scenario A is using problem domain language
-
Secnario B is using problem domain language (Correct)
Explanation:
The first scenario has details like the name of a database table, a browser URL, some CSS selectors. All of these are solution domain artefacts.
When you use problem-domain language in your scenarios, what is the trade-off?
-
You may end up with more complex step definition code behind the features (Correct)
-
You may alienate business people who prefer to read scenarios written in solution-domain language
-
You end up having to write more scenarios
-
Your scenarios can’t be read by everyone on the team anymore
Explanation:
When you really think of your feature files as documentation first, tests second, you will prioritise readability over everything else. As a consequence, sometimes you’ll end up needing to push some complexity down into your step definitions.
That’s why building a support API can be useful, to help you manage this complexity.
The whole reason for using problem-domain language is that your whole team can understand them, meaning they can act as a shared source of truth.
You shouldn’t need any more scenarios just because you use problem-domain language.
9.3. Test all the time
Up until now, we’ve been running Cucumber every time we make a change in order to follow the TDD process. When we find a change that’s too specific to a particular domain model, we usually drop down to the specs for that model and run them individually, again, following the TDD loop.
But, while reworking the code for this lesson, we found out something that we didn’t expect. Our specs were failing, and we didn’t actually notice 🎬 1
bundle exec rspec
FFFF..F.
Failures:
1) Shouty::Network broadcasts a message to a listener within range
Failure/Error: shouter.credits -= (message.scan(/buy/i) || []).size * 5
NoMethodError:
undefined method `credits' for 0:Integer
# ./lib/shouty.rb:48:in `deduct_credits'
# ./lib/shouty.rb:36:in `broadcast'
# ./spec/network_spec.rb:13:in `block (2 levels) in <top (required)>'
2) Shouty::Network does not broadcast a message to a listener out of range
Failure/Error: shouter.credits -= (message.scan(/buy/i) || []).size * 5
NoMethodError:
undefined method `credits' for 0:Integer
# ./lib/shouty.rb:48:in `deduct_credits'
# ./lib/shouty.rb:36:in `broadcast'
# ./spec/network_spec.rb:22:in `block (2 levels) in <top (required)>'
3) Shouty::Network does not broadcast a message to a listener out of range negative distance
Failure/Error: shouter.credits -= (message.scan(/buy/i) || []).size * 5
NoMethodError:
undefined method `credits' for 0:Integer
# ./lib/shouty.rb:48:in `deduct_credits'
# ./lib/shouty.rb:36:in `broadcast'
# ./spec/network_spec.rb:31:in `block (2 levels) in <top (required)>'
4) Shouty::Network does not broadcast a message over 180 characters even if listener is in range
Failure/Error: shouter.credits -= 2 unless short_enough
NoMethodError:
undefined method `credits' for 0:Integer
# ./lib/shouty.rb:47:in `deduct_credits'
# ./lib/shouty.rb:36:in `broadcast'
# ./spec/network_spec.rb:43:in `block (2 levels) in <top (required)>'
5) Shouty::Network broadcasts shouts to the network
Failure/Error: expect(@networkStub).to have_received(:broadcast).with(message, location)
#<Double Shouty::Network> received :broadcast with unexpected arguments
expected: ("Free bagels!", 0)
got: ("Free bagels!", #<Shouty::Person:0x000055a786ad4de0 @messages_heard=[], @network=#<Double Shouty::Network>, @location=0, @credits=0>)
# ./spec/person_spec.rb:30:in `block (2 levels) in <top (required)>'
Finished in 0.01732 seconds (files took 0.17248 seconds to load)
8 examples, 5 failures
Failed examples:
rspec ./spec/network_spec.rb:9 # Shouty::Network broadcasts a message to a listener within range
rspec ./spec/network_spec.rb:18 # Shouty::Network does not broadcast a message to a listener out of range
rspec ./spec/network_spec.rb:27 # Shouty::Network does not broadcast a message to a listener out of range negative distance
rspec ./spec/network_spec.rb:36 # Shouty::Network does not broadcast a message over 180 characters even if listener is in range
rspec ./spec/person_spec.rb:23 # Shouty::Network broadcasts shouts to the network
This was caused by the fact that we were mostly running cucumber and we hadn’t run the specs in a long while.
After dedicating some time to fix the specs 🎬 2
$ bundle exec rspec
..........
Finished in 0.00913 seconds (files took 0.10673 seconds to load)
10 examples, 0 failures
we decided to solve this problem by creating a strategy for running all of our test cases, including both specs and scenarios. This way, we could keep a closer look on the big picture.
In order to do so, we created a Rakefile
🎬 3
$ touch Rakefile
In it, we first added a default task🎬 4 with a command to run rspec 🎬 5, followed by one to run cucumber. 🎬 6
task :default do
sh "bundle exec rspec"
sh "bundle exec cucumber"
end
This works, 🎬 7 except for two small nitpicks:
$ bundle exec rake
bundle exec rspec
..........
Finished in 0.00751 seconds (files took 0.10205 seconds to load)
10 examples, 0 failures
bundle exec cucumber
Using the default profile...
Feature: Premium account
Questions:
* What about the one where the same message is both over-long and contains the word "buy"
* What happens if Sean runs out of credits?
Background: # features/premium_shouts.feature:7
Given the range is 100 # features/step_definitions/steps.rb:10
And Sean is located at 0 # features/step_definitions/steps.rb:14
And Lucy is located at 100 # features/step_definitions/steps.rb:14
Scenario: Sean some messages containing the word “buy” # features/premium_shouts.feature:13
Given Sean has bought 30 credits # features/step_definitions/steps.rb:18
And Sean shouts 3 messages containing the word "buy" # features/step_definitions/steps.rb:46
Then Lucy hears all Sean's messages # features/step_definitions/steps.rb:68
And Sean should have 15 credits # features/step_definitions/steps.rb:72
Feature: Hear shout
Shouty allows users to "hear" other users "shouts" as long as they are close enough to each other.
Scenario: Listener is out of range # features/hear_shout.feature:14
Given the range is 100 # features/step_definitions/steps.rb:10
And Sean is located at 0 # features/step_definitions/steps.rb:14
And Larry is located at 150 # features/step_definitions/steps.rb:14
When Sean shouts # features/step_definitions/steps.rb:22
Then Larry should not hear a shout # features/step_definitions/steps.rb:58
Feature: Premium account
Questions:
* What about the one where the same message is both over-long and contains the word "buy"
* What happens if Sean runs out of credits?
Background: # features/premium_shouts.feature:7
Given the range is 100 # features/step_definitions/steps.rb:10
And Sean is located at 0 # features/step_definitions/steps.rb:14
And Lucy is located at 100 # features/step_definitions/steps.rb:14
Scenario: Sean shouts some over-long messages # features/premium_shouts.feature:20
Given Sean has bought 30 credits # features/step_definitions/steps.rb:18
When Sean shouts 2 over-long messages # features/step_definitions/steps.rb:37
Then Lucy hears all Sean's messages # features/step_definitions/steps.rb:68
And Sean should have 26 credits # features/step_definitions/steps.rb:72
Feature: Hear shout
Shouty allows users to "hear" other users "shouts" as long as they are close enough to each other.
Scenario: Message is too long # features/hear_shout.feature:35
Given the range is 100 # features/step_definitions/steps.rb:10
And Sean is located at 0 # features/step_definitions/steps.rb:14
And Lucy is located at 50 # features/step_definitions/steps.rb:14
When Sean shouts the following message # features/step_definitions/steps.rb:32
"""
This is a really long message
so long in fact that I am not going to
be allowed to send it, at least if I keep
typing like this until the length is over
the limit of 180 characters.
"""
Then Lucy should not hear a shout # features/step_definitions/steps.rb:58
Scenario: Listener is within range # features/hear_shout.feature:7
Given the range is 100 # features/step_definitions/steps.rb:10
And Sean is located at 0 # features/step_definitions/steps.rb:14
And Lucy is located at 50 # features/step_definitions/steps.rb:14
When Sean shouts # features/step_definitions/steps.rb:22
Then Lucy should hear a shout # features/step_definitions/steps.rb:54
Scenario: Two shouts # features/hear_shout.feature:23
Given the range is 100 # features/step_definitions/steps.rb:10
And Sean is located at 0 # features/step_definitions/steps.rb:14
And Lucy is located at 50 # features/step_definitions/steps.rb:14
When Sean shouts "Free bagels!" # features/step_definitions/steps.rb:27
And Sean shouts "Free toast!" # features/step_definitions/steps.rb:27
Then Lucy hears the following messages: # features/step_definitions/steps.rb:62
| Free bagels! |
| Free toast! |
Feature: Premium account
Questions:
* What about the one where the same message is both over-long and contains the word "buy"
* What happens if Sean runs out of credits?
Background: # features/premium_shouts.feature:7
Given the range is 100 # features/step_definitions/steps.rb:10
And Sean is located at 0 # features/step_definitions/steps.rb:14
And Lucy is located at 100 # features/step_definitions/steps.rb:14
@todo
Scenario: BUG #2789 # features/premium_shouts.feature:27
Given Sean has bought 30 credits # features/step_definitions/steps.rb:18
When Sean shouts "buy, buy buy!" # features/step_definitions/steps.rb:27
Then Sean should have 25 credits # features/step_definitions/steps.rb:72
expected: 25
got: 15
(compared using eql?)
(RSpec::Expectations::ExpectationNotMetError)
./features/step_definitions/steps.rb:73:in `nil'
features/premium_shouts.feature:30:in `Sean should have 25 credits'
Failing Scenarios:
cucumber features/premium_shouts.feature:27 # Scenario: BUG #2789
7 scenarios (1 failed, 6 passed)
41 steps (1 failed, 40 passed)
0m0.091s
Randomized with seed 35929
rake aborted!
Command failed with status (1): [bundle exec cucumber...]
/home/fedex/cucumber/content/content/09/code/ruby/00-initial-commit/Rakefile:3:in `block in <top (required)>'
/home/fedex/.rvm/gems/ruby-3.0.2/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/home/fedex/.rvm/gems/ruby-3.0.2/bin/ruby_executable_hooks:22:in `eval'
/home/fedex/.rvm/gems/ruby-3.0.2/bin/ruby_executable_hooks:22:in `<main>'
Tasks: TOP => default
(See full trace by running task with --trace)
The first one is the length of that output.🎬 8 At this point we don’t need this level of detail, we just want to know everything worked. And to focus on that, there’s no better solution than the progress
formatter. 🎬 9
task :default do
sh "bundle exec rspec"
sh "bundle exec cucumber --format progress"
end
Lets see how it looks like 🎬 10
$ bundle exec rake
bundle exec rspec
..........
Finished in 0.00719 seconds (files took 0.10067 seconds to load)
10 examples, 0 failures
bundle exec cucumber --format progress
Using the default profile...
........................................F
(::) failed steps (::)
expected: 25
got: 15
(compared using eql?)
(RSpec::Expectations::ExpectationNotMetError)
./features/step_definitions/steps.rb:73:in `nil'
features/premium_shouts.feature:30:in `Sean should have 25 credits'
Failing Scenarios:
cucumber features/premium_shouts.feature:27 # Scenario: BUG #2789
7 scenarios (1 failed, 6 passed)
41 steps (1 failed, 40 passed)
0m0.054s
Randomized with seed 15282
rake aborted!
Command failed with status (1): [bundle exec cucumber --format progress...]
/home/fedex/cucumber/content/content/09/code/ruby/00-initial-commit/Rakefile:3:in `block in <top (required)>'
/home/fedex/.rvm/gems/ruby-3.0.2/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/home/fedex/.rvm/gems/ruby-3.0.2/bin/ruby_executable_hooks:22:in `eval'
/home/fedex/.rvm/gems/ruby-3.0.2/bin/ruby_executable_hooks:22:in `<main>'
Tasks: TOP => default
(See full trace by running task with --trace)
Great, that’s … shorter. But we still see our failing spec,🎬 11 the one we’re in the middle of fixing. That’s our second nitpick. We don’t want to see unfinished work, we only want to know we haven’t broken anything that was already working before.
If we take a pick at the failing scenario, we can see that it has the @todo
tag.🎬 12 It seems that our team is using that tag as a convention to express that the scenarios marked as @todo
are a Work in Progress. So lets omit this tag from our output. 🎬 13
task :default do
sh "bundle exec rspec"
sh "bundle exec cucumber --tags 'not @todo' --format progress"
end
Now, whenever we run our default rake task, we’re only checking our actual finished work. 🎬 14
$ bundle exec rake
bundle exec rspec
..........
Finished in 0.0076 seconds (files took 0.10149 seconds to load)
10 examples, 0 failures
bundle exec cucumber --tags 'not @todo' --format progress
Using the default profile...
...................................
6 scenarios (6 passed)
35 steps (35 passed)
0m0.046s
Randomized with seed 61983
And it looks much cleaner.
9.3.1. Lesson 3 - Questions
Why do we run the unit tests before the acceptance tests?
-
If the unit tests are failing, it’s unlikely that the acceptance tests will pass (Correct)
-
Unit tests are better than acceptance tests
-
Acceptance tests are always faster so we leave them until the end
Explanation:
We’ve heard it said that acceptance tests help you build the right thing, and unit tests help you build the thing right.
We run the unit tests first to get feedback about whether the pieces of our solution (the units) are fundamentally working. Then, if they are, we can find out whether they play together to deliver the overall system behaviour that we’re looking for.
9.4. Extracting Support Code
We need a new directory to contain our support code. the conventional place to put it is here, in a support
directory underneath features
.🎬 1
mkdir features/support
There’s a special file in support that Cucumber always loads first, called env.rb
.🎬 2 You can use this to boot up the system that you’re testing.
touch features/support/env.rb
In this case, our system is just a domain model, but we can load it here instead of from the step definitions.🎬 3
require 'shouty'
The most obvious duplication in the step definition code is for Sean shouting a message.🎬 4
When "{person} shouts" do |shouter|
shouter.shout("Hello, world")
@messages_shouted_by[shouter.name] << "Hello, world"
end
When "{person} shouts {string}" do |shouter, message|
shouter.shout(message)
@messages_shouted_by[shouter.name] << message
end
When "{person} shouts the following message" do |shouter, message|
shouter.shout(message)
@messages_shouted_by[shouter.name] << message
end
When "{person} shouts {int} over-long messages" do |shouter, count|
count.times do
base_message = "A message from #{shouter.name} that is 181 characters long "
message = base_message + "x" * (181 - base_message.size)
shouter.shout(message)
@messages_shouted_by[shouter.name] << message
end
end
Let’s imagine we had a helper method we could call like this,🎬 5 instead. Wouldn’t that be neater?
When "{person} shouts" do |shouter|
shout from: shouter, message: "Hello, world"
end
Every time Cucumber runs a scenario, it creates a new object, called the World. Each step definition runs inside this instance of the world, almost as though it were a method on that object. We can extend the methods available on the world (and so the methods available to the step defintions) by first defining them on a Ruby module, then registering that module with Cucumber.
We’ll create a new file in the support
directory to contain our extensions to the world.🎬 6
touch features/support/world.rb
Now we define a module,🎬 7
module ShoutyWorld
end
and tell Cucumber to make it part of the world.🎬 8
World(ShoutyWorld)
Finally, we define our new helper method on this module 🎬 9 and paste in the code from the step definition.🎬 10
module ShoutyWorld
def shout(from:, message:)
shouter = from
shouter.shout(message)
@messages_shouted_by[shouter.name] << message
end
end
Let’s run Cucumber to check everything’s still working…🎬 11 Good.
$ bundle exec rake
bundle exec rspec
..........
Finished in 0.00915 seconds (files took 0.11906 seconds to load)
10 examples, 0 failures
bundle exec cucumber --tags 'not @todo' --format progress
Using the default profile...
...................................
6 scenarios (6 passed)
35 steps (35 passed)
0m0.028s
Randomized with seed 22636
Now we can use that new method everywhere…🎬 12
...
When('{person} shouts {string}') do |shouter, message|
shout from: shouter, message: message
end
When "{person} shouts the following message" do |shouter, message|
shout from: shouter, message: message
end
...
and check we haven’t broken anything… done.🎬 13
$ bundle exec rake
bundle exec rspec
..........
Finished in 0.00854 seconds (files took 0.12051 seconds to load)
10 examples, 0 failures
bundle exec cucumber --tags 'not @todo' --format progress
Using the default profile...
...................................
6 scenarios (6 passed)
35 steps (35 passed)
0m0.028s
Randomized with seed 11215
...................................
9.4.1. Move @messages_shouted_by
instance variable behind a helper method
Notice that the helper method we extracted uses that @messages_shouted_by
instance variable.🎬 14
module ShoutyWorld
def shout(from:, message:)
shouter = from
shouter.shout(message)
@messages_shouted_by[shouter.name] << message
end
end
We’re using this instance variable in this assertion step 🎬 15
Then("{person} hears all {person}'s messages") do |listener, shouter|
expect(listener.messages_heard).to match(@messages_shouted_by[shouter.name])
end
This works because the helper methods are mixed into the same World object as the step definitions, so they can all see the same instance variables.
However, we don’t like sharing state across the support API boundary like this. Things can get complicated quickly when you have code spread all over your step definitions and support directory that’s depending on these instance variables.
It’s better to push the state behind the support API.
Let’s extract this variable into a new method on the World 🎬 16 that returns the instance variable,🎬 17 initializing it with an empty hash if it’s never been accessed before.🎬 18
def messages_shouted_by
@messages_shouted_by ||= Hash.new([])
end
Now we can use this method in the step definitions,🎬 19
Then("{person} hears all {person}'s messages") do |listener, shouter|
expect(listener.messages_heard).to match(messages_shouted_by[shouter.name])
end
also in the shout
helper 🎬 20
def shout(from:, message:)
shouter = from
shouter.shout(message)
messages_shouted_by[shouter.name] << message
end
and we no longer need to initialize it in this before hook.🎬 21
Before do
@network = Shouty::Network.new(DEFAULT_RANGE)
end
And run cucumber to check we’re still green.🎬 22
$ bundle exec rake
bundle exec rspec
..........
Finished in 0.00894 seconds (files took 0.11845 seconds to load)
10 examples, 0 failures
bundle exec cucumber --tags 'not @todo' --format progress
Using the default profile...
...................................
6 scenarios (6 passed)
35 steps (35 passed)
0m0.030s
Randomized with seed 62819
9.4.2. Move @network
instance variable behind a helper method
def network
@network ||= Shouty::Network.new(DEFAULT_RANGE)
end
This will enable the {person}
parameter type transformer to pull a network
as soon as it needs one, so we don’t need to create a Network
in a Before
hook anymore.
So we can delete the Before
hook altogether.🎬 25
Before do
@network = Shouty::Network.new(DEFAULT_RANGE)
end
This is changing the behaviour slightly, and for the better. Now we’re not relying on timing - assuming that the hook has created the Network
at a particular point in the scenario’s lifecycle. We just create the Network
as we need it. This is called Lazy loading.
There’s one other place where we’re using the Network
, in this step: 🎬 26
Given "the range is {int}" do |range|
@network = Shouty::Network.new(range)
end
Here, we’re trying to change the range of the network, and the only way to do that at the moment is to create a brand new instance of Network
. This means we have another leak of the @network
instance variable into the step definitions.
This way of doing it is risky. If someone used this step in the middle of a scenario that already had some Person
instances subscribed to a Network
, we’d get some confusing behaviour as the @network
would be overwritten with a new instance with nobody subscribed to it.
All the way through this lesson, we’ve been pushing the how down, moving the implementation details down from step defintions into our automation support code. Here, we can move the implementation right down into our domain model.
Imagine we had a range
setter method on our Network
that allowed us to modify the range on the fly. Then, our step would just look like this: 🎬 27
Given "the range is {int}" do |range|
network.range = range
end
Isn’t that better?
Let’s do it!
First, we’ll add a unit test for the Network
class: 🎬 28
it "can change the range" do
sean = Shouty::Person.new("Sean", network, 0)
laura = Shouty::Person.new("Laura", network, 10)
network.broadcast(message, sean)
expect(laura.messages_heard).to eq [message]
network.range = 5
network.broadcast(message, sean)
expect(laura.messages_heard).to eq [message]
end
First we’ll create a couple of people on the Network, which defaults to a range of 100, so they’re going to be close enough to hear each other at the moment.🎬 29
We’ll broadcast a shout from Sean, which Lucy should be able to hear.🎬 30
Now, we can try setting the range to something much shorter, like 5,🎬 31 so that Sean will now be out of range.
Let’s run the test, and watch it fail like good TDD practitioners.🎬 34
$ bundle exec rspec
....F......
Failures:
1) Shouty::Network can change the range
Failure/Error: network.range = 5
NoMethodError:
undefined method `range=` for #<Shouty::Network:0x000000013b01bc10 @range=100, @listeners=[#<Shouty::Person:0x000000013b01bbc0 @name="Sean", @messages_heard=["Free bagels!"], @network=#<Shouty::Network:0x000000013b01bc10 ...>, @location=0, @credits=0>, #<Shouty::Person:0x000000013b01bb20 @name="Laura", @messages_heard=["Free bagels!"], @network=#<Shouty::Network:0x000000013b01bc10 ...>, @location=10, @credits=0>]>
# ./spec/network_spec.rb:53:in `block (2 levels) in <top (required)>'
Finished in 0.01381 seconds (files took 0.13762 seconds to load)
11 examples, 1 failure
Failed examples:
rspec ./spec/network_spec.rb:46 # Shouty::Network can change the range
Now we can add the attribute writer:🎬 35
class Network
attr_writer :range
...
Let’s run RSpec again, and everything should be passing.🎬 36 Great!
$ bundle exec rspec
...........
Finished in 0.00935 seconds (files took 0.12129 seconds to load)
11 examples, 0 failures
Now we can use that new method in our step: 🎬 37
Given "the range is {int}" do |range|
network.range = range
end
Let’s run all the tests to make sure everything’s working again: 🎬 38
bundle exec rake
Great!
9.4.3. Lesson 4 - Questions
Where can you use a World object?
-
In your step definitions (Correct)
-
In a custom parameter type (Correct)
-
In a
Before
/After
hook (Correct) -
In your application code
Explanation:
Support code, like the World, is created by Cucumber and made available to your step definitions, parameter types and hooks. It doesn’t make sense to have your application code depend on it.
Why would we put code into a World object?
-
Move complexity out of our step definitions (Correct)
-
Hide duplication
-
Share code (Correct)
-
Increase the readability of our step definitions (Correct)
-
Optimizing performance
-
To share state between steps (Correct)
-
Reduce the coupling from the step defintions to the system under test (Correct)
Explanation:
There are many reasons to move code into a support layer!
Adding fields to store state allows us to pass context between steps, so we can make them feel more natural to read.
If we have step definitions spread across multiple files, they can all access the same fields and methods on a shared World object.
Creating an API of methods on the World for interacting with our system makes the step definitions simpler, have less duplication between them, and easier to read. You’ll end up being happy to have many similar step definitions - to match the wording you want to use in your scenarios - because they all delegate to the same underlying code.
When is the state of the World reset?
-
We get a brand new instance of the World every time a step definition is invoked
-
We get one new instance for every scenario (Correct)
-
We get the same instance for the whole test suite
Explanation:
Cucumber will create new instances of the World for each scenario. Every step defininition, parameter type or hook will receive the same instance during that scenario.
9.5. Organinizing step definitions into multiple files
As your test suite grows, you can end up with hundreds of step defintiions, and keeping them all in a single file becomes a maintenance nightmare.
A great benefit of delegating to a support layer is that we can start to organise our step definitions into multiple files, with any step function being able to access the same state and methods on the World, or other support layer objects.
To show you how this works, we can move all the steps to do with shouting over to a separate class.
First, create a file called shout_steps.rb
🎬 1
When "{person} shouts" do |shouter|
shout from: shouter, message: "Hello, world"
end
When('{person} shouts {string}') do |shouter, message|
shout from: shouter, message: message
end
When "{person} shouts the following message" do |shouter, message|
shout from: shouter, message: message
end
When "{person} shouts {int} over-long messages" do |shouter, count|
count.times do
base_message = "A message from #{shouter.name} that is 181 characters long "
message = base_message + "x" * (181 - base_message.size)
shout from: shouter, message: message
end
end
When '{person} shouts {int} messages containing the word {string}' do |shouter, count, word|
count.times do
message = "A message containing the word #{word}"
shout from: shouter, message: message
end
end
And run Cucumber 🎬 3 to check everything’s still working. Great.
Breaking apart your step definitions into multiple files is more of an art than a science, but we can give you a couple of pieces of advice. We’ve seen teams try to organise their step definitions by feature so that, for example, the premium accounts feature would have its own steps file. We find that tends to work out badly, since while feature files live in the problem domain, step definitions are closer to the solution domain. Usually multiple features will need to carry out common actions against the system (think of logging in, for example), so this leaves us with low cohesion: step definitions that sit together in the same file, but don’t have much in common with each other.
We find it’s better to organise your steps by which part of the system they act on. This is a bit hard to illustrate in a tiny solution like Shouty, but bear this in mind when you’re working on your own projects.
We could continue to move code from our step definitions into the support layer. For example, we could write a support method for generating messages of a certain length 🎬 4: Highlight lines 15-16,
When "{person} shouts {int} over-long messages" do |shouter, count|
count.times do
base_message = "A message from #{shouter.name} that is 181 characters long "
message = base_message + "x" * (181 - base_message.size)
shout from: shouter, message: message
end
end
or a method for asserting that a person has heard certain messages. 🎬 5: Highlight lines 22-24
Then "{person} hears the following messages:" do |listener, expected_messages|
actual_messages = listener.messages_heard.map { |message| [ message ] }
expected_messages.diff!(actual_messages)
end
Ideally, each step definition 🎬 6 should only contain one or two lines that delegate to your support code.
Given('{person} has bought {int} credits') do |person, credits|
person.credits = credits
end
When you follow this principle, your step definitions become a translation layer from plain language into code. By keeping the vocabulary consistent as you move across the problem-solution boundary, you start to allow the scenarios to drive the design of your domain model. This is what we call modelling by example.
In this way, we create our own API for automating our application. As this API grows, it becomes easier and easier to write new step definitions, because the actions you need to take are already defined on the API.
It might seem like over-engineering on our little Shouty application, but you’ll be surprised how quickly these test suites grow. Taking time to stamp out complexity early and organise your code to create a good support API is a great investment for the future.
9.5.1. Lesson 5 - Questions
What is a good heuristic for breaking up step definitions into multiple files?
-
Name each step definition file after the feature where the steps are used
-
No more than 2 step definitions in any file
-
Have three files: one for all the `Given`s, one for all the `When`s, and one for all the `Then`s
-
Keep them all in the same file until you see patterns emerge (Correct)
-
Try to maintain cohesion by keeping steps that act on the same part of the system in the same file (Correct)
Explanation:
It’s hard to give you a rule of thumb for how many step definitions should be in the same file. What’s more important is to look for how cohesive they are: if there’s a cluster of steps that are all acting on the same fields or methods in the World, or the same part of the system itself, they probably belong together in one file.
We would always start out by lumping them in to a single file at first, then looking for patterns.
9.5.2. What are some advantages of a well-organised support layer?
-
It makes it easier to use problem-domain language in your scenarios because writing step defintion code is easier (Correct)
-
It keeps step defintions down to one or two lines, so you can easily see whether the language used in the code is consistent with that used in the scenarios (Correct)
-
Business stakeholders will love reading your support code API and giving you feedback on it
-
It enables you to write new scenarios increasingly quickly, since you have invested in the infrastructure for automating your application (Correct)
-
Newcomers should be able to start contributing to your test suite sooner, since complexity is abstracted away into the support layer (Correct)
-
You’ll eliminate the chances of bugs in your test code
Explanation:
Although it would be great to think we can eliminate bugs, this is impossible. Still, the more you apply good software engineering principles (like short methods) to your test code as well as your production code, the easier it will be to maintain.
There’s a chance that, if you’ve really put the work into creating a readable API on your support layer, some of your more technical stakeholders will be able to read it and give you feedback, but that’s really not the main purpose.
By pushing re-usable code down into a support layer, you’re investing in the future by building the scaffolding around your system to make adding more automation easier in the future. Think of it like adding handrails and ladders on a construction site, rather than having people scale the walls with ropes to work on the building.
As you do this, a lovely side-effect is that with neater code it becomes easier to see when you’re using consistent domain language from the scenarios and into the code.
10. Acceptance Tests vs Unit Tests
In the last chapter, we extracted a layer of support code from your step definitions to keep your Cucumber code easy - and cost-effective - to maintain.
We’re going to keep things technical in this chapter. Remember that bug we spotted right back at the beginning of Chapter 7, where the user was over-charged if they mentioned “buy” several times in the same message? It’s finally time to knuckle down and fix it.
As we do so, you’re going to get some more experience of the inner and outer BDD loops that we first introduced you to in Chapter 5. We’ll explore the difference between unit tests and acceptance tests, and learn the value of each.
If you’re someone who doesn’t normally dive deep into code, try not to worry. We think you’ll find it valuable to see how different kinds of tests complement each other in helping you to build a quality product.
10.1. Tidy up the bug report Gherkin
Let’s start by running our failing scenario.
We have it marked with a @todo
tag on it,🎬 2 but those scenarios are filtered out in our default bundle exec rake
command.
@todo
Scenario: BUG #2789
We’re going to start work on the scenario now, so let’s change that tag to a @wip
to indicate that it’s "Work In Progress":
@wip
Scenario: BUG #2789
Given Sean has bought 30 credits
When Sean shouts "buy, buy buy!"
Then Sean should have 25 credits
Now when we run our regular bundle exec rake
command, we should see the failing scenario: 🎬 3
$ bundle exec rake
OK. We’ll work outside-in and start by tidying up the Gherkin specification.
Right now, the scenario is still in the raw form it was in when the bug was first reported, with a name that references an ID in our bug tracking system.🎬 4 This doesn’t make for very good documentation about the intended behaviour.
@wip
Scenario: BUG #2789
Given Sean has bought 30 credits
When Sean shouts "buy, buy buy!"
Then Sean should have 25 credits
It also isn’t sitting under the right rule.
Let’s start by moving it here, under the rule about charging for mentioning the word "buy" 🎬 5: move scenario under first rule
Rule: Mention the word "buy" and you lose 5 credits.
Scenario: Shout several messages containing the word “buy”
Given Sean has bought 30 credits
And Sean shouts 3 messages containing the word "buy"
Then Lucy hears all Sean's messages
And Sean should have 15 credits
@wip
Scenario: BUG #2789
Given Sean has bought 30 credits
When Sean shouts "buy, buy buy!"
Then Sean should have 25 credits
Rule: Over-long messages cost 2 credits
Scenario: Sean shouts some over-long messages
Given Sean has bought 30 credits
When Sean shouts 2 over-long messages
Then Lucy hears all Sean's messages
And Sean should have 26 credits
Now let’s find a better name for the scenario.
Using our Friends Episode naming convention that we introduced in Chapter 7, we could call it something like "(the one where Sean) mentions “buy” several times in one shout"? 🎬 6: change the scenario title🎬 7: run tests
Scenario: Mention "buy" multiple times in one shout
You might be worried about losing this bug ID. We could keep it in a comment 🎬 , a tag 🎬 , or in the description of the Scenario.
Just like under a Feature
keyword, you can write any arbitrary text you want beneath a Scenario
keyword, before the first Given
, When
or Then
keyword.
So let’s do that. 🎬 8: write the bug number in the scenario description
@wip
Scenario: Mention "buy" multiple times in one shout
BUG #2789
We think the values in the example could be changed to make it a little more expressive. If we start Sean off with 100 credits,🎬 9 and end him with 95,🎬 10 it more clearly illustrates the rule that only five credits should be deducted. 🎬 11: run tests
@wip
Scenario: Mention "buy" multiple times in one shout
BUG #2789
Given Sean has bought 100 credits
When Sean shouts "buy, buy buy!"
Then Sean should have 95 credits
OK great, so now we have a failing test we’re more happy with.
The trouble is, although this test tells us what’s wrong with the behaviour of the system - the wrong number of credits are being deducted - it doesn’t give us any clues as to why.
This is why we need a balance of unit tests and acceptance tests: when they fail, acceptance tests tell you what is wrong with the system, but unit tests tell you where you need to go to fix it.
That’s what we’ll look at in the next lesson.
10.1.1. Lesson 1 - Questions
Why did the scenario start running when we changed the tag from @todo
to @wip
-
@wip
is a special tag that Cucumber always runs -
Our setup is configured to filter out scenarios tagged with
@todo
(Correct) -
@wip
is more clear to our business stakeholders -
Cucumber has a known bug with the
@todo
tag
Explanation:
Our setup was configured to filter out scenarios tagged with @todo
. When we changed the tag (we could have changed it to anything else)
that meant it was no longer excluded by the filter.
We like using @wip
so that you can sometimes only run the scenario you’re working on. This is especially useful when you have a lot of scenarios.
Why did we move the scenario under the Rule in the feature file?
-
Each scenario must be under a Rule keyword for Cucumber to be able to run it
-
It makes for better documentation to group the scenario with other scenarios that illustrate the same Rule (Correct)
-
The tests will perform better if we keep the scenarios inside rules
-
We didn’t like having to scroll to the bottom of the file
Explanation:
The Rule:
grouping in a feature file is just for documentation - it doesn’t affect how the scenarios are executed in any way.
You can read more about the relationship between rules and examples in Liz Keogh’s excellent blog post.
Did we need to rename the scenario?
-
No, but it makes for better documentation (Correct)
-
Yes, otherwise the steps would not match
-
Yes, using the word "BUG" always causes Cucumber to fail a scenario
-
No, but we don’t like putting numbers in scenario descriptions (Correct)
Explanation:
Again, the name of a scenario has no bearing on how Cucumber will execute it, but we think it makes for better documentation to talk about the behaviour, rather than just referring to bug numbers or stories in an external issue tracker system.
We moved the bug number to the scenario’s description. Where else could we have put it?
-
Left it in the scenario name (Correct)
-
A comment above the scenario (Correct)
-
A tag (Correct)
-
The first step of the scenario
-
A DataTable
Explanation:
There are many different places in Gherkin we could put references like this. We like using the Description especially because some reporting tools will actually render links, so if you use Markdown in the description you can click a link through to your issue tracker.
Why was it required for us to change the values in the scenario?
-
It wasn’t, but we did it to make it more expressive (Correct)
-
For performance
-
Cucumber would not be able to afford to buy 30 credits
-
To reuse existing step defintions
-
30 credits was too cheap
Explanation:
This change was a minor refinement, and certainly not required. We did it because we always like to think about the person who will find this scenario failing many months from now, and we want to make it as easy as possible for them to grasp what’s wrong. Arbitrary values like "30" and "25" can be distracting.
10.2. Testing ice cream cone
Now that we have tidied up the scenario, we need to find the source of our bug.
We have a passing unit test suite. 🎬 1 So this doesn’t give us any clues as to where the problem lies in the code.
$ rspec
...........
Finished in 0.008 seconds (files took 0.06097 seconds to load)
11 examples, 0 failures
Since we know there’s a bug, we should have at least one unit test failing. So that’s an indication that we might be missing a test. Lets see how we’re testing the mention of the word "buy" in the lower level tests. 🎬 2: Scroll through all the unit tests for the Network class
Oh my! We have absolutely no tests for this feature on our unit test suite! 🎬 3 The only test here about credits is this one for charging for shouts that are over 180 characters. We’ll need to remedy this in order to triage our problem.
context "credits" do
it "deducts 2 credits for a shout over 180 characters" do
long_message = 'x' * 181
sean = Shouty::Person.new("Sean", network, 0)
laura = Shouty::Person.new("Laura", network, 10)
network.subscribe(laura)
network.broadcast(long_message, sean)
expect(sean.credits).to eq(-2)
end
end
This is a common problem we’ve seen time and again in the test suties of teams who adopt Cucumber. In fact, it was a problem on Cucumber’s own codebase in the early days. We were so enamoured by writing new scenarios to describe new behaviour, we used it for everything, and hardly wrote any unit tests. Gradually, we started to realise that this had some significant disadvantages:
-
We had a lot of scenarios, some describing quite obscure edge-cases, making for poor documentation.
-
The tests were slow, because they all ran as full integration tests.
-
When something was broken, there was a lot of code to sift through to try and find the cause of the problem.
-
The internal design of the code wasn’t very modular, because we had no design pressure from unit testing.
There’s a well-known model for how many different types of tests you should have, known as the testing pyramid. It describes this idea that, at the wide base of the pyramid, you should have a large number of unit tests. Then, as you move up the pyramid and it gets wider, you have a smaller number of integration or component tests that exercise subsystems or "chunks" of your application. Finally, at the tip of the pyramid you have a small number of full-stack integation tests.
Floating above the pyramid here is a sweet little fluffy cloud representing the few manual exploratory tests that the team perform on a regular basis.
What we have here is the opposite, which experienced test automation engineer Alister Scott once described as a "testing ice cream cone".
Here, in this anti-pattern, everything is the wrong way around. We have a tiny number of unit tests supporting (or rather failing to support) an excessive number of slower, full-stack integration tests.
Notice that above the ice-cream cone, we have a huge heavy blob of manual regression tests. I’d rather have a real ice-cream!
Many teams find themselves in this situation when they first get into test automation, and it’s not something to get despondent about. It’s possible to shift your distribution of tests toward the pyramid, but it takes deliberate effort and discipline.
In the next lesson we’ll get started on doing that.
10.2.1. Lesson 2 - Questions
What are the characteristics of a test suite that follows the testing ice cream cone anti-pattern?
-
There are a relatively small number of unit tests (Correct)
-
All of the tests run through Cucumber
-
Most of the tests run through the full application stack (Correct)
-
There are almost no integration tests
-
Most of the tests are low-level unit tests
Explanation:
The testing ice-cream-cone represents an inversion of the testing pyramid. This means that you have only a small number of unit tests, and most of the testing is done through full-stack integration tests. You might be using Cucumber for those integration tests, or you might not; that’s not important.
Which is better? Why?
-
The testing ice cream cone is better because ice cream is tasty
-
The testing pyramid is better because each test is at the lowest level possible (Correct)
-
The testing diamond is better because you can never have too many integration tests
-
The testing hourgrlass is better because it’s more balanced, as all things should be in life
Explanation:
The testing pyramid represents a model for the ideal distribution of different types of tests. The idea here is that every behaviour you have in your system should have a low-level unit test, and the key behaviours of bigger chunks of the system should have integration tests to check that those units wire together correctly.
Unit tests tend to be fast, and when they fail they give you a good pointer to the problem because they only cover a small amount of code.
But they’re not enough on their own, because you might have lots of little units that all work OK on their own, but don’t play nicely together. That’s why you need a few integration tests too.
But only a few!
The hourglass and diamond are not really well-known patterns in the industry. It’s certainly possible that your test suite might look like that, but we don’t recomment it!
What are the disadvantages of having a testing ice cream cone?
-
You have so many scenarios, some describing quite obscure edge-cases, that they end up making poor documentation (Correct)
-
The tests run too fast
-
Because so many tests are full integration tests, the whole suite is slow to run (Correct)
-
It’s impossible to debug a failing Cucumber scenario
-
When a test fails, there’s too much code to search through to try and find the cause of the problem (Correct)
-
The internal design of the code tends to be tangled like a plate of spaghetti because there was no need to make it modular for unit testing (Correct)
Explanation:
It’s not impossible to debug a failling Cucumber scenario, but it’s like having a warning light on your car’s dashboard that just says "Something is wrong!". It’s useful for sure to know what something is wrong, but more more useful to have a warning light specifically that the oil is low, or the brakes are not working. Then the mechanic knows what they need to do to fix it.
10.3. Retrofit unit test
Lets add a unit test where Sean shouts a message using the word "buy",🎬 1 and then we assert that five credits have been deducted.🎬 2
it "deducts 5 credits for mentioning the word 'buy'" do
message = 'Come buy these awesome croissants'
sean = Shouty::Person.new("Sean", network, 0, 100)
laura = Shouty::Person.new("Laura", network, 10)
network.subscribe(laura)
network.broadcast(message, sean)
expect(sean.credits).to eq(95)
end
$ rspec
............
Finished in 0.00854 seconds (files took 0.06116 seconds to load)
12 examples, 0 failures
Writing a test that passes the first time it runs gives us no confidence. Remember: Never trust an autommated test that you haven’t seen fail.
We could be adding a spec that tests the wrong thing or that it tests nothing at all and we’d never even realize it, because it "works". So lets force it to fail by commenting the line of code in the solution where we actually deduct the credits. 🎬 5
def deduct_credits(short_enough, message, shouter)
shouter.credits -= 2 unless short_enough
# shouter.credits -= (message.scan(/buy/) || []).size * 5
end
Then we run the unit tests and watch it fail.🎬 6
$ bundle exec rspec
And we can actually see it fails for the right reasons: no credits have been deducted. 🎬 7
Now that we’re confident our unit test is testing the right thing, we can make it pass again by uncommenting the line.🎬 8
def deduct_credits(short_enough, message, shouter)
shouter.credits -= 2 unless short_enough
shouter.credits -= (message.scan(/buy/) || []).size * 5
end
And run the entire suite to make sure everything’s still in place.🎬 9
$ rspec
Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please call `DidYouMean.correct_error(error_name, spell_checker)' instead.
............
Finished in 0.00879 seconds (files took 0.06039 seconds to load)
12 examples, 0 failures
Alright, it seems we’re finally in a position to be able to write a unit test to trap this bug.
10.3.1. Lesson 3 - Questions
Why did we write a test for behaviour that had already been implemented?
-
When we are going to fix the bug, we want to make sure we don’t break existing behaviour (Correct)
-
Our manager wants us to achieve 100% test coverage
-
We’re getting paid by the hour so it gives us something to do to keep busy
-
It will help us move from ice-cream-cone to pyramid (Correct)
Explanation:
If we had been doing TDD, we would have already implemented this test. Still, better late than never as my mum used to say.
Why did we force the test to fail?
-
We don’t trust tests that pass right away: we want to see it fail for the right reason (Correct)
-
We want to see what it will look like when it fails, and how readable the error is (Correct)
-
This is what test-driven development looks like
-
So that we will have an ice-cream cone distribution of tests
Explanation:
This is not quite test-driven development, but it’s as close as we can get when we’re retro-fitting tests for existing behaviour.
It’s really important to see an automated test fail, otherwise you have no confidence that you’ve implemented it correctly, and that it will actually save you from introducing bugs. So deliberately introduce a bug, and see if the test catches it.
10.4. TDD our fix with a unit test
Now that we’ve added a unit test for charging when we mention the word "buy", we can add another one for when we mention it multiple times, which is the bug we were trying to solve in the first place.
We’ll just copy and paste the previous spec and add buy three times.🎬 1
The result should be the same.🎬 2
it "deducts 5 credits even if you mention the word 'buy' several times" do
message = 'Come buy buy buy these awesome croissants'
sean = Shouty::Person.new("Sean", network, 0, 100)
laura = Shouty::Person.new("Laura", network, 10)
network.subscribe(laura)
network.broadcast(message, sean)
expect(sean.credits).to eq(95)
end
Now we make sure it fails.🎬 3
$ rspec
Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please call `DidYouMean.correct_error(error_name, spell_checker)' instead.
.......F.....
Failures:
1) Shouty::Network credits deducts 5 credits even if you mention the word 'buy' several times
Failure/Error: expect(sean.credits).to eq(95)
expected: 95
got: 85
(compared using ==)
# ./spec/network_spec.rb:92:in `block (3 levels) in <top (required)>'
Finished in 0.01892 seconds (files took 0.06003 seconds to load)
13 examples, 1 failure
Failed examples:
rspec ./spec/network_spec.rb:83 # Shouty::Network credits deducts 5 credits even if you mention the word 'buy' several times
With this in place, we can focus on fixing the bug.🎬 4
def deduct_credits(short_enough, message, shouter)
shouter.credits -= 2 unless short_enough
shouter.credits -= 5 if message.match(/buy/)
end
Removing this g
flag means the regular expression won’t match "globally" - so it will return just the first match, rather than all of them.
And run the specs.🎬 5
$ rspec
Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please call `DidYouMean.correct_error(error_name, spell_checker)' instead.
.............
Finished in 0.00798 seconds (files took 0.05958 seconds to load)
13 examples, 0 failures
Well look at that. One character! Just one little extra character caused us to do so much work! Sometimes computers can be so picky!
Aaannyway.
With our unit test suit all green, we’re confident to remove the BUG reference and the @todo
tag.
Scenario: Mention "buy" multiple times in one shout
Given Sean has bought 100 credits
When Sean shouts "buy, buy buy!"
Then Sean should have 95 credits
And we can run the full test suite to make sure everything’s working as expected.🎬 6
$ rake
Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated
. Please call `DidYouMean.correct_error(error_name, spell_checker)' instead.
bundle exec rspec
Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated
. Please call `DidYouMean.correct_error(error_name, spell_checker)' instead.
Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated
. Please call `DidYouMean.correct_error(error_name, spell_checker)' instead.
.............
Finished in 0.00609 seconds (files took 0.06971 seconds to load)
13 examples, 0 failures
bundle exec cucumber --tags 'not @todo' --format progress
Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated
. Please call `DidYouMean.correct_error(error_name, spell_checker)' instead.
Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated
. Please call `DidYouMean.correct_error(error_name, spell_checker)' instead.
Using the default profile...
.........................................
7 scenarios (7 passed)
41 steps (41 passed)
0m0.019s
Randomized with seed 42310
And they all pass.
Fantastic.
10.4.1. Lesson 4 - Questions
Why did we add a test for mentioning the word buy
many times if we already had that tested on our scenarios?
-
Because we don’t really trust our scenarios
-
Because we want to have insurance at the unit test level that everything is workin as we expect (Correct)
Why did we remove the bug reference from our scenario?
-
It’s now outdated, since the bug has been fixed (Correct)
Was it OK to remove the @todo
tag from scenario once we fixed the bug?
-
No, we should have left it in, since we want that specific test to be ignored when we run the full test suite
-
Yes, since it’s not something we need to fix any more (Correct)
-
Yes, since we’re not using the
@todo
tag anymore in our project
Why did we run just unit tests at first and later the whole suite once we fixed the bug?
-
Because we wanted to make sure that the fix was working with a fast suite test first (Correct)
-
Because we like Ice-Cones
-
Because Cucumber won’t run unless we run the unit tests before
10.5. Another edge case
One nice thing about working with unit tests (or microtests as Mike Hill likes to call them) is that they bring your focus down to a very narrow part of the code. Wheras a Cucumber scenario is for zooming out and thinking abou the big picture, a unit test puts a microscope on one tiny part of the system.
Sometimes that helps us to see things we hadn’t anticipated.
Let’s take a look at the deduct_credits
method.🎬 1
def deduct_credits(short_enough, message, shouter)
shouter.credits -= 2 unless short_enough
shouter.credits -= 5 if message.match(/buy/)
end
There’s something on this method that caught our attention: the regex we’re using is case sensitive.🎬 2
shouter.credits -= 5 if message.match(/buy/)
Which raises the question, what should happen if the word has a different capitalization? What if they literally shout "BUY" in capitals?
To answer this question, we have a quick chat with Paula, who confirms that it should have the same effect as if it were lowercased: deduct 5 credits.
So lets first add a spec.🎬 3
it "deducts 5 credits if the word buy is capitalized" do
message = 'Come Buy these awesome croissants'
sean = Shouty::Person.new("Sean", network, 0, 30)
laura = Shouty::Person.new("Laura", network, 10)
network.subscribe(laura)
network.broadcast(message, sean)
expect(sean.credits).to eq(25)
end
Now we run the spec to watch it fail.🎬 6
$ rspec
Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has beeell_checker)' instead.
........F.....
Failures:
1) Shouty::Network credits deducts 5 credits if the word buy is capitalized
Failure/Error: expect(sean.credits).to eq(25)
expected: 25
got: 30
(compared using ==)
# ./spec/network_spec.rb:104:in `block (3 levels) in <top (required)>'
Finished in 0.15783 seconds (files took 0.0838 seconds to load)
14 examples, 1 failure
Failed examples:
rspec ./spec/network_spec.rb:95 # Shouty::Network credits deducts 5 credits if t
Now we write the code that’ll make it pass.🎬 7
shouter.credits -= 5 if message.match(/buy/i)
And finally, we run the entire test suite to watch that beautiful green.🎬 8
$ rspec
Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please call `DidYouMean.correct_error(error_name, spell_checker)' instead.
..............
Finished in 0.01407 seconds (files took 0.08364 seconds to load)
14 examples, 0 failures
Great. So working with unit tests here has helped us to notice and fix another small edge-case. Normally if we were doing test-driven development, we’d have spotted this at the time this code was written, especially if we were doing ensemble programming and had some other folks thinking about and reviewing the code as we wrote it.
10.5.1. Lesson 5 - Questions
If we find an untested edge case we didn’t expect to find in our code. What should we do?
-
Remove the feature
-
Leave the feature in
-
Talk to the product owner to ensure the code is doing what’s expected (Correct)
Once we get an answer, what do we do next?
-
Refactor the code in order to make it more readable
-
Add a test to ensure we don’t have regresions (Correct)
-
File a complaint about the developer who came before us
Is it necessary to run the whole test suite once we know the unit tests are passing with the new feature in?
-
Yes, we always want to make sure we didn’t break something at a distance (Correct)
-
Nah, we only changed a small amount of code
-
No, if the unit tests passed, that’s enough for us
10.6. Re-balancing our test suite
Finally, we can sit down with Paula and Tammy and discuss the fact that we have this edge-case tested both at the unit and feature levels. Do we need to keep both tests?
Since it’s not in the happy path, we decide to remove the scenario and leave the unit test in place.🎬 7
Rule: Mention the word "buy" and you lose 5 credits.
Scenario: Sean some messages containing the word “buy”
Given Sean has bought 30 credits
And Sean shouts 3 messages containing the word "buy"
Then Lucy hears all Sean's messages
And Sean should have 15 credits
Rule: Over-long messages cost 2 credits
Scenario: Sean shouts some over-long messages
Given Sean has bought 30 credits
When Sean shouts 2 over-long messages
Then Lucy hears all Sean's messages
And Sean should have 26 credits
This is a great move from the team, moving the balance back from ice-cream cone towards pyramid. It’s often not easy getting the team to let go of any of their automated tests, especially since unit tests are something that not all team members can easily see and get confidence from.
But it’s the right thing to do. There’s no point wasting the computer’s time testing the same behaviour in two different ways, and it leaves behind more test automation code for the people who’ll have to maintain the system in the future.
Kent Beck, who is famous for popularising test-driven development and has written some great books on it, once said that "I get paid for code that works, not for tests. My philosophy is to test as little as possible to reach a given level of confidence."
It’s important to remember that while we absolutely need automated tests in order to keep our code easy and safe to change in the future, we need as few of them as possible. That means the whole team need to be involved in testing, and deciding where is the best place to connect an automated test in order to give you that confidence. Very often, a unit test will be the better choice, but you have to work together to decide that.
10.6.1. Lesson 6 - Questions
Do we need to keep a scenario AND a unit test that test the same code?
-
Yes, always. This is called double-entry testing
-
No! DRY!
-
We decide by talking it though with our team and assessing the pros and cons of taking or leaving it (Correct)
11. Epilogue
This concludes our epic journey to get you started using Cucumber as it was intended - a tool to help you and your team decide what to build, build it, and maintain it for years to come.
I’ve been working with these techniques for 20 years now, and I’m still learning new stuff every day. So don’t get disheartened if it seems overwhelming sometimes.
There’s a great supportive community of other practitioners waiting for you in our Community Slack and there are a wealth of great books you can pick up for further study.
There’s John Fergusson Smart’s BDD in Action.
There’s Richard Lawrence and Paul Rayner’s book Behavior-Driven Development with Cucumber
And last but definitely not least, there’s Seb Rose and Gaspar Nagy’s series of three, The BDD Books: Discovery, Formulation and Automation.
If you’re keep to see more courses here on Cucumber School on other topics, or you’d just like to give us some feedback on this course, please come into the Slack and let us know. We’d love to hear from you.