In Progress
Unit 1, Lesson 21
In Progress

Selectively Run Tests

Do you have a large and long-running test suite?

If that’s the case, in order to maintain a healthy testing rhythm you need ways to selectively run just the tests you’re interested in as you develop new functionality. In this episode, you’ll learn how to do just that, for both RSpec and MiniTest-based suites. You’ll even find out how to do it in the case where your tests are run from a rake task.

Video transcript & code

In a perfect world, we'd all have 100% test coverage and a test suite that ran in under 10 seconds. Very few of us live in that world, however, so it's good to have some techniques in hand for dealing with large, slow-running test suites.

One of the most basic techniques is the ability to limit a test run to just the test or tests you are currently interested in. You may already know how to do this in your test framework of choice, in which case feel free to skip this episode. Otherwise, keep watching…

Let's start with MiniTest, Ruby's built-in unit testing library. Here's a little example test suite with three tests:

require 'minitest/autorun'

class TestArithmetic < MiniTest::Unit::TestCase
  def test_addition
    assert_equal 5, 2 + 3
  end

  def test_addition_is_commutative
    assert_equal 5, 3 + 2
  end

  def test_subtraction
    assert_equal 1, 3 - 2
  end
end

Here's how we run the whole file:

$ ruby arithmetic_test.rb -v
Run options: -v --seed 11757

# Running tests:

TestArithmetic#test_subtraction = 0.00 s = .
TestArithmetic#test_addition_is_commutative = 0.00 s = .
TestArithmetic#test_addition = 0.00 s = .


Finished tests in 0.001039s, 2887.8893 tests/s, 2887.8893 assertions/s.

3 tests, 3 assertions, 0 failures, 0 errors, 0 skips

We can run a single test from this file by specifying the --name command-line option when running the tests.

$ ruby arithmetic_test.rb -v --name=test_addition
Run options: -v --name=test_addition --seed 22782

# Running tests:

TestArithmetic#test_addition = 0.00 s = .


Finished tests in 0.000720s, 1389.7671 tests/s, 1389.7671 assertions/s.

1 tests, 1 assertions, 0 failures, 0 errors, 0 skips

We can also run all tests whose names match a given regular expression by surrounding the argument with slashes:

$ ruby arithmetic_test.rb -v --name=/addition/
Run options: -v --name=test_/addition/ --seed 25108

# Running tests:

TestArithmetic#test_addition_is_commutative = 0.00 s = .
TestArithmetic#test_addition = 0.00 s = .


Finished tests in 0.001212s, 1650.3284 tests/s, 1650.3284 assertions/s.

2 tests, 2 assertions, 0 failures, 0 errors, 0 skips

You might be accustomed to running your tests from a Rake task. You might even have a project where the tests can only be run from a Rake task, because of setup that the Rake prerequisites take care of. We can still use the --name option when running tests from Rake. We just have to pass the options as the value of an environment variable called TESTOPTS.

$ rake test TESTOPTS="--name=test_addition"
/home/avdi/.rvm/rubies/ruby-1.9.3-p327/bin/ruby -I"lib" -I"/home/avdi/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib" "/home/avdi/.rvm/gems/ruby-1.9.3-p327/gems/rake-10
.0.3/lib/rake/rake_test_loader.rb" "./arithmetic_test.rb" --name=test_addition                                                                                         
Run options: --name=test_addition --seed 64879

# Running tests:

.

Finished tests in 0.000567s, 1762.5462 tests/s, 1762.5462 assertions/s.

1 tests, 1 assertions, 0 failures, 0 errors, 0 skips

Note that everything we've used so far should also work for Test::Unit-style tests, since Test::Unit is just a compatibility layer over MiniTest in recent versions of Ruby.

Now let's take a look at the RSpec side of the world. Here's the same test suite, translated to RSpec. Notice that we've explicitly grouped the addition examples into a nested description block.

describe 'Arithmetic' do
  describe 'addition' do
    it 'adds numbers' do
      (2 + 3).should eq(5)
    end

    it 'adds numbers commutatively' do
      (3 + 2).should eq(5)
    end
  end

  describe 'subtraction' do
    it 'subtracts numbers' do
      (3 - 2).should eq(1)
    end
  end
end

To run the whole spec, we use the rspec command with the spec filename as an argument. In this example we're also using the -fs option to format test results in "specdoc" style.

$ rspec -fs arithmetic_spec.rb 

Arithmetic
  addition
    adds numbers
    adds numbers commutatively
  subtraction
    subtracts numbers

Finished in 0.00174 seconds
3 examples, 0 failures

To run a single example by name, we can use the -e option and the name of the example.

$ rspec -fs arithmetic_spec.rb -e 'subtracts numbers'
Run options: include {:full_description=>/subtracts\ numbers/}

Arithmetic
  subtraction
    subtracts numbers

Finished in 0.00128 seconds
1 example, 0 failures

We don't have to pass the whole name of an example; RSpec will also match substrings. This means that we can run all of the examples matching a substring at once.

$ rspec -fs arithmetic_spec.rb -e 'adds'
Run options: include {:full_description=>/adds/}

Arithmetic
  addition
    adds numbers
    adds numbers commutatively

Finished in 0.00159 seconds
2 examples, 0 failures

The names of context and describe blocks are included in the match, so we can specify the name of a describe block instead of a specific example.

$ rspec -fs arithmetic_spec.rb -e 'subtraction'
Run options: include {:full_description=>/subtraction/}

Arithmetic
  subtraction
    subtracts numbers

Finished in 0.0011 seconds
1 example, 0 failures

RSpec also gives us another option for choosing examples to run: we can give it the line number of an example with the -l option.

$ rspec -fs arithmetic_spec.rb -l 8
Run options: include {:line_numbers=>[8]}

Arithmetic
  addition
    adds numbers commutatively

Finished in 0.00164 seconds
1 example, 0 failures

The line number doesn't have to be the first line of the example; it can be any line in the example. This is particularly helpful for writing editor macros to run the test where the cursor is currently located.

$ rspec -fs arithmetic_spec.rb -l 9
Run options: include {:line_numbers=>[9]}

Arithmetic
  addition
    adds numbers commutatively

Finished in 0.00115 seconds
1 example, 0 failures

We can also specify the line number of a describe or context block to run all the examples within that block.

$ rspec -fs arithmetic_spec.rb -l 3
Run options: include {:line_numbers=>[3]}

Arithmetic
  addition
    adds numbers
    adds numbers commutatively

 Finished in 0.00185 seconds
2 examples, 0 failures

As with MiniTest, we may sometimes prefer to run our specs from a Rake task rather than directly using the rspec command.

As with the MiniTest example, to pass our options through to RSpec we need to use an environment variable passed to Rake. In this case, the variable is called SPEC_OPTS.

$ rake spec SPEC_OPTS='-e a
dds'                                                                              
/home/avdi/.rvm/rubies/ruby-1.9.3-p327/bin/ruby -S rspec ./arithmetic_spec.rb -fs
Run options: include {:full_description=>/adds/}

Arithmetic
  addition
    adds numbers
    adds numbers commutatively

Finished in 0.00173 seconds
2 examples, 0 failures

Now you know how to run individual tests and examples using the most popular Ruby testing frameworks. If you've been avoiding the red-green-refactor cycle because running the full suite takes too long, you no longer have that excuse. Happy hacking!

Responses