In Progress
Unit 1, Lesson 1
In Progress

RSpec Tags

From running only tests related to a specific ticket, to selecting tests based on Ruby version number, RSpec’s test-tagging system makes it easy to slice and dice your test runs along whatever axes you want. In this episode, guest chef Erin Dees will teach you how to use RSpec tags to test exactly what you want, when you want.

Video transcript & code

In Episode 508, we talked about using RSpec's focusing feature to run only the test cases you need, in order to speed up debugging time.

RSpec.describe GameShelf do
  context 'with no games' do
    fit 'is initially empty' do
    end

    it 'remembers the games I add' do
    end
  end
end

At the end of the episode, we saw that focusing on a test with fit is shorthand for tagging it with the focus metadata.

RSpec.describe GameShelf do
  context 'with no games' do
    it 'is initially empty', focus: true do
    end

    it 'remembers the games I add' do
    end
  end
end

This powerful metadata system isn't just for RSpec's implementers. It's available for you to use in your own tests. In this episode, we're going to see how RSpec's metadata can help you run exactly the specs you need.

Tagging Your Specs

Sometimes, you'll have several specs scattered throughout your suite that have something in common. For instance, you might have a subset of integration specs that touch the disk or the network.

To make this idea more concrete, let's turn back to the board game project from Episode 508. As our project grows, we might add the ability to load and save our board game collection on disk.

RSpec.describe GameShelf do
  it 'loads board games from a file' do
    # ...
  end

  it 'saves board games to a file' do
    # ...
  end
end

We'd like to keep track of which specs perform file I/O, so that we can run them all together, or perhaps run everything except them. To do so, let's come up with a tag name that indicates that a particular example or group touches the disk. disk sounds like a good name.

We can set the value of disk to true by including it in a hash after an example's description.

RSpec.describe GameShelf do
  it 'loads board games from a file', disk: true do
    # ...
  end

  it 'saves board games to a file' do
    # ...
  end
end

To set this metadata for all examples in a group, we can add it to a describe or context block instead.

RSpec.describe GameShelf, disk: true do
  it 'loads board games from a file' do
    # ...
  end

  it 'saves board games to a file' do
    # ...
  end
end

As shorthand for setting a metadata value to true, we can just include its name as a symbol with no value attached.

RSpec.describe GameShelf, :disk do
  it 'loads board games from a file' do
    # ...
  end

  it 'saves board games to a file' do
    # ...
  end
end

Running Tagged Examples

Now that we've set metadata on some of our examples, let's see how to use that information when we run our tests. If we start RSpec without any arguments, it will run all the specs in our suite.

$ bundle exec rspec

GameShelf
  with multiple games
    knows my highest-rated game
  with no games
    remembers the games I add
    is initially empty

GameShelf
  loads board games from a file
  saves board games to a file

Finished in 0.20678 seconds (files took 0.10273 seconds to load)
5 examples, 0 failures

But if we run RSpec using the -t or --tag option with the name of our tag, we can tell it to run only the specs that perform disk I/O—in this case, our integration specs.

$ bundle exec rspec --tag disk
Run options: include {:disk=>true}

GameShelf
  loads board games from a file
  saves board games to a file

Finished in 0.20115 seconds (files took 0.09254 seconds to load)
2 examples, 0 failures

If we preface the tag name with a tilde, we can run everything except the disk-related specs, in this case our much faster unit specs. Depending on your shell, you may need to add quotes around the negated tag name.

$ bundle exec rspec --tag '~disk'
Run options: exclude {:disk=>true}

GameShelf
  with no games
    remembers the games I add
    is initially empty
  with multiple games
    knows my highest-rated game

Finished in 0.0037 seconds (files took 0.10296 seconds to load)
3 examples, 0 failures

Running just a subset of your tests like this can be handy when you're making rapid edits back to back, or when you want to do a quick pre-commit test before you push your code up to your build server for a full run.

Tags Can Have Any Values

The tags we've seen thus far (focus and disk) have had Boolean values. You can use any value for a tag though, not just true or false.

For instance, if we're looking into a bug from our issue tracker concerning how we save games to disk, we could tag all the affected examples with a key/value pair containing the bug ID.

RSpec.describe GameShelf,  do
  it 'loads board games from a file' do
    # ...
  end

  it 'saves board games to a file' do
    # ...
  end
end

Then, we can use the same --tag option from before to run just the examples related to the bug we're chasing.

$ bundle exec rspec -t bug:123
Run options: include {:bug=>123}

GameShelf
  saves board games to a file

Finished in 0.10374 seconds (files took 0.09416 seconds to load)
1 example, 0 failures

Including/Excluding Tags Automatically

In these examples, we've used command-line options to select what tags we want to apply to the current test run. That's great for one-offs like working on a specific bug. But sometimes you'll want to use the same set of criteria across many test runs.

For instance, suppose we've got specs that exercise features only available on certain Ruby versions.

RSpec.describe GameShelf do
  it 'loads board games from a file' do
    # ...
  end

  it 'saves board games to a file' do
    # ...
  end

  it 'uses the new Ruby 2.5 IO#pread API for thread safety', min_ruby: '2.5.0' do
    # ...
  end
end

Rather than passing the --tag option to RSpec depending on which Ruby version we're running, we can put that logic into our test configuration.

We want to tell RSpec to exclude any examples that require a newer Ruby version than the one we're running. In our spec_helper.rb file, we can define a setting that will do just that.

RSpec.configure do |config|
  if Gem::Version.new(RUBY_VERSION) < Gem::Version.new('2.5.0')
    config.filter_run_excluding min_ruby: '2.5.0'
  end
end

The filter_run_excluding option will prevent certain examples from running, depending on what metadata they're tagged with. Here, we filter out anything tagged with a minimum Ruby version of 2.5. We only want to apply the filter if we're running on an older Ruby.

As it stands, this configuration only checks for Ruby 2.5 specifically. If we want to be more flexible, we can pass in a Proc object containing whatever arbitrary version logic we want.

First, we wrap the existing comparison inside a lambda

RSpec.configure do |config|
  newer_than_ours = lambda do |value|
    Gem::Version.new(RUBY_VERSION) < Gem::Version.new('2.5.0')
  end

  config.filter_run_excluding min_ruby: '2.5.0'
end

…making sure to use the block parameter in place of our hard-coded Ruby version.

RSpec.configure do |config|
  newer_than_ours = lambda do |value|
    Gem::Version.new(RUBY_VERSION) < Gem::Version.new(value)
  end

  config.filter_run_excluding min_ruby: '2.5.0'
end

Finally, we update our call to filter_run_excluding to pass in our Proc.

RSpec.configure do |config|
  newer_than_ours = lambda do |value|
    Gem::Version.new(RUBY_VERSION) < Gem::Version.new(value)
  end

  config.filter_run_excluding min_ruby: newer_than_ours
end

Now, when we kick off our integration specs with Ruby 2.5, the version-specific example will run alongside all the others.

$ env RBENV_VERSION=2.5.0-dev bundle exec rspec spec/integration
Run options: exclude {:min_ruby=>#<Proc:./spec/spec_helper.rb:21>}

GameShelf
  uses the new Ruby 2.5 IO#pread API for thread safety
  saves board games to a file
  loads board games from a file

Finished in 0.20696 seconds (files took 0.1041 seconds to load)
3 examples, 0 failures

But when we're on an older Ruby, that same example gets filtered out.

$ env RBENV_VERSION=2.4.2 bundle exec rspec spec/integration
Run options: exclude {:min_ruby=>#<Proc:./spec/spec_helper.rb:21>}

GameShelf
  loads board games from a file
  saves board games to a file

Finished in 0.2044 seconds (files took 0.09036 seconds to load)
2 examples, 0 failures

Tagging Examples Automatically

So far, we've been tagging our examples manually, by adding metadata to a describe, context, or it block. As an alternative, you can let RSpec do this clerical work for you.

For our board game project, we realize that all the specs in the integration directory perform file I/O.


├── lib
│   ├── game_shelf.rb
│   └── spreadsheet_store.rb
└── spec
    ├── integration
    │   └── game_shelf_integration_spec.rb
    ├── spec_helper.rb
    └── unit
        └── game_shelf_spec.rb

Instead of manually adding the disk tag to all the spec files in the integration directory, we can write a rule that says that every example in the folder gets the tag.

Inside the config block in our spec_helper.rb file, we add a call to RSpec's define_derived_metadata method.

RSpec.configure do |config|
  config.define_derived_metadata(file_path: %r{spec/integration}) do |meta|
    meta[:disk] = true
  end

This method call has two parts: a set of conditions that say when we set metadata, and a block of code that says what we'll set. In this case, the condition is that we want to tag only examples in the integration directory.

If the file path matches the rule, RSpec runs our block. Inside it, we do the work of actually setting the disk attribute.

We're almost done with our configuration; we need just one final touch. In the future, we may need an exception to the rule that all integration specs perform disk I/O. We could tag any such stray examples with disk: false, but our configuration code will unconditionally overwrite it to true.

To allow for these kinds of edge cases, let's only set the disk attribute for examples that don't already have it set.

RSpec.configure do |config|
  config.define_derived_metadata(file_path: %r{spec/integration}) do |meta|
    meta[:disk] = true unless meta.has_key?(:disk)
  end

With RSpec's metadata system, you can run exactly the specs you need to in order to get the feedback you want. For instance, you could run just your database-related specs, or just your fastest ones, or just the ones related to a particular bug.

In addition to helping you filter your test runs, metadata can help you in a ton of other ways as well. Our book has a whole chapter that dives deep into RSpec's metadata system and all the different ways you can use it to slice and dice your test suite.

Thanks to Avdi and crew for having me on as a guest chef. I hope some of the techniques we've talked about will help you get the most out of your tests.

Happy hacking!

Responses