In Progress
Unit 1, Lesson 1
In Progress

Code Coverage with SimpleCov

There are lots of reasons to write automated tests for code. One of those reasons is to make sure that we don’t accidentally break working code with our refactorings. But how do we know if the tests are sufficient to detect breakage?

Today, guest chef Ernesto Tagwerker is here to introduce us to the fundamental metric for tests: code coverage. He’ll demo the SimpleCov gem, and show you how you can use it to ensure your tests don’t have blind spots. Enjoy!

Video transcript & code

Calculating Test Coverage with SimpleCov

Script


Today we're going to talk about one of my favorite Ruby gems: SimpleCov

screenshot

SimpleCov is a tool for calculating code coverage. What is code coverage? Basically, it’s a measure of how much of our program’s code is exercised, or “covered”, by tests.

screenshot

Why does this matter? The better our code is covered by tests, the more confidence we can have that any regressions introduced by changes will be caught by those tests.

One place where this can really come in handy is when we’re refactoring: we can find out if the file we’re about to refactor has good code coverage before we start making potentially breaking changes to it!

Let's say we just inherited a new project and we want to find out how much of the code in the application is covered by tests.

screenshot

We are going to use an open source Roda application called gggiiifff.com:

screenshot

We see that the application already has a spec directory with some test scenarios:

That is a great aspect of the project. Many projects you will find in your career will have no tests at all, so you will know right away that code coverage is 0%.

Now we want to know HOW well covered this project really is.

We can start by adding the simplecov gem to the Gemfile.

Considering we don't want to require it every single time we run the test suite,

we make sure that require is set to false.

group :test do
  gem "rspec"
  gem "rack-test"
  gem "capybara"
  gem "simplecov", require: false
end

Great!

Now we want to add a snippet to the spec_helper.rb file to make sure we load SimpleCov before running the entire test suite:

require 'simplecov'
SimpleCov.start do
  add_group "Models", "models"
  add_filter "/spec/"
  add_filter "/config/"
  track_files "**/*.rb"
end

We’ll talk about the meaning of each of these configuration statements a little later.

Now we can go ahead and run the test suite.

When it finishes running our tests, we will see a snippet of text that tells us where SimpleCov placed our code coverage report:

bundle exec rspec spec

Randomized with seed 36108
...
Finished in 0.10308 seconds (files took 0.81455 seconds to load)
5 examples, 0 failures

Randomized with seed 36108

Coverage report generated for RSpec to /Users/etagwerker/Projects/fastruby/gggiiifff.com/coverage. 41 / 44 LOC (93.18%) covered.

Calculating test coverage will make the tests take longer to finish.

So let’s make it optional. We can do this by loading SimpleCov ONLY when we pass an environment variable.

In order to do this we will use an environment variable named COVERAGE:

if ENV["COVERAGE"] == "true"
  require 'simplecov'
  SimpleCov.start do
    add_group "Models", "models"
    add_filter "/spec/"
    add_filter "/config/"
    track_files "**/*.rb"
  end
end

Now when we want to generate the coverage report, we can set that variable to “true” in the command line:

COVERAGE=true bundle exec rspec spec

Randomized with seed 36108
...
Finished in 0.10308 seconds (files took 0.81455 seconds to load)
5 examples, 0 failures

Randomized with seed 36108

Coverage report generated for RSpec to /Users/etagwerker/Projects/fastruby/gggiiifff.com/coverage. 41 / 44 LOC (93.18%) covered.

The cool thing about SimpleCov is that it has a bunch of ways to generate output.

STDOUT is a great way to get a quick glimpse at its overall test coverage.

By default, SimpleCov will generate an HTML report in the coverage/ directory.

We can open it by going to "/path/to/project/coverage/index.html"

We can play around with that report and find what files are least covered.

For instance, you can see that models/query.rb has 100% code coverage:

01-simplecov

That is great news!

You can click on the file, which will show you the source code highlighted with green or red. Considering query.rb is fully covered, every single line in this file is highlighted with green.

02-simplecov

That means that you could refactor this file and you will be covered by automated RSpec tests.

Now let's look at another file that is lacking some code coverage. In the list of files we can see that app.rb is not at 100%:

03-simplecov

When we click on the file's name, we can see that there is one endpoint that is not exercised by tests

04-simplecov

We see that the /gifs/new endpoint is not being tested by Rspec. At this point, we know that if we wanted to refactor that endpoint we should be very careful. Maybe we should write a test for that endpoint before we refactor it.

I strongly recommend you review test coverage of a file BEFORE you start refactoring it. Especially if you are new to it.

So far I showed you how you would set up and run SimpleCov for a Ruby application, but what if you are using Rails?

Well, the good news is that SimpleCov has some special support for you!

You don't need to write so much code to set things up.

You can pass the special string “rails” to the SimpleCov.start command:

require "simplecov"
SimpleCov.start "rails"

When you call SimpleCov like that,

it is as if you were starting SimpleCov with this configuration:

SimpleCov.start do
  add_filter %r{^/config/}
  add_filter %r{^/db/}

  add_group "Controllers", "app/controllers"
  add_group "Channels", "app/channels"
  add_group "Models", "app/models"
  add_group "Mailers", "app/mailers"
  add_group "Helpers", "app/helpers"
  add_group "Jobs", %w[app/jobs app/workers]
  add_group "Libraries", "lib/"

  track_files "{app,lib}/**/*.rb"
end

SimpleCov calls a set of configuration statements like this one a "profile".

track_files specifies a glob of all the files we want to track when we run the test suite.

Filters are a way to tell SimpleCov to ignore all the files that match that regular expression. In Rails, we don't want to know about code coverage in the db and config directories.

A group is a way for SimpleCov to display coverage data for a set of files in a tab section.

In a Rails application, a SimpleCov profile will show us tabs for controllers, channels, mailers, models, jobs, helpers, and libraries.

That way you can get a glimpse of code coverage for each of those "layers".

This is how it might look like:

05-simplecov

Then we can look at the controllers tab, which has a few files that are not covered with tests at all.

06-simplecov

Whether you are building a Ruby or Rails application, automated tests are a great way to have confidence that we’re not accidentally breaking our code’s functionality as we are making changes.

I really like to look at code coverage every time I inherit a new application. It shows me a snapshot of the state of testing in the project. In my professional experience usually the files which have the least coverage are the ones that are harder to maintain.

SimpleCov is a great way to measure code coverage and increase your confidence in the code base. I really hope you can use it in your next refactoring adventure!

Happy hacking!

Responses