In Progress
Unit 1, Lesson 1
In Progress

Exception Test

Video transcript & code

Back in episode #296, we saw some RSpec code that tested an error was raised in a particular circumstance. In the comments, Myron Marston made a great point about how I was using RSpec's exception assertions. Today I thought we might return to that code and dig into Myron's words of caution.

require "rspec/autorun"

class Thermostat
  def initialize(thermometer:,furnace:)
    @thermometer = thermometer
    @furnace     = furnace
  end

  def check_temperature
    temp = @thermometer.temp_f
    if temp <= 67
      @furnace.turn_on or fail "Furnace could not be lit"
    end
  end
end

RSpec.describe Thermostat do
  # ...

  it "raises an exception when the furnace fails to light" do
    thermometer = double(temp_f: 67)
    furnace     = double(turn_on: false)
    thermostat  = Thermostat.new(thermometer: thermometer, furnace: furnace)

    expect { thermostat.check_temperature }.to raise_error
  end

end

# >> .
# >>
# >> Finished in 0.001 seconds (files took 0.10382 seconds to load)
# >> 1 example, 0 failures
# >>

Here's how this example works: first, we create a thermometer double object which will return a slightly chilly temperature. Next, we create another test double to represent the furnace. We stub its turn_on method to return false, which in the context of this codebase means that something has gone wrong and the furnace was unable to be lit.

Then we create a Thermostat object to be tested, and inject our test doubles into it. Finally, we start an expectation with a block. Inside the block, we send the #check_temperature message. Then we state that we expect this block to produce an error of some kind.

This test passes. The question is, is it passing for the right reason? And it turns out that when testing errors, this is a particularly important question to ask.

Let's introduce a mistake to the code being tested. We'll send the power_on method instead of the switch_on method. At the same time, we'll remove the code that raises the exception we are testing for.

require "rspec/autorun"

class Thermostat
  def initialize(thermometer:,furnace:)
    @thermometer = thermometer
    @furnace     = furnace
  end

  def check_temperature
    temp = @thermometer.temp_f
    if temp <= 67
      @furnace.power_on
    end
  end
end

RSpec.describe Thermostat do
  # ...

  it "raises an exception when the furnace fails to light" do
    thermometer = double(temp_f: 67)
    furnace     = double(turn_on: false)
    thermostat  = Thermostat.new(thermometer: thermometer, furnace: furnace)

    expect { thermostat.check_temperature }.to raise_error
  end

end

# >> .
# >>
# >> Finished in 0.00179 seconds (files took 0.18533 seconds to load)
# >> 1 example, 0 failures
# >>

The test still passes. Why? Let's remove the expectation and run the test again:

require "rspec/autorun"

class Thermostat
  def initialize(thermometer:,furnace:)
    @thermometer = thermometer
    @furnace     = furnace
  end

  def check_temperature
    temp = @thermometer.temp_f
    if temp <= 67
      @furnace.power_on
    end
  end
end

RSpec.describe Thermostat do
  # ...

  it "raises an exception when the furnace fails to light" do
    thermometer = double(temp_f: 67)
    furnace     = double(turn_on: false)
    thermostat  = Thermostat.new(thermometer: thermometer, furnace: furnace)

    thermostat.check_temperature
  end

end

# >> F
# >>
# >> Failures:
# >>
# >>   1) Thermostat raises an exception when the furnace fails to light
# >>      Failure/Error: @furnace.power_on or fail "Furnace could not be lit"
# >>        Double received unexpected message :power_on with (no args)
# >>      # xmptmp-in4470vam.rb:12:in `check_temperature'
# >>      # xmptmp-in4470vam.rb:25:in `block (2 levels) in <main>'
# >>
# >> Finished in 0.00048 seconds (files took 0.08434 seconds to load)
# >> 1 example, 1 failure
# >>
# >> Failed examples:
# >>
# >> rspec  # Thermostat raises an exception when the furnace fails to light
# >>

Without the expect block to capture it, this code raises an RSpec exception because a test double received a message it didn't expect. This is not the exception we were testing for. But it's still an exception, and that's all that we specified.

Any number of changes to the code could generate an exception and trigger this false pass condition. Or, we could even make a mistake in the test itself and have that produce a passed test for the wrong reasons.

require "rspec/autorun"

class Thermostat
  def initialize(thermometer:,furnace:)
    @thermometer = thermometer
    @furnace     = furnace
  end

  def check_temperature
    temp = @thermometer.temp_f
    if temp <= 67
      @furnace.turn_on
    end
  end
end

RSpec.describe Thermostat do
  # ...

  it "raises an exception when the furnace fails to light" do
    thermometer = double(temp_f: 67)
    furnace     = double(turn_on: false)
    thermostat  = Thermostat.new(thermometer: thermometer, furnace: furnace)

    expect { thermostat.check_temperaturexxx }.to raise_error
  end

end

# >> .
# >>
# >> Finished in 0.00141 seconds (files took 0.12777 seconds to load)
# >> 1 example, 0 failures
# >>

This time, the test passes because the NoMethodError generated by sending an unhandled message is enough to satisfy our assertion

As we can begin to see from these examples, it's very easy to cause an exception assertion to pass by accident. Any exception is enough to satisfy it, and mistakes in coding often produce exceptions.

So how can we make this a more valuable test? We need to make the assertion more specific. One way to do this is by matching on the error message.

By passing a string to raise_error, we can tell RSpec that we expect an exception with that exact error message.

require "rspec/autorun"

class Thermostat
  def initialize(thermometer:,furnace:)
    @thermometer = thermometer
    @furnace     = furnace
  end

  def check_temperature
    temp = @thermometer.temp_f
    if temp <= 67
      @furnace.turn_on
    end
  end
end

RSpec.describe Thermostat do
  # ...

  it "raises an exception when the furnace fails to light" do
    thermometer = double(temp_f: 67)
    furnace     = double(turn_on: false)
    thermostat  = Thermostat.new(thermometer: thermometer, furnace: furnace)

    expect { thermostat.check_temperaturexxx }.to raise_error("furnace could not be lit")
  end

end

# >> F
# >>
# >> Failures:
# >>
# >>   1) Thermostat raises an exception when the furnace fails to light
# >>      Failure/Error: expect { thermostat.check_temperaturexxx }.to raise_...
# >>        expected Exception with "Furnace could not be lit", got #<NoMetho...
# >>          # xmptmp-in4470J2m.rb:25:in `block (3 levels) in <main>'
# >>          # xmptmp-in4470J2m.rb:25:in `block (2 levels) in <main>'
# >>      # xmptmp-in4470J2m.rb:25:in `block (2 levels) in <main>'
# >>
# >> Finished in 0.00198 seconds (files took 0.12233 seconds to load)
# >> 1 example, 1 failure
# >>
# >> Failed examples:
# >>
# >> rspec  # Thermostat raises an exception when the furnace fails to light
# >>

The drawback of this approach is that we've gone from underspecified to over-specified. Let's say that we reinstate the code that is supposed to make this pass.

require "rspec/autorun"

class Thermostat
  def initialize(thermometer:,furnace:)
    @thermometer = thermometer
    @furnace     = furnace
  end

  def check_temperature
    temp = @thermometer.temp_f
    if temp <= 67
      @furnace.turn_onor fail "Furnace could not be lit"
    end
  end
end

RSpec.describe Thermostat do
  # ...

  it "raises an exception when the furnace fails to light" do
    thermometer = double(temp_f: 67)
    furnace     = double(turn_on: false)
    thermostat  = Thermostat.new(thermometer: thermometer, furnace: furnace)

    expect { thermostat.check_temperature }.to raise_error("furnace could not be lit")
  end

end

# >> F
# >>
# >> Failures:
# >>
# >>   1) Thermostat raises an exception when the furnace fails to light
# >>      Failure/Error: $SiB.record_result(25, (expect { thermostat.check_temperature }.to raise_error("furnace could not be lit")))
# >>        expected Exception with "furnace could not be lit", got #<RuntimeError: Furnace could not be lit> with backtrace:
# >>          # xmptmp-in4470WOV.rb:12:in `check_temperature'
# >>          # xmptmp-in4470WOV.rb:25:in `block (3 levels) in <main>'
# >>          # xmptmp-in4470WOV.rb:25:in `block (2 levels) in <main>'
# >>      # xmptmp-in4470WOV.rb:25:in `block (2 levels) in <main>'
# >>
# >> Finished in 0.00249 seconds (files took 0.09082 seconds to load)
# >> 1 example, 1 failure
# >>
# >> Failed examples:
# >>
# >> rspec  # Thermostat raises an exception when the furnace fails to light
# >>

We still get a failure. The reason is that the error message we specified starts with a lowercase letter, and the error message that was actually raised starts with a capital. This is less than ideal. It means that every time we tweak the spelling or punctuation of our error messages, we're going to have to update our tests as well.

Fortunately, we don't have to use a literal string for the match. We can also use a regular expression. Let's replace the string with a regular expression, using the regex option telling it to ignore the case of letters.

require "rspec/autorun"

class Thermostat
  def initialize(thermometer:,furnace:)
    @thermometer = thermometer
    @furnace     = furnace
  end

  def check_temperature
    temp = @thermometer.temp_f
    if temp <= 67
      @furnace.turn_onor fail "Furnace could not be lit"
    end
  end
end

RSpec.describe Thermostat do
  # ...

  it "raises an exception when the furnace fails to light" do
    thermometer = double(temp_f: 67)
    furnace     = double(turn_on: false)
    thermostat  = Thermostat.new(thermometer: thermometer, furnace: furnace)

    expect { thermostat.check_temperature }.to raise_error(/furnace could not be lit/i)
  end

end

# >> .
# >>
# >> Finished in 0.00114 seconds (files took 0.08665 seconds to load)
# >> 1 example, 0 failures
# >>

This version is a big improvement over what we started with. We know that if it passes, it is almost certainly because the right thing happened. And if it fails, it probably will be because we broke something, and not because we made a minor change in error message punctuation or capitalization.

There is another, even more robust way to verify that the right exception is raised. But we'll tackle that on another day. Happy hacking!

Responses