In Progress
Unit 1, Lesson 21
In Progress

Communicative Assertion

Tests can be communicative from multiple angles. How well does your assertion state intent when we read the code? On failure, does your message point us in the right direction?

Video transcript & code

Communicative Assertion

So let's say we're using test-driven development to code up a connection pool class.

In one test, we create a new connection pool instance, specifying a pool size of 3.

Now, what's something basic and easy to test...

Let's just check that the connections list isn't empty.


class TestConnectionPool < Minitest::Test
  def test_creates_connections
    pool = ConnectionPool.new(size: 3)
    assert !pool.connections.empty?
  end
end

When we run this, we get a failure, as expected.


Expected false to be truthy.

1 runs, 1 assertions, 1 failures, 0 errors, 0 skips

OK, let's go write the code to make the test pass. Except... hang on a sec.

An important step in TDD is remembering to read the failure to make sure we're addressing the right thing. What does this one say?


Expected false to be truthy.

"Expected false to be truthy". This is, indeed, the expected failure. When it no longer tells us this, we'll be ready for the next test. And right now, we have enough context to know that this message corresponds to expecting the connections list to be not-empty.

But let's cast our minds forwards a few weeks or months. If we accidentally break this functionality, are we going to know what this message is telling us?

Instead of focusing on the implementation, let's take a little detour into making our assertions more communicative.

Tests can be communicative from at least two different angles. First, there's static communication: how well does the assertion state intent when we read the code? And second, on failure, does the message point us in the right direction?

Let's start with the static side.


    assert !pool.connections.empty?

What we're saying here is that we expect the connection pool to not be empty. Is there a clearer way to state this? Well, I think removing negations usually makes code easier to understand.

We can do that by using minitest's refute instead of assert.


    refute pool.connections.empty?

I think this reads a little better.

Let's run it.


Expected true to not be truthy.

"Expected true not to be truthy". Still not particularly useful.

Can minitest help us be more specific in our assertion? As a matter of fact it can.

We can use refute_predicate to use the test framework to apply the test for emptiness.


refute_predicate pool.connections, :empty?

When we run this, we get a slightly more useful message:


Expected [] to not be empty?.

Now at least instead of talking about truthiness, the message is saying something about expecting an array to not be empty.

Is this the most intention-revealing way to state our expectation using minitest? Not quite.

Because checking that a collection is or is not empty is a particularly common occurrence in tests, minitest actually supplies assertions specifically for it.


    refute_empty pool.connections

When we run this, we get the same message as before.


Expected [] to not be empty.

But the code at least states the assertion at a higher level of meaning.


    refute_empty pool.connections

One lesson here is that it pays off to become familiar with the assertions supplied by our preferred testing framework.

Now as I mentioned earlier, I always like to look for ways to make code read positively rather than negatively. Is there some way we could state our assertion in completely positive terms?

Maybe instead of saying the connection list shouldn't be empty... we could say that we expect there to be something in it.


    assert_predicate pool.connections, :any?

Let's run the test.


Expected [] to be any?.

I don't think this message is quite as nice as "expected to not be empty". But it's still reasonably intention-revealing. And we've eliminated negatives from our test.

Let's take a baby step forwards in making our test pass, by initializing the connection list with an array of the requested size.


    @connections = Array.new(size)

When we run the tests again, we get an interesting new message.


Expected [nil, nil, nil] to be any?.

This reveals an unexpected bonus of stating our assertion in all-positive terms. By using the any? predicate, we've implicitly made two assertions: first, that the collection won't be empty. And second, that it will have something other than nils in it.

So that's nice. But looking back at the assertion, it feels like we've both gained and lost clarity. Sure, we've stated our assertion at a higher level of intention. But this assert_predicate business is kind of ungainly and awkward.

One alternative is to switch over to using the RSpec test framework. This lets us write our assertions in a vaguely english-like DSL.


RSpec.describe ConnectionPool do
  it "creates connections" do
    pool = ConnectionPool.new(size: 3)
    expect(pool.connections).to be_any
  end
end

When we run this, the failure is quite descriptive.


  1) ConnectionPool creates connections
     Failure/Error: expect(pool.connections).to be_any
       expected `[nil, nil, nil].any?` to return true, got false
     # ./examples/07.rb:31:in `block (2 levels) in

'

It shows us the assertion that was made, and tells us that it applied the any? predicate to an array and got false.

That said, though, we've departed even further from just writing ordinary Ruby code.


    expect(pool.connections).to be_any

It feels like we're being forced to choose between clarity of failure messages, and writing straightforward code in our assertions.

Let's explore a couple of alternative approaches.

First off, we can go back to our minitest version, and go back to using a plain assert.


class TestConnectionPool < Minitest::Test
  def test_creates_connections
    pool = ConnectionPool.new(size: 3)
    assert pool.connections.any?
  end
end

But then we add a second argument to assert, which is a custom failure message.


class TestConnectionPool < Minitest::Test
  def test_creates_connections
    pool = ConnectionPool.new(size: 3)
    assert pool.connections.any?, "connections should be populated"
  end
end

Now when we run the tests, our failure message is specific to what we were looking for.


connections should be populated

This is a great way of making tests more communicative and self-documenting!

But like forms of documentation, it does mean we have to put in some extra effort. And we need to be sure to keep our failure messages in sync with the assertions they document!

Another approach is to use the power_assert gem.


require "minitest/power_assert"

Here we're using the minitest-power_assert plugin gem.

This adds some new abilities to assertions.

Instead of passing the test code expression as an argument to assert, we put it in a block given to assert.


    assert { pool.connections.any? }

Let's run this.


    assert { pool.connections.any? }
             |                |
             |                false
             #

This output is very different from anything we've seen before!

First, it shows the exact code of the assertion. But then, it breaks down the values of different parts of the expression! We can see that the connection pool contains a @connections array full of nils, and we can see that the any? predicate returned false.

This is not quite as intention-revealing as our custom failure message. But it enabled us to assert a very straightforward Ruby expression, and still provide a useful failure report when the expectation is not met.

Today we've seen several techniques for improving the communicativity of assertions in Ruby tests. We've used more specific types of assertion; looked for more positive ways of stating our expectations; applied custom failure messages, and finally tried out the power_assert gem.

Which of these approaches is right for your project? That's up to you and your team. But the important point to remember is this: between seeing a test fail and making it pass, you have an opportunity to make your test assertions more communicative. This can take your tests from being just design aid and regression checks, to being documentation of your intentions for writing the code. Pausing here might even help you understand your own purposes better, resulting in an improved design! List of Communicative Assertions So before you fix that test failure, stop for a second and ask yourself if the assertion and failure messages make sense, and if not, take a few minutes to improve them. Happy hacking!

Responses