Writing Better Code with Mutant

In the last episode, I introduced the concept of mutation testing and showed how it can reveal weaknesses in your test suite. Today, I want to show you how it can also improve your ruby code.

Today we'll discuss how mutant can help you (and)

class ProductsController < ApplicationController
  def show
    @product = Product.find(params[:id])
  end
end

Imagine you have a simple Rails controller with a show action.

mutant will emit this mutation

#fetch raises an error if the expected key isn't present. If we know id will always be provided, we should use #fetch. Otherwise, we should write a test that asserts the behavior when the key isn't provided. This catches a surprising amount of nil bugs.

If your id should be an integer, you might consider calling <a href="https://www.rubytapas.com/out/ruby--string-to_i">#to_i</a>.

class ProductsController < ApplicationController
  def show
    @product = Product.find(params.fetch(:id).to_i)
  end
end

Mutant will recommend Integer(...) instead:

Why? #to_i is very flexible, and we might not always want it to be.

nil.to_i   # => 0
''.to_i    # => 0
' '.to_i   # => 0
'lol'.to_i # => 0

'1st place!'.to_i # => 1

#to_i coerces nil and strings without numbers to 0. For ambiguous strings, it may just pull out the first number it sees.

Integer(...) is strict. All of the previous examples throw an error. If you want to learn more about constructor functions, check out Episode #207.

Let's consider another example.

class TasksController < ApplicationController
  def create
    Tasks.create(
      due_date:    Date.parse(params.fetch(:due_date)),
      description: params.fetch(:description)
    )
  end
end

Imagine you are working on a todo app. You might have a controller that creates tasks with due dates like this.

Mutant, however, discourages the permissive Date.parse and suggests more specific alternatives.

Date.parse("2018-05-01")                           __
Date.parse("H30.05.01")                            __
Date.parse("Tue May 01 00:00:00 2018")             __
Date.parse("Tue, 01 May 2018 00:00:00 +0000")      __
Date.parse("Tue, 01 May 2018 00:00:00 GMT")        __
Date.parse("May")                                  __
Date.parse("Hey, Luke! May the force be with you!")

What do you think the output would be for these examples?

Date.parse("2018-05-01")                            # => #<Date: 2018-05-01>
Date.parse("H30.05.01")                             # => #<Date: 2018-05-01>
Date.parse("Tue May 01 00:00:00 2018")              # => #<Date: 2018-05-01>
Date.parse("Tue, 01 May 2018 00:00:00 +0000")       # => #<Date: 2018-05-01>
Date.parse("Tue, 01 May 2018 00:00:00 GMT")         # => #<Date: 2018-05-01>
Date.parse("May")                                   # => #<Date: 2018-05-01>
Date.parse("Hey, Luke! May the force be with you!") # => #<Date: 2018-05-01>

Surprisingly, every one will result in May 1st of this year!

Date.iso8601("2018-05-01")                            # => #<Date: 2018-05-01>
Date.iso8601("H30.05.01")                             # => #<ArgumentError: invalid date>
Date.iso8601("Tue May 01 00:00:00 2018")              # => #<ArgumentError: invalid date>
Date.iso8601("Tue, 01 May 2018 00:00:00 +0000")       # => #<ArgumentError: invalid date>
Date.iso8601("Tue, 01 May 2018 00:00:00 GMT")         # => #<ArgumentError: invalid date>
Date.iso8601("May")                                   # => #<ArgumentError: invalid date>
Date.iso8601("Hey, Luke... may the force be with you.") # => #<ArgumentError: invalid date>

Date.iso8601, however, only parses the ISO 8601 format. Everything else raises an error.

Mutant has hundreds of hand crafted method mutations like these--each one guiding you towards a more precise usage of the Ruby standard library.

class Worker < MyWorker
  attr_reader :condition

  def initialize(condition)
    @condition = condition
  end

  def perform
    unless @condition
      do_the_thing!
    end
  end
end

Have you ever spent hours tearing apart your code, only to realize you misspelled an instance variable?

class Worker < MyWorker
  attr_reader :condition

  def initialize(condition)
    @condition = condition
  end

  def perform
    unless @condtion
      do_the_thing!
    end
  end
end

In ruby, references to uninitialized instance variables return nil.

Mutant will try to replace instance variables with nil which can be a helpful hint that something is not right.

When you fix the typo

class Worker < MyWorker
  attr_reader :condition

  def initialize(condition)
    @condition = condition
  end

  def perform
    unless @condition
      do_the_thing!
    end
  end
end

mutant will encourage you to use a reader method instead, if it is available.

Now a typo in condition will raise a helpful NoMethodError instead of silently returning nil.

Pop quiz! What are the differences between a proc and a lambda? Can't remember? No problem, Mutant can help

require 'logger'

module MyLogger
  def self.generate
    logger = Logger.new($stdout)

    logger.formatter =
      Proc.new do |severity, datetime, _name, msg|
        "[#{severity}] #{datetime} -- #{msg}\n"
      end

    logger
  end
end

If you want to customize your log output, you might specify a formatting proc like this

Mutant will suggest you use a lambda instead--but why?

formatter = proc { |a, b| [a, b] }

formatter.call()          # => [nil, nil]
formatter.call(1)         # => [1, nil]
formatter.call(1, 2)      # => [1, 2]
formatter.call(1, 2, 3)   # => [1, 2]
formatter.call([])        # => [nil, nil]
formatter.call([1])       # => [1, nil]
formatter.call([1, 2])    # => [1, 2]
formatter.call([1, 2, 3]) # => [1, 2]

Procs fill in missing arguments with nil and a single array argument gets automatically splatted. What about a lambda?

formatter = lambda { |a, b| [a, b].inspect }

formatter.call()          # => ArgumentError
formatter.call(1)         # => ArgumentError
formatter.call(1, 2)      # => [1, 2]
formatter.call(1, 2, 3)   # => ArgumentError
formatter.call([])        # => ArgumentError
formatter.call([1])       # => ArgumentError
formatter.call([1, 2])    # => ArgumentError
formatter.call([1, 2, 3]) # => ArgumentError

Now an ArgumentError is raised unless exactly two arguments are provided.

Take a look at this this ActiveRecord method. Notice anything?

class Product < ApplicationModel
  def find_by_name(name)
    where(name: name).first
  end
end

ActiveRecord already defines this method for us!

Mutant points out this method is unnecessary by replacing the body with <a href="https://www.rubytapas.com/out/ruby--super">super</a>.

Mutant can also detect redundant arguments. Consider our Products controller from before:

class ProductsController < ApplicationController
  def show
    @product = Product.find(params.fetch(:id))
    render json: @product.serialize, status: :success
  end
end

Here we simply serialize our product and set an OK status.

But mutant helps us realize that Rails will default to :success when we don't provide the status keyword.

Mutant will try removing every bit of code it encounters, making it an effective tool for eliminating dead or unnecessary code.

Regular expressions are notoriously tricky and often overlooked in tests.

For instance, the ruby bson gem introduced a vulnerability by changing their id validation regex to one that did not use the appropriate line anchors.

CVE-2015-4412

BSON injection vulnerability in the legal? function in BSON (bson-ruby) gem before 3.0.4 for Ruby allows remote attackers to cause a denial of service (resource consumption) or inject arbitrary data via a crafted string.

- /\A\h{24}\Z/
+ /^[0-9a-f]{24}$/i

In ruby, dollar and caret mean beginning and end of line, but A and z mean the beginning and end of string.

The original code actually had a more subtle bug, confusing lowercase z and uppercase Z, which allowed whitespace to corrupt ids.

Mutant always promotes weaker anchors to stronger ones.

- /\A\h{24}\Z/
+ /\A\h{24}\z/

- /^[0-9a-f]{24}$/i
+ /\A[0-9a-f]{24}$/i

- /^[0-9a-f]{24}$/i
+ /^[0-9a-f]{24}\z/i

Regular expressions can also have untested branches.

module Tapas
  def self.tapas_contributor?(author_bio)
    author_bio.match?(/John|Daniel|Larry/)
  end
end

My friend John helped with this video, but Larry doesn't belong.

Mutant tries removing each branch of the union to make sure we've considered every kind of input it can match.

Mutant can also modify character groups, suggest non-capturing groups, and adjust quantifiers.

I hope I've convinced you that Mutant can be a powerful feedback tool. This video focused on ways to improve your code, but keep in mind that there are always two ways to kill a mutation:

Only one of these is appropriate in a given context.

Let's revisit our Date.parse example.

class TasksController < ApplicationController
  def create
    Tasks.create(
      due_date:    Date.parse(params[:due_date]),
      description: params[:description]
    )
  end
end

What if our customers actually provide multiple date formats? In that case, we should write additional tests, not update our code.

Mutations are a reminder that you need to make a choice: is this a feature or a bug? Never blindly accept a mutation. Instead, ask whether changing the code produces the behavior you want it to. If not, write a test to document your intentions.

Mutation testing can be hard at first, but after using it for years now, I've found myself automatically writing more testable code and better tests. Give it a try, and I bet you'll learn a lot.

Special thanks to John Backus for providing me with valuable feedback and support when creating these episodes and for introducing me to mutation testing in the first place.