Better Code with Mutant
In the first episode in this series, guest chef Daniel Gollahon showed you how to use mutation testing tools to discover holes in your test coverage. But there’s more to mutation testing. Today you’ll see how mutation testing tools can actually make suggestions to improve the quality of your application code. Enjoy!
Video transcript & code
Writing Better Code with Mutant
In the last episode, I introduced the concept of mutation testing and showed how it can reveal weaknesses in your test suite. Today, I want to show you how it can also improve your ruby code.
Today we'll discuss how mutant can help you
(and)
class ProductsController < ApplicationController
def show
@product = Product.find(params[:id])
end
end
Imagine you have a simple Rails controller with a show action.
mutant
will emit this mutation
#fetch
raises an error if the expected key isn't present. If we know id
will always be provided, we should use #fetch
. Otherwise, we should write a test that asserts the behavior when the key isn't provided. This catches a surprising amount of nil
bugs.
If your id should be an integer, you might consider calling <a href="https://www.rubytapas.com/out/ruby--string-to_i">#to_i</a>
.
class ProductsController < ApplicationController
def show
@product = Product.find(params.fetch(:id).to_i)
end
end
Mutant will recommend Integer(...)
instead:
Why? #to_i
is very flexible, and we might not always want it to be.
nil.to_i # => 0
''.to_i # => 0
' '.to_i # => 0
'lol'.to_i # => 0
'1st place!'.to_i # => 1
#to_i
coerces nil
and strings without numbers to 0
. For ambiguous strings, it may just pull out the first number it sees.
Integer(...)
is strict. All of the previous examples throw an error. If you want to learn more about constructor functions, check out Episode #207.
Let's consider another example.
class TasksController < ApplicationController
def create
Tasks.create(
due_date: Date.parse(params.fetch(:due_date)),
description: params.fetch(:description)
)
end
end
Imagine you are working on a todo app. You might have a controller that creates tasks with due dates like this.
Mutant, however, discourages the permissive Date.parse
and suggests more specific alternatives.
Date.parse("2018-05-01") __
Date.parse("H30.05.01") __
Date.parse("Tue May 01 00:00:00 2018") __
Date.parse("Tue, 01 May 2018 00:00:00 +0000") __
Date.parse("Tue, 01 May 2018 00:00:00 GMT") __
Date.parse("May") __
Date.parse("Hey, Luke! May the force be with you!")
What do you think the output would be for these examples?
Date.parse("2018-05-01") # => #<Date: 2018-05-01>
Date.parse("H30.05.01") # => #<Date: 2018-05-01>
Date.parse("Tue May 01 00:00:00 2018") # => #<Date: 2018-05-01>
Date.parse("Tue, 01 May 2018 00:00:00 +0000") # => #<Date: 2018-05-01>
Date.parse("Tue, 01 May 2018 00:00:00 GMT") # => #<Date: 2018-05-01>
Date.parse("May") # => #<Date: 2018-05-01>
Date.parse("Hey, Luke! May the force be with you!") # => #<Date: 2018-05-01>
Surprisingly, every one will result in May 1st of this year!
Date.iso8601("2018-05-01") # => #<Date: 2018-05-01>
Date.iso8601("H30.05.01") # => #<ArgumentError: invalid date>
Date.iso8601("Tue May 01 00:00:00 2018") # => #<ArgumentError: invalid date>
Date.iso8601("Tue, 01 May 2018 00:00:00 +0000") # => #<ArgumentError: invalid date>
Date.iso8601("Tue, 01 May 2018 00:00:00 GMT") # => #<ArgumentError: invalid date>
Date.iso8601("May") # => #<ArgumentError: invalid date>
Date.iso8601("Hey, Luke... may the force be with you.") # => #<ArgumentError: invalid date>
Date.iso8601
, however, only parses the ISO 8601 format. Everything else raises an error.
Mutant has hundreds of hand crafted method mutations like these--each one guiding you towards a more precise usage of the Ruby standard library.
class Worker < MyWorker
attr_reader :condition
def initialize(condition)
@condition = condition
end
def perform
unless @condition
do_the_thing!
end
end
end
Have you ever spent hours tearing apart your code, only to realize you misspelled an instance variable?
class Worker < MyWorker
attr_reader :condition
def initialize(condition)
@condition = condition
end
def perform
unless @condtion
do_the_thing!
end
end
end
In ruby, references to uninitialized instance variables return nil
.
Mutant will try to replace instance variables with nil
which can be a helpful hint that something is not right.
When you fix the typo
class Worker < MyWorker
attr_reader :condition
def initialize(condition)
@condition = condition
end
def perform
unless @condition
do_the_thing!
end
end
end
mutant will encourage you to use a reader method instead, if it is available.
Now a typo in condition
will raise a helpful NoMethodError
instead of silently returning nil
.
Pop quiz! What are the differences between a proc and a lambda? Can't remember? No problem, Mutant can help
require 'logger'
module MyLogger
def self.generate
logger = Logger.new($stdout)
logger.formatter =
Proc.new do |severity, datetime, _name, msg|
"[#{severity}] #{datetime} -- #{msg}\n"
end
logger
end
end
If you want to customize your log output, you might specify a formatting proc like this
Mutant will suggest you use a lambda instead--but why?
formatter = proc { |a, b| [a, b] }
formatter.call() # => [nil, nil]
formatter.call(1) # => [1, nil]
formatter.call(1, 2) # => [1, 2]
formatter.call(1, 2, 3) # => [1, 2]
formatter.call([]) # => [nil, nil]
formatter.call([1]) # => [1, nil]
formatter.call([1, 2]) # => [1, 2]
formatter.call([1, 2, 3]) # => [1, 2]
Procs fill in missing arguments with nil
and a single array argument gets automatically splatted. What about a lambda?
formatter = lambda { |a, b| [a, b].inspect }
formatter.call() # => ArgumentError
formatter.call(1) # => ArgumentError
formatter.call(1, 2) # => [1, 2]
formatter.call(1, 2, 3) # => ArgumentError
formatter.call([]) # => ArgumentError
formatter.call([1]) # => ArgumentError
formatter.call([1, 2]) # => ArgumentError
formatter.call([1, 2, 3]) # => ArgumentError
Now an ArgumentError
is raised unless exactly two arguments are provided.
Take a look at this this ActiveRecord method. Notice anything?
class Product < ApplicationModel
def find_by_name(name)
where(name: name).first
end
end
ActiveRecord
already defines this method for us!
Mutant points out this method is unnecessary by replacing the body with <a href="https://www.rubytapas.com/out/ruby--super">super</a>
.
Mutant can also detect redundant arguments. Consider our Products controller from before:
class ProductsController < ApplicationController
def show
@product = Product.find(params.fetch(:id))
render json: @product.serialize, status: :success
end
end
Here we simply serialize our product and set an OK status.
But mutant
helps us realize that Rails will default to :success
when we don't provide the status
keyword.
Mutant will try removing every bit of code it encounters, making it an effective tool for eliminating dead or unnecessary code.
Regular expressions are notoriously tricky and often overlooked in tests.
For instance, the ruby bson
gem introduced a vulnerability by changing their id validation regex to one that did not use the appropriate line anchors.
CVE-2015-4412
BSON injection vulnerability in the legal? function in BSON (bson-ruby) gem before 3.0.4 for Ruby allows remote attackers to cause a denial of service (resource consumption) or inject arbitrary data via a crafted string.
- /\A\h{24}\Z/
+ /^[0-9a-f]{24}$/i
In ruby, dollar and caret mean beginning and end of line, but A and z mean the beginning and end of string.
The original code actually had a more subtle bug, confusing lowercase z and uppercase Z, which allowed whitespace to corrupt ids.
Mutant always promotes weaker anchors to stronger ones.
- /\A\h{24}\Z/
+ /\A\h{24}\z/
- /^[0-9a-f]{24}$/i
+ /\A[0-9a-f]{24}$/i
- /^[0-9a-f]{24}$/i
+ /^[0-9a-f]{24}\z/i
Regular expressions can also have untested branches.
module Tapas
def self.tapas_contributor?(author_bio)
author_bio.match?(/John|Daniel|Larry/)
end
end
My friend John helped with this video, but Larry doesn't belong.
Mutant tries removing each branch of the union to make sure we've considered every kind of input it can match.
Mutant can also modify character groups, suggest non-capturing groups, and adjust quantifiers.
I hope I've convinced you that Mutant can be a powerful feedback tool. This video focused on ways to improve your code, but keep in mind that there are always two ways to kill a mutation:
Only one of these is appropriate in a given context.
Let's revisit our Date.parse example.
class TasksController < ApplicationController
def create
Tasks.create(
due_date: Date.parse(params[:due_date]),
description: params[:description]
)
end
end
What if our customers actually provide multiple date formats? In that case, we should write additional tests, not update our code.
Mutations are a reminder that you need to make a choice: is this a feature or a bug? Never blindly accept a mutation. Instead, ask whether changing the code produces the behavior you want it to. If not, write a test to document your intentions.
Mutation testing can be hard at first, but after using it for years now, I've found myself automatically writing more testable code and better tests. Give it a try, and I bet you'll learn a lot.
Special thanks to John Backus for providing me with valuable feedback and support when creating these episodes and for introducing me to mutation testing in the first place.
Responses