yield_self
One of the most useful metaphors in software development is the idea of a “pipeline”. When we can find a way to organize a complex process as a pipeline of operations, it becomes a lot easier to isolate discrete parts of the process and understand the dependencies between them.
In today’s episode, guest chef Nithin Bekal joins us to demonstrate how using the yield_self method from Ruby 2.5 can help clarify these kinds of pipelines. I hope you find the sequence of refactorings in this episode as satisfying as I do. Enjoy!
Video transcript & code
yield_self
Often you run into code that is composed of nested method calls. The argument to a method is another method call, and that in turn takes an argument which is yet another method call, and so on.
parse_response(fetch_response(build_url(params)))
Imagine this as a chain of operations. We can reshape this code by assigning the result of each step into a variable, and then passing that variable as an argument to the next step.
url = build_url(params)
response = fetch_response(url)
parse_response(response)
Let's look at an example. Here we write a method that uses the Google Books API to lookup a book by its ISBN.
def find_by_isbn(isbn)
# ...
end
The ISBN string we get could contain dashes. We remove them first.
def find_by_isbn(isbn)
isbn = isbn.gsub('-', '')
end
We then build the URI
object containing the ISBN as a query parameter.
require 'uri'
BASE_URL = 'https://www.googleapis.com/books/v1/volumes'
def find_by_isbn(isbn)
isbn = isbn.gsub('-', '')
url = URI("#{BASE_URL}?q=isbn:#{isbn}")
end
We then make an HTTP
call to fetch the response.
require 'net/http'
# ...
def find_by_isbn(isbn)
isbn = isbn.gsub('-', '')
url = URI("#{BASE_URL}?q=isbn:#{isbn}")
response = Net::HTTP.get(url)
end
And then convert the JSON response to a hash object.
require 'json'
# ...
def find_by_isbn(isbn)
isbn = isbn.gsub('-', '')
url = URI("#{BASE_URL}?q=isbn:#{isbn}")
response = Net::HTTP.get(url)
json = JSON.parse(response)
end
Finally, we use the dig
method to extract the relevant part of the hash.
def find_by_isbn(isbn)
isbn = isbn.gsub('-', '')
url = URI("#{BASE_URL}?q=isbn:#{isbn}")
response = Net::HTTP.get(url)
json = JSON.parse(response)
json.dig('items', 0, 'volumeInfo')
end
We've assigned quite a few variables here, but it's getting harder to see the shape of the pipeline of operations. When I know that the code has a pattern, I like it to be reflected in the shape of the code.
def find_by_isbn(isbn)
isbn = isbn.gsub('-', '')
url = URI("#{BASE_URL}?q=isbn:#{isbn}")
response = Net::HTTP.get(url)
json = JSON.parse(response)
json.dig('items', 0, 'volumeInfo')
end
book = find_by_isbn('978-1-93778-549-9')
book['title'] #=> "Programming Ruby 1.9 & 2.0"
What if, instead of creating all these variables, we express this as a series of transformations on the isbn
argument?
We often do this with enumerable objects using methods like map
or reduce
.
def squares(size)
1.step
.first(size)
.map { |n| n**2 }
end
squares(5)
# => [1, 4, 9, 16, 25]
Ruby 2.5 introduced a method called yield_self
that is available on every Ruby object. It yields the object to a given block and returns the value returned by that block.
Let's look at a very simple example. We take a string and call yield_self
with a block that calls upcase on its argument.
'ruby'.yield_self { |s| s.upcase }
The return value is the upcased string.
'ruby'.yield_self { |s| s.upcase } #=> 'RUBY'
This might not seem very impressive by itself, but we'll see how this allows us to rewrite the find_by_isbn
method as a chain of transformations.
We start off with the ISBN string, strip off the dashes, and then use yield_self
to build a URI
object.
require 'uri'
BASE_URL = "https://www.googleapis.com/books/v1/volumes"
def find_by_isbn(isbn)
isbn.gsub('-', '')
.yield_self { |isbn| URI("#{BASE_URL}?q=isbn:#{isbn}") }
end
Next, we chain another yield_self
to make the HTTP
call.
require 'net/http'
# ...
def find_by_isbn(isbn)
isbn.gsub('-', '')
.yield_self { |isbn| URI("#{BASE_URL}?q=isbn:#{isbn}") }
.yield_self { |uri| Net::HTTP.get(uri) }
end
We then add another yield_self
block to parse the JSON
response.
require 'json'
# ...
def find_by_isbn(isbn)
isbn.gsub('-', '')
.yield_self { |isbn| URI("#{BASE_URL}?q=isbn:#{isbn}") }
.yield_self { |uri| Net::HTTP.get(uri) }
.yield_self { |response| JSON.parse(response) }
end
We can now extract the required information from the hash.
def find_by_isbn(isbn)
isbn.gsub('-', '')
.yield_self { |isbn| URI("#{BASE_URL}?q=isbn:#{isbn}") }
.yield_self { |uri| Net::HTTP.get(uri) }
.yield_self { |response| JSON.parse(response) }
.yield_self { |response| response.dig('items', 0, 'volumeInfo') }
end
This code now represents its pipeline nature much more visually.
Let's clean this up even more. First, we'll extract each block in the chain into its own method.
BASE_URL = "https://www.googleapis.com/books/v1/volumes"
def find_by_isbn(isbn)
isbn.gsub('-', '')
.yield_self { |isbn| URI("#{BASE_URL}?isbn:#{isbn}") }
.yield_self { |uri| Net::HTTP.get(uri) }
.yield_self { |response| JSON.parse(response) }
.yield_self { |response| response.dig('items', 0, 'volumeInfo') }
end
We'll start with a build_uri
method for the first block.
# ...
def build_uri(isbn)
URI("#{BASE_URL}?isbn:#{isbn}")
end
def find_by_isbn(isbn)
isbn.gsub('-', '')
.yield_self { |isbn| build_uri(isbn) }
.yield_self { |uri| Net:HTTP.get(uri) }
.yield_self { |response| JSON.parse(response) }
.yield_self { |response| response.dig('items', 0, 'volumeInfo') }
end
And the next block becomes a fetch_response
method.
def fetch_response(uri)
Net::HTTP.get(uri)
end
def find_by_isbn(isbn)
isbn.gsub('-', '')
.yield_self { |isbn| build_uri(isbn) }
.yield_self { |uri| fetch_response(uri) }
.yield_self { |response| JSON.parse(response) }
.yield_self { |response| response.dig('items', 0, 'volumeInfo') }
end
Next we extract a parse_response
method.
def parse_response(response)
JSON.parse(response)
end
def find_by_isbn(isbn)
isbn.gsub('-', '')
.yield_self { |isbn| build_uri(isbn) }
.yield_self { |uri| fetch_response(uri) }
.yield_self { |response| parse_response(response) }
.yield_self { |response| response.dig('items', 0, 'volumeInfo') }
end
And the final block becomes an extract_volume_info
method.
def extract_volume_info(response)
response.dig('items', 0, 'volumeInfo')
end
def find_by_isbn(isbn)
isbn.gsub('-', '')
.yield_self { |isbn| build_uri(isbn) }
.yield_self { |uri| fetch_response(uri) }
.yield_self { |response| parse_response(response) }
.yield_self { |response| extract_volume_info(response) }
end
book = find_by_isbn('978-1-93778-549-9')
book['title'] #=> "Programming Ruby 1.9 & 2.0"
We can also extract the gsub
method call to name the operation that it performs. Let's call it remove_dashes
.
# ...
def remove_dashes(isbn)
isbn.gsub('-', '')
end
def find_by_isbn(isbn)
isbn
.yield_self { |isbn| remove_dashes(isbn) }
.yield_self { |isbn| build_uri(isbn) }
.yield_self { |uri| fetch_response(uri) }
.yield_self { |response| parse_response(response) }
.yield_self { |response| extract_volume_info(response) }
end
We can tighten this code even more by using method objects and the &
operator to convert those methods into proc
s.
def find_by_isbn(isbn)
isbn.gsub('-', '')
.yield_self(&method(:build_uri))
.yield_self(&method(:fetch_response))
.yield_self(&method(:parse_response))
.yield_self(&method(:extract_volume_info))
end
book = find_by_isbn('978-1-93778-549-9')
book['title'] #=> "Programming Ruby 1.9 & 2.0"
This code is an example of the composed method pattern. This pattern describes a method which is implemented as a series of steps that are all at the same level of abstraction. This was described by Kent Beck in his book Smalltalk Best Practice Patterns.
Ruby 2.6, which may be out by the time you see this, introduced a new alias for yield_self
, called then
. With the alias, the code now looks like this.
# ...
def find_by_isbn(isbn)
isbn
.then(&method(:remove_dashes))
.then(&method(:build_uri))
.then(&method(:fetch_response))
.then(&method(:parse_response))
.then(&method(:extract_volume_info))
end
RUBY_VERSION #=> "2.6.0"
find_by_isbn('978-1-93778-549-9') #=>
The Elixir programming language has a pipeline operator, which allows you to do something very similar. The elegance of pipelines in Elixir is what prompted me to try using yield_self
in Ruby.
def find_by_isbn(isbn) do
isbn
|> remove_dashes
|> build_url
|> fetch_response
|> parse_response
|> extract_volume_info
end
defp remove_dashes(isbn) do
String.replace(isbn, "-", "")
end
defp build_url(isbn) do
"#{@api_url}?q=isbn:#{isbn}"
end
# defp fetch_response ...
# defp parse_response ...
# defp extract_volume_info ...
Rewriting a bunch of variable assignments into a chain of transformations can make the code more readable and tell a coherent story of what the method does at a high level.
That's all for this episode. Happy hacking!
Responses