OpenAPI and Apivore – with Ariel Caplan
In this era of rich client-side frameworks, for more and more web applications, exposing a JSON-based RESTful API is as important, if not more important, than producing HTML. But implementing an API is just the first step. If we expect programmers other than ourselves to use the API, we need to document it. And then, we need to ensure that the API always behaves the way the documentation claims that it does.
What if we could combine a couple of these steps into one? What if our API documentation could also function as a testable specification for our API, ensuring that the two never fall out of sync? In today’s episode, guest chef Ariel Caplan is going to show us how to do just that, using OpenAPI specifications and the Apivore tool. Enjoy!
[su_note note_color=”#eeeeee” radius=”0″]Further Reading on Swagger and Apivore[/su_note]
Video transcript & code
Have you ever integrated an API into your application, only to find out that the real-life behavior of the API wasn't what you'd expect from the documentation?
Maybe you've been on the other side, developing an API with documentation that always seemed to fall out of date.
Let's relive that pain for just a moment, and then we'll see what we can do as API developers to make things better for ourselves and for our users.
In this made-up case, we have a Rails API for managing widgets.
$ curl -i http://localhost:3000/widgets/blue.json
When we send a GET
request to /widgets/blue
, where "blue
" is the slug of the widget, we expect the API to return either a 200 status code and a JSON representation of a widget, or, if the widget we're looking for is not found, a 404 status code and a descriptive error.
Instead, we get back a 500 status code and a vague error message.
$ curl -i http://localhost:3000/widgets/blue.json
HTTP/1.1 500 Internal Server Error
Content-Type: application/json; charset=utf-8
X-Request-Id: 95bf9d6f-3d68-4dc0-8600-275e06082a17
X-Runtime: 0.021941
Content-Length: 46
{"status":500,"error":"Internal Server Error"}
Digging into the application code, it seems that the problem starts in the controller. If the widget with the requested slug is not found, the @widget
instance variable is set to nil
. That nil
is passed to the view , which tries to render attributes of the object. Since nil
doesn't have an id
, name
, slug
, etc., we end up with an ambiguous NoMethodError
.
# app/controllers/widgets_controller.rb
def set_widget
@widget = Widget.find_by(slug: params[:id])
end
# app/views/widgets/show.json.jbuilder
json.partial! "widgets/widget", widget: @widget
# app/views/widgets/_widget.json.jbuilder
json.extract! widget, :id, :name, :slug, :description, :price, :created_at, :updated_at
json.url widget_url(widget, format: :json)
This case is probably covered in our documentation, and the scary part is, we probably told our users explicitly to expect a 404. Even if we correct the problem in our code, there's nothing programmatically linking our docs and our code to make sure there aren't tons of other little issues like this lurking all over our API.
Today we're going to talk about a way to generate friendly documentation using YAML or JSON, and more importantly, to make sure it's always accurate.





/docs
route. 

/widgets/{slug}
endpoint will return either a 200
or a 404
.

Apivore::SwaggerChecker
. This will actually be the same object across all your tests, which comes in handy, as you'll see in just a moment. The rest of the boilerplate just sets up hooks to let us specify headers, query params, and request body more easily in any test where we need to.
# Gemfile
gem 'apivore'
# spec/requests/api_spec.rb
require 'rails_helper'
RSpec.describe 'the API', type: :apivore, order: :defined do
subject { Apivore::SwaggerChecker.instance_for('/docs.json') }
let(:url_params) {{}}
let(:headers) {{}}
let(:query_string_params) {{}}
let(:data_params) {{}}
let(:params) {
url_params.merge(
'_headers' => headers.merge('Accept' => 'application/json'),
'_query_string' => query_string_params.to_query,
'_data' => data_params
)
}
end
The first thing we'll do is take advantage of a really nifty feature of Apivore, namely writing a spec for our specs. In this case, we're going to assert that every endpoint in the documentation is covered by a passing test.
context 'tying it all together' do
it 'tests all documented routes' do
expect(subject).to validate_all_paths
end
end
It's important that you have order: :defined
in the context of your tests, and you put this test last.
This way, as you run your tests, the SwaggerChecker
instance you use - remember how I said it's the same instance every time? - that SwaggerChecker
instance will keep track of which endpoints and statuses you've tested, and it'll give you a big red failure if you haven't yet tested every endpoint and status in your documentation.
For example, if we run our test now, looks like the /widgets/{slug}
endpoint is untested for both 200 and 404 response codes. Having this sort of to-do list is a huge help both in backfilling a documented API with tests, and in making sure your documentation will never leave your tests behind.
$ rspec
F
Failures:
1) the API tying it all together tests all documented routes
Failure/Error: expect(subject).to validate_all_paths
get /widgets/{slug} is untested for response code 200
get /widgets/{slug} is untested for response code 404
# ./spec/requests/api_spec.rb:19:in `block (3 levels) in '
Finished in 0.14786 seconds (files took 1.57 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./spec/requests/api_spec.rb:18 # the API tying it all together tests all documented routes
The first test is going to be fairly straightforward. We'll test the 404 case for the /widgets/{slug}
endpoint, since it doesn't require any database setup. We use our handy url_params
from that boilerplate to specify what we want to use in the dynamic portion of the URL, in this case the slug "blue"
.
Then, we'll use RSpec's implicit subject to call validate
on our SwaggerChecker
with the request method, GET
, the URL under test, the 404 response code, and the request parameters. Apivore will then make a request to our application, filling in the dynamic {slug}
in the URL, and check that the response matches the documentation for the 404 case.
describe "GET '/widgets/{slug}'" do
let(:url_params) {{ 'slug' => 'blue' }}
context 'the widget does not exist' do
it { is_expected.to validate(:get, '/widgets/{slug}', 404, params) }
end
end
As expected, our test fails because of our NoMethodError
we saw earlier.
$ rspec
FF
Failures:
1) the API GET '/widgets/{slug}' the widget does not exist should validate that get /widgets/{slug} returns 404
Failure/Error: json.extract! widget, :id, :name, :slug, :description, :price, :created_at, :updated_at
ActionView::Template::Error:
undefined method `id' for nil:NilClass
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/jbuilder-2.7.0/lib/jbuilder.rb:260:in `public_send'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/jbuilder-2.7.0/lib/jbuilder.rb:260:in `block in _extract_method_values'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/jbuilder-2.7.0/lib/jbuilder.rb:260:in `each'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/jbuilder-2.7.0/lib/jbuilder.rb:260:in `_extract_method_values'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/jbuilder-2.7.0/lib/jbuilder.rb:219:in `extract!'
# ./app/views/widgets/_widget.json.jbuilder:1:in `_app_views_widgets__widget_json_jbuilder__1560270032508288162_70267474213860'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/jbuilder-2.7.0/lib/jbuilder/jbuilder_template.rb:128:in `_render_partial'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/jbuilder-2.7.0/lib/jbuilder/jbuilder_template.rb:122:in `_render_partial_with_options'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/jbuilder-2.7.0/lib/jbuilder/jbuilder_template.rb:215:in `_render_explicit_partial'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/jbuilder-2.7.0/lib/jbuilder/jbuilder_template.rb:22:in `partial!'
# ./app/views/widgets/show.json.jbuilder:1:in `_app_views_widgets_show_json_jbuilder___4056245668939668145_70267448600180'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/rack-2.0.4/lib/rack/etag.rb:25:in `call'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/rack-2.0.4/lib/rack/conditional_get.rb:25:in `call'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/rack-2.0.4/lib/rack/head.rb:12:in `call'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/railties-5.1.5/lib/rails/rack/logger.rb:36:in `call_app'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/railties-5.1.5/lib/rails/rack/logger.rb:24:in `block in call'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/railties-5.1.5/lib/rails/rack/logger.rb:24:in `call'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/rack-2.0.4/lib/rack/runtime.rb:22:in `call'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/rack-2.0.4/lib/rack/sendfile.rb:111:in `call'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/railties-5.1.5/lib/rails/engine.rb:522:in `call'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/rack-test-0.8.3/lib/rack/mock_session.rb:29:in `request'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/rack-test-0.8.3/lib/rack/test.rb:251:in `process_request'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/rack-test-0.8.3/lib/rack/test.rb:119:in `request'
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/apivore-1.6.2/lib/apivore/validator.rb:22:in `matches?'
# ./spec/requests/api_spec.rb:21:in `block (4 levels) in '
# ------------------
# --- Caused by: ---
# NoMethodError:
# undefined method `id' for nil:NilClass
# /Users/acaplan/.rvm/gems/ruby-2.5.0/gems/jbuilder-2.7.0/lib/jbuilder.rb:260:in `public_send'
2) the API tying it all together tests all documented routes
Failure/Error: expect(subject).to validate_all_paths
get /widgets/{slug} is untested for response code 200
get /widgets/{slug} is untested for response code 404
# ./spec/requests/api_spec.rb:27:in `block (3 levels) in '
Finished in 0.13557 seconds (files took 2.36 seconds to load)
2 examples, 2 failures
Failed examples:
rspec ./spec/requests/api_spec.rb:21 # the API GET '/widgets/{slug}' the widget does not exist should validate that get /widgets/{slug} returns 404
rspec ./spec/requests/api_spec.rb:26 # the API tying it all together tests all documented routes
Luckily, it's an easy fix! First, we'll modify our set_widget
method to use the bang version of find_by
so it raises a more descriptive error when the requested record doesn't exist in our system. Running the test again verifies we now have an ActiveRecord::RecordNotFound
error.
# app/controllers/widgets_controller.rb
def set_widget
@widget = Widget.find_by!(slug: params[:id])
end
$ rspec
FF
Failures:
1) the API GET '/widgets/{slug}' the widget does not exist should validate that get /widgets/{slug} returns 404
Failure/Error: @widget = Widget.find_by!(slug: params[:id])
ActiveRecord::RecordNotFound:
Couldn't find Widget
# etc...
Finally, we'll add some code to rescue that type of error, and render an error message with a 404.
# app/controllers/application_controller.rb
class ApplicationController < ActionController::API
include ActionView::Rendering
rescue_from ActiveRecord::RecordNotFound, with: :not_found
def not_found(error)
render json: { error: 'Requested record not found.' }, status: 404
end
end
Run the test again, and check it out! We have one passing test, and just one response code, 200, left to test.
$ rspec
.F
Failures:
1) the API tying it all together tests all documented routes
Failure/Error: expect(subject).to validate_all_paths
get /widgets/{slug} is untested for response code 200
# ./spec/requests/api_spec.rb:27:in `block (3 levels) in '
Finished in 0.12543 seconds (files took 1.46 seconds to load)
2 examples, 1 failure
Failed examples:
rspec ./spec/requests/api_spec.rb:26 # the API tying it all together tests all documented routes
$
Let's look back at our documentation to remember how this is supposed to look. We expect to send a request which specifies a slug, and receive a bunch of attributes in the response: id
as an integer, name
as a string, price
as a number... you get the idea.
Again, we'll start by writing the test first. This time, we'll create an example model with a full set of attributes, and verify that it's returned in the documented format for a 200 response code. Notice that our test is just a quick one-liner, because Apivore is using our documentation to figure out what to expect in the response.
# spec/requests/api_spec.rb
describe "GET '/widgets/{slug}'" do
let(:url_params) {{ 'slug' => 'blue' }}
context 'the widget does not exist' do
it { is_expected.to validate(:get, '/widgets/{slug}', 404, params) }
end
context 'the widget exists' do
before do
Widget.create(name: 'blue', slug: 'blue', price: 14.90, description: 'A blue widget!')
end
it { is_expected.to validate(:get, '/widgets/{slug}', 200, params) }
end
end
Now when we run our test, something interesting happens. Our documentation said to expect price as a number, but it turns out we're actually returning a String. And now we have to ask: Is that actually such a bad thing?
We could return price as a floating point number, but perhaps the complexity of floating point will mean that the number of decimal places may be more or less than 2.
$ rspec
.F.
Failures:
1) the API GET '/widgets/{slug}' the widget exists should validate that get /widgets/{slug} returns 200
Failure/Error: it { is_expected.to validate(:get, '/widgets/{slug}', 200, params) }
'/widgets/blue?#/price' of type string did not match the following type: number
Response body:
{
"id": 14,
"name": "blue",
"slug": "blue",
"description": "A blue widget!",
"price": "14.9",
"created_at": "2018-03-07T00:05:22.537Z",
"updated_at": "2018-03-07T00:05:22.537Z",
"url": "http://www.example.com/widgets/14.json"
}
# ./spec/requests/api_spec.rb:29:in `block (4 levels) in '
Finished in 0.13562 seconds (files took 1.66 seconds to load)
3 examples, 1 failure
Failed examples:
rspec ./spec/requests/api_spec.rb:29 # the API GET '/widgets/{slug}' the widget exists should validate that get /widgets/{slug} returns 200
Thinking through things more carefully, returning as a String with 2 decimal places seems to be the best idea for our application, so we'll update our documentation to say so. We modify the price
attribute of the widget to be of the "string"
type, using the pattern of one or more digits, followed by a period and exactly 2 more digits.
# in the middle of config/swagger.yml
price:
type: "string"
pattern: "^\\d+\\.\\d{2}$"
And just based on that change to documentation, our test now expects a price-formatted String as the price.
$ rspec
.F.
Failures:
1) the API GET '/widgets/{slug}' the widget exists should validate that get /widgets/{slug} returns 200
Failure/Error: it { is_expected.to validate(:get, '/widgets/{slug}', 200, params) }
'/widgets/blue?#/price' value "14.9" did not match the regex '^\d+\.\d{2}$'
Response body:
{
"id": 15,
"name": "blue",
"slug": "blue",
"description": "A blue widget!",
"price": "14.9",
"created_at": "2018-03-07T00:06:19.377Z",
"updated_at": "2018-03-07T00:06:19.377Z",
"url": "http://www.example.com/widgets/15.json"
}
# ./spec/requests/api_spec.rb:29:in `block (4 levels) in '
Finished in 0.1509 seconds (files took 1.7 seconds to load)
3 examples, 1 failure
Failed examples:
rspec ./spec/requests/api_spec.rb:29 # the API GET '/widgets/{slug}' the widget exists should validate that get /widgets/{slug} returns 200
At this point, all that's left is to actually implement converting price to a properly formatted String.
# app/views/widgets/_widget.json.jbuilder
json.extract! widget, :id, :name, :slug, :description, :created_at, :updated_at
json.price sprintf('%.2f', widget.price)
json.url widget_url(widget, format: :json)
And we're done! All the documented paths are tested, and those tests are passing.
$ rspec
...
Finished in 0.1504 seconds (files took 2.23 seconds to load)
3 examples, 0 failures
I want to make sure you noticed something important here: We didn't just fix our application to match our documentation, we also revised our documentation because the exercise of testing it put us into our users' shoes and helped us think about what would be most useful to them.
Having a guarantee that your documentation is accurate is really, incredibly valuable, and the exercise is worthwhile for that value alone. But I've found that the way I build APIs changes when I drive testing from documentation in this way. I think a lot more about the user's perspective, and I become more aware of bad design decisions, or areas that may cause confusion or frustration for users.
I hope you'll find that documentation testing will also help you apply user-centered design principles automatically, just by virtue of the tools you've chosen to use.
That's all for today. Happy hacking!
Responses