JSON API in Rails – Part 2
In the first episode in this series, guest chef Youssef Chaker pointed out some of the risks in adding a quick, ad-hoc JSON endpoint. Today Youssef returns. He’s going to show how with the help of some extra gems, we can leverage the JSON-API standard and create RESTful HTTP interfaces that are consistent, predictable, powerful, and well-documented. Enjoy!
Video transcript & code
In the previous episode, we looked at building an API and how an unstructured response can create a bunch of issues that we would have to deal with.
# Potential Risks:
# 1. Cost of change
# 2. Documentation
# 3. Collaboration
# 4. Flexibility
We summarized the risks under the following 4 categories: Cost of change: what's the effort required of us to update the API response any time we introduce a change to the app? Documentation: is our API easy to document, and how fast does the documentation go stale after changes? Collaboration: how much energy do we have to exert to make sure collaboration with fellow developers on the API is a smooth process that follows the same standards? Flexibility: can the client choose what response to get back from the API or is it rigid and forces everyone to deal with the same objects being returned even if it doesn't fit their needs?
It takes a lot of work, especially upfront work as a team, to resolve these issues. Luckily we don't have to do that work ourselves because a bunch of really smart people came up with a solution.
Enter the JSON API spec.
The JSON API website describes it as:
JSON API is a specification for how a client should request that resources be fetched or modified, and how a server should respond to those requests. JSON API is designed to minimize both the number of requests and the amount of data transmitted between clients and servers. This efficiency is achieved without compromising readability, flexibility, or discoverability.
Let's unpack that a bit. "JSON API is a specification for how a client should request that resources be fetched or modified" What that means is that the spec provides a common way for how any client (a mobile app, web app, or any consumer of an API) sends requests to the API to fetch a resource, or in other terms GET/PUT/POST/DELETE requests.
"and how a server should respond to those requests" means that the spec provides a common way the API structures any and all responses to requests the API receives.
# Potential Risks:
# 1. Cost of change
# 2. Documentation
# 3. Collaboration
# 4. Flexibility
Let's look at how using JSON API would alleviate the risks we discussed earlier.
# Potential Risks:
# 1. Cost of change
# 2. Documentation
# 3. Collaboration
# 4. Flexibility
Since the the spec provides a convention over configuration framework for an API, the cost of change is reduced. We will look later at how using the proper tools will reduce the amount of work we would have to do as devs.
# Potential Risks:
# 1. Cost of change
# 2. Documentation
# 3. Collaboration
# 4. Flexibility
Using the fact that JSON API is a spec that our API would follow, the spec itself becomes part of the shared documentation and a reference we can use for our own API. We would still have to do some work to document the specific end points our API provides, but we wouldn't need to provide specific documentation on how to consume the API because that's all part of the spec. Tools like Swagger, which you might have seen in Episode #527, could also be used to automate the documentation and reduce the amount of effort we need to put into it.
# Potential Risks:
# 1. Cost of change
# 2. Documentation
# 3. Collaboration
# 4. Flexibility
The problems we faced previously resulting from collaborating with other team members to create our API will automatically disappear by the nature of us following a spec.
# Potential Risks:
# 1. Cost of change
# 2. Documentation
# 3. Collaboration
# 4. Flexibility
Finally, the spec provides a way for the client to specify what to request from the API, and thus introduces a level of flexibility that didn't exist in the implementation we initially came up with.
It may seem to good to be true, that the risks we talked about magically disappear by using the JSON API spec. And that's because this is precisely why the spec was created. There's still a bit of work on our end, so let's show how we can create the same API for our recipe app using JSON API.
For this portion, we will use a gem called jsonapi_suite
, which is technically a collection of gems that we can use to support the JSON API spec in our Ruby on Rails application.
source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }
ruby '2.5.0'
# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '~> 5.2.0'
# Use postgresql as the database for Active Record
gem 'pg', '>= 0.18', '< 2.0' # Use Puma as the app server gem 'puma', '~> 3.11'
# Build JSON APIs with ease. Read more: https://github.com/rails/jbuilder
# gem 'jbuilder', '~> 2.5'
# Use ActiveModel has_secure_password
# gem 'bcrypt', '~> 3.1.7'
# Use Capistrano for deployment
# gem 'capistrano-rails', group: :development
# Reduces boot times through caching; required in config/boot.rb
gem 'bootsnap', '>= 1.1.0', require: false
# Use Rack CORS for handling Cross-Origin Resource Sharing (CORS), making cross-origin AJAX possible
# gem 'rack-cors'
gem 'jsonapi_suite', '~> 0.7'
gem 'jsonapi-rails', '~> 0.3.0'
gem 'kaminari', '~> 0.17'
group :development, :test do
# Call 'byebug' anywhere in the code to stop execution and get a debugger console
gem 'byebug', platforms: [:mri, :mingw, :x64_mingw]
gem 'foreman', '~> 0.84.0', require: false
# Annotate models, test, fixtures, etc with the table schema
gem 'annotate'
end
group :development do
gem 'listen', '>= 3.0.5', '< 3.2' # Spring speeds up development by keeping your application running in the background. Read more: https://github.com/rails/spring gem 'spring' gem 'spring-watcher-listen', '~> 2.0.0'
end
# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby]
We'll start by adding the gem to our Gemfile
and running bundle install
.
$ bin/rails generate jsonapi_suite:install
Then we'll run the installer that comes with the gem.
$ bin/rails generate jsonapi_suite:install
Running via Spring preloader in process 182
create config/initializers/jsonapi.rb
create config/initializers/strong_resources.rb
insert app/controllers/application_controller.rb
insert app/controllers/application_controller.rb
This will create a couple of files and insert some code into our ApplicationController
.
You can check out the gem's documentation for further detail about this step.
$ bin/rails g
Jsonapi:
jsonapi:initializer
jsonapi:resource
jsonapi:serializable
The jsonapi_suite
gem comes with a couple of handy generators, namely jsonapi:resource
and jsonapi:serializable
.
We will use the jsonapi:resource
generator the first time out to see what the expected code looks like.
Feel free to continue using the generator afterwards or manually creating the files once you've got a good handle on what is required.
$ bin/rails g jsonapi:resource Recipe --skip-namespace
Since the main resource we are dealing with is a Recipe
, that's what we'll use in the generator.
We are also specifying the --skip-namespace
flag since our app is strictly an API app, so we do not need the /api
namespace or any other namespace for that matter.
This command generates a few files, so let's take a look at each individually.
# app/controllers/recipes_controller.rb
class RecipesController < ApplicationController
# Mark this as a JSONAPI controller, associating with the given resource
jsonapi resource: RecipeResource
# Reference a strong resource payload defined in
# config/initializers/strong_resources.rb
strong_resource :recipe
# Run strong parameter validation for these actions.
# Invalid keys will be dropped.
# Invalid value types will log or raise based on the configuration
# ActionController::Parameters.action_on_invalid_parameters
before_action :apply_strong_params, only: [:create, :update]
# Start with a base scope and pass to render_jsonapi
def index
recipes = Recipe.all
render_jsonapi(recipes)
end
# Call jsonapi_scope directly here so we can get behavior like
# sparse fieldsets and statistics.
def show
scope = jsonapi_scope(Recipe.where(id: params[:id]))
instance = scope.resolve.first
raise JsonapiCompliable::Errors::RecordNotFound unless instance
render_jsonapi(instance, scope: false)
end
# jsonapi_create will use the configured Resource (and adapter) to persist.
# This will handle nested relationships as well.
# On validation errors, render correct error JSON.
def create
recipe, success = jsonapi_create.to_a
if success
render_jsonapi(recipe, scope: false)
else
render_errors_for(recipe)
end
end
# jsonapi_update will use the configured Resource (and adapter) to persist.
# This will handle nested relationships as well.
# On validation errors, render correct error JSON.
def update
recipe, success = jsonapi_update.to_a
if success
render_jsonapi(recipe, scope: false)
else
render_errors_for(recipe)
end
end
# Renders 200 OK with empty meta
# http://jsonapi.org/format/#crud-deleting-responses-200
def destroy
recipe, success = jsonapi_destroy.to_a
if success
render json: { meta: {} }
else
render_errors_for(recipe)
end
end
end
This is the controller that gets generated.
There's a lot of boilerplate code in here.
It doesn't look too different from a regular Ruby on Rails controller, especially if you've ever used the responders gem.
The main feature we will focus on is the use of the render_jsonapi
method. Under the hood this method uses the other files that got generated to form the response that will conform to the JSON API spec.
# app/resources/recipe_resource.rb
# Define how to query and persist a given model.
# Further Resource documentation: https://jsonapi-suite.github.io/jsonapi_compliable/JsonapiCompliable/Resource.html
class RecipeResource < ApplicationResource # Used for associating this resource with a given input. # This should match the 'type' in the corresponding serializer. type :recipes # Associate to a Model object so we know how to persist. model Recipe # Customize your resource here. Some common examples: # # === Allow ?filter[name] query parameter === # allow_filter :name # # === Enable total count, when requested === # allow_stat total: [:count] # # === Allow sideloading/sideposting of relationships === # belongs_to :foo, # foreign_key: :foo_id, # resource: FooResource, # scope: -> { Foo.all }
#
# === Custom sorting logic ===
# sort do |scope, att, dir|
# ... code ...
# end
#
# === Change default sort ===
# default_sort([{ title: :asc }])
#
# === Custom pagination logic ===
# paginate do |scope, current_page, per_page|
# ... code ...
# end
#
# === Change default page size ===
# default_page_size(10)
#
# === Change how we resolve the scope ===
# def resolve(scope)
# ... code ...
# end
end
The resource file is how we tie the JSON API resource to the underlying model, including how to query, filter, paginate, or modify a given model. We'll get back to this file in a bit.
# app/serializers/serializable_recipe.rb
# Serializers define the rendered JSON for a model instance.
# We use jsonapi-rb, which is similar to active_model_serializers.
class SerializableRecipe < JSONAPI::Serializable::Resource
type :recipes
# Add attributes here to ensure they get rendered, .e.g.
#
# attribute :name
#
# To customize, pass a block and reference the underlying @object
# being serialized:
#
# attribute :name do
# @object.name.upcase
# end
end
The serializer determines how the model is transformed into a JSON object, and what data is sent back in the payload. If the API wants to make an attribute available in the response it should be specified here. The client will be able to specify in the request whether to include all of the attributes or a select list, but only attributes set here would be available. This is where you can choose to not make certain attributes public, such as a password or any other sensitive data.
# curl "localhost:3000/recipes.json?filter%5Bdate%5D=2018-07-04"
{
"errors": [
{
"code": "internal_server_error",
"status": "500",
"title": "Error",
"detail": "We've notified our engineers and hope to address this issue shortly.",
"meta": {}
}
]
}
Let's test out our current setup, using the same curl
command we used previously.
This time we get an error. Before we look into the reason for this error, check out the structured error information that the jsonapi_suite gem provides! There are separate fields for status code, title, details, etc.
Started GET "/recipes.json?filter%5Bdate%5D=2018-07-04"
Processing by RecipesController#index as JSON
Parameters: {"filter"=>{"date"=>"2018-07-04"}}
ERROR: JsonapiCompliable::Errors::BadFilter: JsonapiCompliable::Errors::BadFilter
app/controllers/recipes_controller.rb:17:in `index'
After looking at the logs, we see that we sent a request that the API did not expect. Mainly the filter param that we sent is not supported.
# curl "localhost:3000/recipes.json"
{
"data": [
{
"id": "1",
"type": "recipes"
},
{
"id": "3",
"type": "recipes"
},
{
"id": "4",
"type": "recipes"
},
{
"id": "2",
"type": "recipes"
}
],
"meta": {},
"jsonapi": {
"version": "1.0"
}
}
After removing the filter params we start getting back a response. Albeit not a very useful one, but still a response. This response matches the format expected by the JSON API spec. So we know we're on track. What we need to do is specify the fields we want the JSON response to include as well as enabling the filtering.
# Serializers define the rendered JSON for a model instance.
# We use jsonapi-rb, which is similar to active_model_serializers.
class SerializableRecipe < JSONAPI::Serializable::Resource
type :recipes
attribute :name
attribute :description
attribute :url
end
A simple edit to the serializer file to specify the attributes we want the JSON response to include should to the trick.
# curl "localhost:3000/recipes.json"
{
"data": [
{
"id": "1",
"type": "recipes",
"attributes": {
"name": "Fattoush",
"description": "Fattoush is one of the most well known Middle Eastern salads and a standard dish on the 'mezza' (small dishes) table. It's a colorful tossed salad with a lemony garlic dressing, and if you've never made a single Arabic dish, this is a delicious and healthy place to start.",
"url": "https://www.allrecipes.com/recipe/223439/arabic-fattoush-salad/?internalSource=streams&referringId=235&referringContentType=recipe%20hub&clickId=st_trending_b"
}
},
{
"id": "3",
"type": "recipes",
"attributes": {
"name": "Baba Ghanoush",
"description": "This wonderful dip is made with roasted eggplant, tahini, lemon and lots of roasted garlic. Even if you don't usually like eggplant, you just may love this. It is the perfect starter for a Middle Eastern dinner. I like to serve this with generous strips of green, red and orange bell peppers for dipping. (Remember that each bell pepper is about 6g net carbs). It is also good stuffed into mushroom, celery or cherry tomatoes.",
"url": "https://www.allrecipes.com/recipe/81603/baba-ghanoush/"
}
},
{
"id": "4",
"type": "recipes",
"attributes": {
"name": "Shish Tawook",
"description": "Shish Tawook is a traditional marinated chicken shish kebab of Middle Eastern cuisine.",
"url": "https://www.allrecipes.com/recipe/150251/shish-tawook-grilled-chicken/?internalSource=hub%20recipe&referringId=235&referringContentType=recipe%20hub&clickId=cardslot%2022"
}
},
{
"id": "2",
"type": "recipes",
"attributes": {
"name": "Baked Falafel",
"description": "The baked and healthier version of the traditional falafel. Falafel is a deep-fried ball, doughnut or patty made from ground chickpeas, fava beans, or both.",
"url": "https://www.allrecipes.com/recipe/183947/baked-falafel/?internalSource=hub%20recipe&referringContentType=search%20results&clickId=cardslot%209"
}
}
],
"meta": {},
"jsonapi": {
"version": "1.0"
}
}
That looks much better!
# Define how to query and persist a given model.
# Further Resource documentation: https://jsonapi-suite.github.io/jsonapi_compliable/JsonapiCompliable/Resource.html
class RecipeResource < ApplicationResource # Used for associating this resource with a given input. # This should match the 'type' in the corresponding serializer. type :recipes # Associate to a Model object so we know how to persist. model Recipe belongs_to :day, foreign_key: :day_id, resource: DayResource, scope: -> { Day.all }
allow_filter :date do |scope, value|
scope.includes(:day)
.where(days: { date: value })
end
end
By adding the allow_filter
block we've told our API that we want to allow clients to pass the filter[date]
param and expect it to filter the results, based on the date provided.
# Define how to query and persist a given model.
# Further Resource documentation: https://jsonapi-suite.github.io/jsonapi_compliable/JsonapiCompliable/Resource.html
class DayResource < ApplicationResource # Used for associating this resource with a given input. # This should match the 'type' in the corresponding serializer. type :days # Associate to a Model object so we know how to persist. model Day has_many :recipes, foreign_key: :day_id, resource: RecipeResource, scope: -> { Recipe.all }
allow_filter :date do |scope, value|
scope.where(date: value)
end
end
Since we're now referencing the Day
model, we need to make sure we've setup our app to support the Day
resource.
Let's look at what goes in there...
# Define how to query and persist a given model.
# Further Resource documentation: https://jsonapi-suite.github.io/jsonapi_compliable/JsonapiCompliable/Resource.html
class DayResource < ApplicationResource # Used for associating this resource with a given input. # This should match the 'type' in the corresponding serializer. type :days # Associate to a Model object so we know how to persist. model Day has_many :recipes, foreign_key: :day_id, resource: RecipeResource, scope: -> { Recipe.all }
allow_filter :date do |scope, value|
scope.where(date: value)
end
end
First we want to specify the type, which is typically the plural form of the model name we're working with. This will be used in the response to indicate to the client what type a specific object is.
# Define how to query and persist a given model.
# Further Resource documentation: https://jsonapi-suite.github.io/jsonapi_compliable/JsonapiCompliable/Resource.html
class DayResource < ApplicationResource # Used for associating this resource with a given input. # This should match the 'type' in the corresponding serializer. type :days # Associate to a Model object so we know how to persist. model Day has_many :recipes, foreign_key: :day_id, resource: RecipeResource, scope: -> { Recipe.all }
allow_filter :date do |scope, value|
scope.where(date: value)
end
end
If we plan on allowing consumers of the API to update or persist objects, we need to tell the jsonapi_suite
gem which model to use for all of the ActiveRecord calls.
# Define how to query and persist a given model.
# Further Resource documentation: https://jsonapi-suite.github.io/jsonapi_compliable/JsonapiCompliable/Resource.html
class DayResource < ApplicationResource # Used for associating this resource with a given input. # This should match the 'type' in the corresponding serializer. type :days # Associate to a Model object so we know how to persist. model Day has_many :recipes, foreign_key: :day_id, resource: RecipeResource, scope: -> { Recipe.all }
allow_filter :date do |scope, value|
scope.where(date: value)
end
end
Then any associations that will be exposed to the API should be specified here as well. You could have more associations in the actual model that you don't expose to the API if you like. If you do add associations you need to tell the code how to find the association and how to represent it by indicating the foreign_key
and the resource
attributes, as well as whether the objects to return for this association are scoped or not.
# Define how to query and persist a given model.
# Further Resource documentation: https://jsonapi-suite.github.io/jsonapi_compliable/JsonapiCompliable/Resource.html
class DayResource < ApplicationResource # Used for associating this resource with a given input. # This should match the 'type' in the corresponding serializer. type :days # Associate to a Model object so we know how to persist. model Day has_many :recipes, foreign_key: :day_id, resource: RecipeResource, scope: -> { Recipe.all }
allow_filter :date do |scope, value|
scope.where(date: value)
end
end
Lastly, any filters that are available through the API should be defined here as well using the allow_filter
block, which includes the current scope to apply the filter on and the value of the filter passed through the API call.
# Serializers define the rendered JSON for a model instance.
# We use jsonapi-rb, which is similar to active_model_serializers.
class SerializableDay < JSONAPI::Serializable::Resource
type :days
attribute :date
has_many :recipes
end
Along with the resource class we will need the corresponding serializer. This class defines the representation of the model in the final response, so here's where you want to specify the attributes you want to include using the attribute
method, and whichever associations that you added in the resource class. The type
attribute should match the one used in the resource class above.
# curl "localhost:3000/recipes.json?filter%5Bdate%5D=2018-07-04"
{
"data": [
{
"id": "1",
"type": "recipes",
"attributes": {
"name": "Fattoush",
"description": "Fattoush is one of the most well known Middle Eastern salads and a standard dish on the 'mezza' (small dishes) table. It's a colorful tossed salad with a lemony garlic dressing, and if you've never made a single Arabic dish, this is a delicious and healthy place to start.",
"url": "https://www.allrecipes.com/recipe/223439/arabic-fattoush-salad/?internalSource=streams&referringId=235&referringContentType=recipe%20hub&clickId=st_trending_b"
},
"relationships": {
"day": {
"meta": {
"included": false
}
}
}
},
{
"id": "2",
"type": "recipes",
"attributes": {
"name": "Baked Falafel",
"description": "The baked and healthier version of the traditional falafel. Falafel is a deep-fried ball, doughnut or patty made from ground chickpeas, fava beans, or both.",
"url": "https://www.allrecipes.com/recipe/183947/baked-falafel/?internalSource=hub%20recipe&referringContentType=search%20results&clickId=cardslot%209"
},
"relationships": {
"day": {
"meta": {
"included": false
}
}
}
}
],
"meta": {},
"jsonapi": {
"version": "1.0"
}
}
So now when we pass in the date as a filter, the response we get back has been filtered appropriately.
Note that part of the response includes relationship information between the recipe and day.
Yet the response says that the day object is not included in the response. What would it take to include it?
Actually, not that much. We've done all of the work we need already, without even realizing it. The gem we are using will take care of this next step for us.
All we needed to do is specify the include=day
param in the URL...
...execute our request...
...and the information about the day is included in the response automagically!
There's something we should take special note of here. Rather than being included in each of the Recipe resources, the Day
information is listed separately at the end.
Each Recipe references the included Day
resource by ID.
# curl "localhost:3000/recipes.json?filter%5Bdate%5D=2018-07-04&include=day"
{
"data": [
{
"id": "1",
"type": "recipes",
"attributes": {
"name": "Fattoush",
"description": "Fattoush is one of the most well known Middle Eastern salads and a standard dish on the 'mezza' (small dishes) table. It's a colorful tossed salad with a lemony garlic dressing, and if you've never made a single Arabic dish, this is a delicious and healthy place to start.",
"url": "https://www.allrecipes.com/recipe/223439/arabic-fattoush-salad/?internalSource=streams&referringId=235&referringContentType=recipe%20hub&clickId=st_trending_b"
},
"relationships": {
"day": {
"data": {
"type": "days",
"id": "1"
}
}
}
},
{
"id": "2",
"type": "recipes",
"attributes": {
"name": "Baked Falafel",
"description": "The baked and healthier version of the traditional falafel. Falafel is a deep-fried ball, doughnut or patty made from ground chickpeas, fava beans, or both.",
"url": "https://www.allrecipes.com/recipe/183947/baked-falafel/?internalSource=hub%20recipe&referringContentType=search%20results&clickId=cardslot%209"
},
"relationships": {
"day": {
"data": {
"type": "days",
"id": "1"
}
}
}
}
],
"included": [
{
"id": "1",
"type": "days",
"attributes": {
"date": "2018-07-04"
},
"relationships": {
"recipes": {
"meta": {
"included": false
}
}
}
}
],
"meta": {},
"jsonapi": {
"version": "1.0"
}
}
So even though we've got 2 recipes associated to the one day with date "2018-07-04"
, the day object itself is included only once in the response. This minimize the data transmitted and makes our API faster. The client will have to do the work to re-associated the recipe and the day objects, but if you are using any of the JSON API libraries like jsonapi_suite
, the library will handle doing all of that work for you.
This illustrates what makes this solution flexible. Without any work on the backend side, the client can specify what to include in the response and limit the amount of data transmitted to only the items required. The same is true for attributes.
# Benefits of using a spec
# 1. Decreased effort to implement changes
# 2. Flexible by default
# 3. Self documenting API
# 4. Consistency
To summarize, by following a spec we can
# Benefits of using a spec
# 1. Decreased effort to implement changes
# 2. Flexible by default
# 3. Self documenting API
# 4. Consistency
reduce the amount of work we need to do in order to support changes,
# Benefits of using a spec
# 1. Decreased effort to implement changes
# 2. Flexible by default
# 3. Self documenting API
# 4. Consistency
we are able to support flexibility out of the box,
# Benefits of using a spec
# 1. Decreased effort to implement changes
# 2. Flexible by default
# 3. Self documenting API
# 4. Consistency
we adopt a self documenting API,
# Benefits of using a spec
# 1. Decreased effort to implement changes
# 2. Flexible by default
# 3. Self documenting API
# 4. Consistency
and we promote consistency across our API making it much easier to consume on the client side and for developers to work with on the backend side.
Next time you are thinking of designing and building an API I hope you give the JSON API spec a look.
Happy Hacking!
Responses