In Progress
Unit 1, Lesson 1
In Progress

JSON API in Rails – Part 1

So it’s Monday, and the top ticket on the board is to add a simple RESTfu JSON-based API to your application. Using Ruby on Rails, this is simple enough to implement, and you know you can bang it out by the end of the day.

But it’s one thing to add a basic endpoint to spew some JSON data at a client. It’s another to gracefully grow an API that’s consistent, predictable, well-documented, has sensible error reporting, and supports more complex queries from the client side. Fortunately, the JSON API standard has emerged to help take some of the guesswork out of how to structure a robust and extensible HTTP API. In today’s episode, guest chef Youssef Chaker joins us to discuss some of the risks in adding a quick, ad-hoc JSON endpoint. In part 2 of this series, he’ll return and show us how to use JSON-API gems to quickly and painlessly add an industrial-grade RESTful interface to our applications. Enjoy!

Video transcript & code

API Diagram

APIs have become ubiquitous, and chances are that if you're working on a web application that you'll be developing your own API as well. Whether it's an API to be consumed by third party apps, or an internal API that is consumed by the client side like a mobile app or your Javascript framework of choice.

Rails Routes Guide

Convention around APIs focuses on the RESTful aspect of APIs, so essentially the format of the path and HTTP verb to use, as well as the response codes.

Here's an example from the Rails guides that highlights the structure of a RESTful API.

But what about the convention around the structure of the response returned by the API?

To illustrate what I mean, let's take a look at an example.

Our app is a recipe app for a catering company. The API powers mobile apps that display to the user the recipes we are serving on a particular day.


# == Schema Information
#
# Table name: days
#
#  id         :bigint(8)        not null, primary key
#  date       :date
#  created_at :datetime         not null
#  updated_at :datetime         not null
#

class Day < ApplicationRecord
  has_many :recipes
end

We've setup a Day model in our app which has a date attribute, and a Day has many recipes.


# == Schema Information
#
# Table name: recipes
#
#  id          :bigint(8)        not null, primary key
#  description :text
#  name        :string
#  created_at  :datetime         not null
#  updated_at  :datetime         not null
#  day_id      :bigint(8)
#
# Indexes
#
#  index_recipes_on_day_id  (day_id)
#
# Foreign Keys
#
#  fk_rails_...  (day_id => days.id)
#

class Recipe < ApplicationRecord
  belongs_to :day
end

The Recipe model has a name and a description.


date    = '2018-07-04'
day     = Day.where(date: date).first
recipes = day.recipes

# => [
#  #,
#  #
# ]

We can fetch the recipes for a particular day by searching for the Day record that matches the date and then return the recipes associated with it.


class RecipesController < ApplicationController
  respond_to :json

  def index
    if params[:filter] && params[:filter].fetch(:date) { nil }
      day = Day.where(date: params[:filter][:date]).first

      @recipes = day.recipes
    else
      @recipes = Recipe.all
    end

    respond_with(@recipes)
  end
end

The corresponding API for our app would need a controller that performs the same query to the database, using the params object to filter on the date.


#
# /recipes.json?filter[date]=2018-07-04
# curl "localhost:3000/recipes.json?filter%5Bdate%5D=2018-07-04"
# => [
#   {
#     "id": 1,
#     "day_id": 1,
#     "name": "Fattoush",
#     "description": "Fattoush is one of the most well known Middle Eastern salads and a standard dish on the 'mezza' (small dishes) table. It's a colorful tossed salad with a lemony garlic dressing, and if you've never made a single Arabic dish, this is a delicious and healthy place to start.",
#     "created_at": ...,
#     "updated_at": ...
#   },
#   {
#     "id": 2,
#     "day_id": 1,
#     "name": "Baked Falafel",
#     "description": "The baked and healthier version of the traditional falafel. Falafel is a deep-fried ball, doughnut or patty made from ground chickpeas, fava beans, or both.",
#     "created_at": ...,
#     "updated_at": ...
#   }
# ]

The response would look something like this:

Great! But what if the requirements changed? Just like any other app, our recipes app will keep receiving improvements and updates.


class AddUrlToRecipes < ActiveRecord::Migration[5.2]
  def change
    add_column :recipes, :url, :string
  end
end

Let's say we want to add a allrecipes.com url to our recipes so that our users can follow instructions and make a meal at home.

What kind of effort would it require to reflect this update in the API?


# curl "localhost:3000/recipes.json?filter%5Bdate%5D=2018-07-04"
# => [
#   {
#     "id": 1,
#     "day_id": 1,
#     "name": "Fattoush",
#     "description": "Fattoush is one of the most well known Middle Eastern salads and a standard dish on the 'mezza' (small dishes) table. It's a colorful tossed salad with a lemony garlic dressing, and if you've never made a single Arabic dish, this is a delicious and healthy place to start.",
#     "created_at": ...,
#     "updated_at": ...,
#     "url": "https://www.allrecipes.com/recipe/223439/arabic-fattoush-salad/?internalSource=streams&referringId=235&referringContentType=recipe%20hub&clickId=st_trending_b"
#   },
#   {
#     "id": 2,
#     "day_id": 1,
#     "name": "Baked Falafel",
#     "description": "The baked and healthier version of the traditional falafel. Falafel is a deep-fried ball, doughnut or patty made from ground chickpeas, fava beans, or both.",
#     "created_at": ...,
#     "updated_at": ...,
#     "url": "https://www.allrecipes.com/recipe/183947/baked-falafel/?internalSource=hub%20recipe&referringContentType=search%20results&clickId=cardslot%209"
#   }
# ]

As it turns out, we don't have to do anything at all. Since we are using the Responders gem, all we need to do is add the attribute on the model and it is automatically reflected in the API response.

That wasn't bad, but that was also a very simplistic example. Let's look at a more complex scenario.

We've decided to show ingredients with our recipes to give people the chance to identify any allergens that might be present.


# == Schema Information
#
# Table name: ingredients
#
#  id               :bigint(8)        not null, primary key
#  measurement_type :string
#  name             :string
#  quantity         :float
#  unit             :string
#  created_at       :datetime         not null
#  updated_at       :datetime         not null
#  recipe_id        :bigint(8)
#
# Indexes
#
#  index_ingredients_on_recipe_id  (recipe_id)
#
# Foreign Keys
#
#  fk_rails_...  (recipe_id => recipes.id)
#

class Ingredient < ApplicationRecord
  belongs_to :recipe
end

The Ingredient model is simple. It contains a name and some measurement and quantity information.


class RecipesController < ApplicationController
  respond_to :json

  def index
    if params[:filter] && params[:filter].fetch(:date) { nil }
      day = Day.where(date: params[:filter][:date]).first

      @recipes = day.recipes.includes(:ingredients)
    else
      @recipes = Recipe.all
    end

    respond_with(@recipes)
  end
end

To return the ingredients in the API response, it is not enough to make sure the ingredients are fetched from the database along with the recipes.


# app/views/recipes/index.json.jbuilder
json.array! @recipes, partial: 'recipes/recipe', as: :recipe

First, let's update our view to use jbuilder, a gem that gives us a simple DSL for declaring JSON structures. In current versions of rails, this gem is included by default.


# app/views/recipes/_recipe.json.jbuilder
json.name        recipe.name
json.description recipe.description
json.url         recipe.url

json.ingredients do
  json.array! recipe.ingredients, partial: 'recipes/ingredient', as: :ingredient
end

Here's where we can instruct our API to return the corresponding ingredients with a recipe.


# app/views/recipes/_ingredient.json.jbuilder
json.name             ingredient.name
json.measurement_type ingredient.measurement_type
json.quantity         ingredient.quantity
json.unit             ingredient.unit


And the JSON object for an ingredient will include the following attributes.


# curl "localhost:3000/recipes.json?filter%5Bdate%5D=2018-07-04"
# => [
#   {
#     "name": "Fattoush",
#     "description": "Fattoush is one of the most well known Middle Eastern salads and a standard dish on the 'mezza' (small dishes) table. It's a colorful tossed salad with a lemony garlic dressing, and if you've never made a single Arabic dish, this is a delicious and healthy place to start.",
#     "url": "https://www.allrecipes.com/recipe/223439/arabic-fattoush-salad/?internalSource=streams&referringId=235&referringContentType=recipe%20hub&clickId=st_trending_b",
#     "ingredients": [
#       {
#         "name": "vegetable oil for frying",
#         "measurement_type": "weight",
#         "quantity": 1.0,
#         "unit": "tablespoon"
#       },
#       ...
#     ]
#   },
#   {
#     "name": "Baked Falafel",
#     "description": "The baked and healthier version of the traditional falafel. Falafel is a deep-fried ball, doughnut or patty made from ground chickpeas, fava beans, or both.",
#     "url": "https://www.allrecipes.com/recipe/183947/baked-falafel/?internalSource=hub%20recipe&referringContentType=search%20results&clickId=cardslot%209",
#     "ingredients": [
#       {
#         "name": "chopped onion",
#         "measurement_type": "weight",
#         "quantity": 0.5,
#         "unit": "cup"
#       },
#       ...
#     ]
#   }
# ]

The request response now looks something like this.

At this point you might be thinking that things are going well.


# Potential Risks:

From many years of building APIs on small and large projects, I can confidently say that this approach leaves us vulnerable to many risks


# Potential Risks:
# 1. Cost of change

Besides the work that needs to be done on the API app itself, we need to keep in mind the cost of development whenever we introduce a change on the client side. After all, APIs are built to be consumed. Even if we introduce versioning to our API, the cost of migrating from one version to another could be very high.


# Potential Risks:
# 1. Cost of change
# 2. Documentation

We looked at one example with our recipe app, but consider the evolution of it into something that could have a dozen endpoints with 10s of models. What's the effort required to document every endpoint and every model, with every optional parameter and filter that you may support? It can be very daunting to document something like this. And even if the API is an internal one, you still need to maintain documentation for your fellow developers and your future self 6 months down the line.


# Potential Risks:
# 1. Cost of change
# 2. Documentation
# 3. Collaboration

Chances are that you're not the only developer working on this app. Turns out, while we were talking about the risks, a colleague was working on another branch of the app, and their branch introduced a new endpoint to return all of the days for which we have recipes.


# curl "localhost:3000/days.json"
# => {
#   "days": [
#     {
#       "date": "2018-07-04"
#     },
#     {
#       "date": "2018-07-05"
#     }
#   ]
# }

This is the response of the API that our colleague wrote. Notice anything odd or different?


# curl "localhost:3000/days.json"    |   # curl "localhost:3000/recipes.json?filter%5Bdate%5D=2018-07-04"
# => {                               |   # => [
#   "days": [                        |   #   {
#     {                              |   #     "name": "Fattoush",
#       "date": "2018-07-04"         |   #     ...
#     },                             |   #   },
#     {                              |   #   {
#       "date": "2018-07-05"         |   #     "name": "Baked Falafel",
#     }                              |   #     ...
#   ]                                |   #   },
# }                                  |   # ]

Here are the two responses side by side


# curl "localhost:3000/days.json"    |   # curl "localhost:3000/recipes.json?filter%5Bdate%5D=2018-07-04"
# => {                               |   # => [
#   "days": [                        |   #   {
#     {                              |   #     "name": "Fattoush",
#       "date": "2018-07-04"         |   #     ...
#     },                             |   #   },
#     {                              |   #   {
#       "date": "2018-07-05"         |   #     "name": "Baked Falafel",
#     }                              |   #     ...
#   ]                                |   #   },
# }                                  |   # ]

The recipes response on the right returns an array of JSON objects, each representing a single recipe. Whereas the days response on the left returns a JSON object with a key called days, and the value corresponding to that key is an array of days. The difference is subtle, but important. This type of inconsistency can cause confusion, and requires the developers consuming your API to have to consult your documentation to figure out what format to expect. A friendlier version of the API would be consistent and thus allowing the consumers of the API to assume a consistent format.


# Potential Risks:
# 1. Cost of change
# 2. Documentation
# 3. Collaboration
# 4. Flexibility

Another risk is flexibility. In the response object of the recipes API we are now bound to always return the ingredients. The consumer of the API does not have the flexibility to specify what to include or not in the response. You might be tempted to say that passing a flag in the request params would work in showing or hiding the ingredients section, but that's not a flexible or scalable solution. What happens when the app grows and includes more attributes and more associations? Would you create a flag for each one of them?

If you're thinking that there has to be a better, more flexible, and more scalable way to build APIs. A way that is self documenting the way beautiful ruby code is. Then you're in luck.

Stay tuned to the second part of this series where we introduce a solution to all of the risks we just discussed.

 

Responses