JSON API in Rails – Part 2

In the previous episode, we looked at building an API and how an unstructured response can create a bunch of issues that we would have to deal with.

# Potential Risks:
# 1. Cost of change
# 2. Documentation
# 3. Collaboration
# 4. Flexibility

We summarized the risks under the following 4 categories: Cost of change: what's the effort required of us to update the API response any time we introduce a change to the app? Documentation: is our API easy to document, and how fast does the documentation go stale after changes? Collaboration: how much energy do we have to exert to make sure collaboration with fellow developers on the API is a smooth process that follows the same standards? Flexibility: can the client choose what response to get back from the API or is it rigid and forces everyone to deal with the same objects being returned even if it doesn't fit their needs?

It takes a lot of work, especially upfront work as a team, to resolve these issues. Luckily we don't have to do that work ourselves because a bunch of really smart people came up with a solution.

Enter the JSON API spec.

The JSON API website describes it as:

JSON API is a specification for how a client should request that resources be fetched or modified, and how a server should respond to those requests. JSON API is designed to minimize both the number of requests and the amount of data transmitted between clients and servers. This efficiency is achieved without compromising readability, flexibility, or discoverability.

Let's unpack that a bit. "JSON API is a specification for how a client should request that resources be fetched or modified" What that means is that the spec provides a common way for how any client (a mobile app, web app, or any consumer of an API) sends requests to the API to fetch a resource, or in other terms GET/PUT/POST/DELETE requests.

"and how a server should respond to those requests" means that the spec provides a common way the API structures any and all responses to requests the API receives.

# Potential Risks:
# 1. Cost of change
# 2. Documentation
# 3. Collaboration
# 4. Flexibility

Let's look at how using JSON API would alleviate the risks we discussed earlier.

# Potential Risks:
# 1. Cost of change
# 2. Documentation
# 3. Collaboration
# 4. Flexibility

Since the the spec provides a convention over configuration framework for an API, the cost of change is reduced. We will look later at how using the proper tools will reduce the amount of work we would have to do as devs.

# Potential Risks:
# 1. Cost of change
# 2. Documentation
# 3. Collaboration
# 4. Flexibility

Using the fact that JSON API is a spec that our API would follow, the spec itself becomes part of the shared documentation and a reference we can use for our own API. We would still have to do some work to document the specific end points our API provides, but we wouldn't need to provide specific documentation on how to consume the API because that's all part of the spec. Tools like Swagger, which you might have seen in Episode #527, could also be used to automate the documentation and reduce the amount of effort we need to put into it.

# Potential Risks:
# 1. Cost of change
# 2. Documentation
# 3. Collaboration
# 4. Flexibility

The problems we faced previously resulting from collaborating with other team members to create our API will automatically disappear by the nature of us following a spec.

# Potential Risks:
# 1. Cost of change
# 2. Documentation
# 3. Collaboration
# 4. Flexibility

Finally, the spec provides a way for the client to specify what to request from the API, and thus introduces a level of flexibility that didn't exist in the implementation we initially came up with.

It may seem to good to be true, that the risks we talked about magically disappear by using the JSON API spec. And that's because this is precisely why the spec was created. There's still a bit of work on our end, so let's show how we can create the same API for our recipe app using JSON API.

For this portion, we will use a gem called jsonapi_suite, which is technically a collection of gems that we can use to support the JSON API spec in our Ruby on Rails application.

source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }

ruby '2.5.0'

# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '~> 5.2.0'
# Use postgresql as the database for Active Record
gem 'pg', '>= 0.18', '< 2.0' # Use Puma as the app server gem 'puma', '~> 3.11'
# Build JSON APIs with ease. Read more: https://github.com/rails/jbuilder
# gem 'jbuilder', '~> 2.5'
# Use ActiveModel has_secure_password
# gem 'bcrypt', '~> 3.1.7'

# Use Capistrano for deployment
# gem 'capistrano-rails', group: :development

# Reduces boot times through caching; required in config/boot.rb
gem 'bootsnap', '>= 1.1.0', require: false

# Use Rack CORS for handling Cross-Origin Resource Sharing (CORS), making cross-origin AJAX possible
# gem 'rack-cors'

gem 'jsonapi_suite', '~> 0.7'
gem 'jsonapi-rails', '~> 0.3.0'
gem 'kaminari',      '~> 0.17'

group :development, :test do
  # Call 'byebug' anywhere in the code to stop execution and get a debugger console
  gem 'byebug', platforms: [:mri, :mingw, :x64_mingw]

  gem 'foreman', '~> 0.84.0', require: false

  # Annotate models, test, fixtures, etc with the table schema
  gem 'annotate'
end

group :development do
  gem 'listen', '>= 3.0.5', '< 3.2' # Spring speeds up development by keeping your application running in the background. Read more: https://github.com/rails/spring gem 'spring' gem 'spring-watcher-listen', '~> 2.0.0'
end


# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby]

We'll start by adding the gem to our Gemfile and running bundle install.

$ bin/rails generate jsonapi_suite:install

Then we'll run the installer that comes with the gem.

$ bin/rails generate jsonapi_suite:install
Running via Spring preloader in process 182
  create  config/initializers/jsonapi.rb
  create  config/initializers/strong_resources.rb
  insert  app/controllers/application_controller.rb
  insert  app/controllers/application_controller.rb

This will create a couple of files and insert some code into our ApplicationController. You can check out the gem's documentation for further detail about this step.

$ bin/rails g
Jsonapi:
  jsonapi:initializer
  jsonapi:resource
  jsonapi:serializable

The jsonapi_suite gem comes with a couple of handy generators, namely jsonapi:resource and jsonapi:serializable. We will use the jsonapi:resource generator the first time out to see what the expected code looks like. Feel free to continue using the generator afterwards or manually creating the files once you've got a good handle on what is required.

$ bin/rails g jsonapi:resource Recipe --skip-namespace

Since the main resource we are dealing with is a Recipe, that's what we'll use in the generator. We are also specifying the --skip-namespace flag since our app is strictly an API app, so we do not need the /api namespace or any other namespace for that matter. This command generates a few files, so let's take a look at each individually.

# app/controllers/recipes_controller.rb
class RecipesController < ApplicationController
  # Mark this as a JSONAPI controller, associating with the given resource
  jsonapi resource: RecipeResource

  # Reference a strong resource payload defined in
  # config/initializers/strong_resources.rb
  strong_resource :recipe
  # Run strong parameter validation for these actions.
  # Invalid keys will be dropped.
  # Invalid value types will log or raise based on the configuration
  # ActionController::Parameters.action_on_invalid_parameters
  before_action :apply_strong_params, only: [:create, :update]

  # Start with a base scope and pass to render_jsonapi
  def index
    recipes = Recipe.all
    render_jsonapi(recipes)
  end

  # Call jsonapi_scope directly here so we can get behavior like
  # sparse fieldsets and statistics.
  def show
    scope = jsonapi_scope(Recipe.where(id: params[:id]))
    instance = scope.resolve.first
    raise JsonapiCompliable::Errors::RecordNotFound unless instance
    render_jsonapi(instance, scope: false)
  end

  # jsonapi_create will use the configured Resource (and adapter) to persist.
  # This will handle nested relationships as well.
  # On validation errors, render correct error JSON.
  def create
    recipe, success = jsonapi_create.to_a

    if success
      render_jsonapi(recipe, scope: false)
    else
      render_errors_for(recipe)
    end
  end

  # jsonapi_update will use the configured Resource (and adapter) to persist.
  # This will handle nested relationships as well.
  # On validation errors, render correct error JSON.
  def update
    recipe, success = jsonapi_update.to_a

    if success
      render_jsonapi(recipe, scope: false)
    else
      render_errors_for(recipe)
    end
  end

  # Renders 200 OK with empty meta
  # http://jsonapi.org/format/#crud-deleting-responses-200
  def destroy
    recipe, success = jsonapi_destroy.to_a

    if success
      render json: { meta: {} }
    else
      render_errors_for(recipe)
    end
  end
end

This is the controller that gets generated. There's a lot of boilerplate code in here. It doesn't look too different from a regular Ruby on Rails controller, especially if you've ever used the responders gem. The main feature we will focus on is the use of the render_jsonapi method. Under the hood this method uses the other files that got generated to form the response that will conform to the JSON API spec.

# app/resources/recipe_resource.rb
# Define how to query and persist a given model.
# Further Resource documentation: https://jsonapi-suite.github.io/jsonapi_compliable/JsonapiCompliable/Resource.html
class RecipeResource < ApplicationResource # Used for associating this resource with a given input. # This should match the 'type' in the corresponding serializer. type :recipes # Associate to a Model object so we know how to persist. model Recipe # Customize your resource here. Some common examples: # # === Allow ?filter[name] query parameter === # allow_filter :name # # === Enable total count, when requested === # allow_stat total: [:count] # # === Allow sideloading/sideposting of relationships === # belongs_to :foo, # foreign_key: :foo_id, # resource: FooResource, # scope: -> { Foo.all }
  #
  # === Custom sorting logic ===
  # sort do |scope, att, dir|
  #   ... code ...
  # end
  #
  # === Change default sort ===
  # default_sort([{ title: :asc }])
  #
  # === Custom pagination logic ===
  # paginate do |scope, current_page, per_page|
  #   ... code ...
  # end
  #
  # === Change default page size ===
  # default_page_size(10)
  #
  # === Change how we resolve the scope ===
  # def resolve(scope)
  #   ... code ...
  # end
end

The resource file is how we tie the JSON API resource to the underlying model, including how to query, filter, paginate, or modify a given model. We'll get back to this file in a bit.

# app/serializers/serializable_recipe.rb
# Serializers define the rendered JSON for a model instance.
# We use jsonapi-rb, which is similar to active_model_serializers.
class SerializableRecipe < JSONAPI::Serializable::Resource
  type :recipes

  # Add attributes here to ensure they get rendered, .e.g.
  #
  # attribute :name
  #
  # To customize, pass a block and reference the underlying @object
  # being serialized:
  #
  # attribute :name do
  #   @object.name.upcase
  # end
end

The serializer determines how the model is transformed into a JSON object, and what data is sent back in the payload. If the API wants to make an attribute available in the response it should be specified here. The client will be able to specify in the request whether to include all of the attributes or a select list, but only attributes set here would be available. This is where you can choose to not make certain attributes public, such as a password or any other sensitive data.

# curl "localhost:3000/recipes.json?filter%5Bdate%5D=2018-07-04"
{
  "errors": [
    {
      "code": "internal_server_error",
      "status": "500",
      "title": "Error",
      "detail": "We've notified our engineers and hope to address this issue shortly.",
      "meta": {}
    }
  ]
}

Let's test out our current setup, using the same curl command we used previously.

This time we get an error. Before we look into the reason for this error, check out the structured error information that the jsonapi_suite gem provides! There are separate fields for status code, title, details, etc.

Started GET "/recipes.json?filter%5Bdate%5D=2018-07-04"
Processing by RecipesController#index as JSON
Parameters: {"filter"=>{"date"=>"2018-07-04"}}
ERROR: JsonapiCompliable::Errors::BadFilter: JsonapiCompliable::Errors::BadFilter
app/controllers/recipes_controller.rb:17:in `index'

After looking at the logs, we see that we sent a request that the API did not expect. Mainly the filter param that we sent is not supported.

# curl "localhost:3000/recipes.json"
{
  "data": [
    {
      "id": "1",
      "type": "recipes"
    },
    {
      "id": "3",
      "type": "recipes"
    },
    {
      "id": "4",
      "type": "recipes"
    },
    {
      "id": "2",
      "type": "recipes"
    }
  ],
  "meta": {},
  "jsonapi": {
    "version": "1.0"
  }
}

After removing the filter params we start getting back a response. Albeit not a very useful one, but still a response. This response matches the format expected by the JSON API spec. So we know we're on track. What we need to do is specify the fields we want the JSON response to include as well as enabling the filtering.

# Serializers define the rendered JSON for a model instance.
# We use jsonapi-rb, which is similar to active_model_serializers.
class SerializableRecipe < JSONAPI::Serializable::Resource
  type :recipes

  attribute :name
  attribute :description
  attribute :url
end

A simple edit to the serializer file to specify the attributes we want the JSON response to include should to the trick.

# curl "localhost:3000/recipes.json"
{
  "data": [
    {
      "id": "1",
      "type": "recipes",
      "attributes": {
        "name": "Fattoush",
        "description": "Fattoush is one of the most well known Middle Eastern salads and a standard dish on the 'mezza' (small dishes) table. It's a colorful tossed salad with a lemony garlic dressing, and if you've never made a single Arabic dish, this is a delicious and healthy place to start.",
        "url": "https://www.allrecipes.com/recipe/223439/arabic-fattoush-salad/?internalSource=streams&referringId=235&referringContentType=recipe%20hub&clickId=st_trending_b"
      }
    },
    {
      "id": "3",
      "type": "recipes",
      "attributes": {
        "name": "Baba Ghanoush",
        "description": "This wonderful dip is made with roasted eggplant, tahini, lemon and lots of roasted garlic. Even if you don't usually like eggplant, you just may love this. It is the perfect starter for a Middle Eastern dinner. I like to serve this with generous strips of green, red and orange bell peppers for dipping. (Remember that each bell pepper is about 6g net carbs). It is also good stuffed into mushroom, celery or cherry tomatoes.",
        "url": "https://www.allrecipes.com/recipe/81603/baba-ghanoush/"
      }
    },
    {
      "id": "4",
      "type": "recipes",
      "attributes": {
        "name": "Shish Tawook",
        "description": "Shish Tawook is a traditional marinated chicken shish kebab of Middle Eastern cuisine.",
        "url": "https://www.allrecipes.com/recipe/150251/shish-tawook-grilled-chicken/?internalSource=hub%20recipe&referringId=235&referringContentType=recipe%20hub&clickId=cardslot%2022"
      }
    },
    {
      "id": "2",
      "type": "recipes",
      "attributes": {
        "name": "Baked Falafel",
        "description": "The baked and healthier version of the traditional falafel. Falafel is a deep-fried ball, doughnut or patty made from ground chickpeas, fava beans, or both.",
        "url": "https://www.allrecipes.com/recipe/183947/baked-falafel/?internalSource=hub%20recipe&referringContentType=search%20results&clickId=cardslot%209"
      }
    }
  ],
  "meta": {},
  "jsonapi": {
    "version": "1.0"
  }
}

That looks much better!

# Define how to query and persist a given model.
# Further Resource documentation: https://jsonapi-suite.github.io/jsonapi_compliable/JsonapiCompliable/Resource.html
class RecipeResource < ApplicationResource # Used for associating this resource with a given input. # This should match the 'type' in the corresponding serializer. type :recipes # Associate to a Model object so we know how to persist. model Recipe belongs_to :day, foreign_key: :day_id, resource: DayResource, scope: -> { Day.all }

  allow_filter :date do |scope, value|
    scope.includes(:day)
      .where(days: { date: value })
  end
end

By adding the allow_filter block we've told our API that we want to allow clients to pass the filter[date] param and expect it to filter the results, based on the date provided.

# Define how to query and persist a given model.
# Further Resource documentation: https://jsonapi-suite.github.io/jsonapi_compliable/JsonapiCompliable/Resource.html
class DayResource < ApplicationResource # Used for associating this resource with a given input. # This should match the 'type' in the corresponding serializer. type :days # Associate to a Model object so we know how to persist. model Day has_many :recipes, foreign_key: :day_id, resource: RecipeResource, scope: -> { Recipe.all }

  allow_filter :date do |scope, value|
    scope.where(date: value)
  end
end

Since we're now referencing the Day model, we need to make sure we've setup our app to support the Day resource. Let's look at what goes in there...

# Define how to query and persist a given model.
# Further Resource documentation: https://jsonapi-suite.github.io/jsonapi_compliable/JsonapiCompliable/Resource.html
class DayResource < ApplicationResource # Used for associating this resource with a given input. # This should match the 'type' in the corresponding serializer. type :days # Associate to a Model object so we know how to persist. model Day has_many :recipes, foreign_key: :day_id, resource: RecipeResource, scope: -> { Recipe.all }

  allow_filter :date do |scope, value|
    scope.where(date: value)
  end
end

First we want to specify the type, which is typically the plural form of the model name we're working with. This will be used in the response to indicate to the client what type a specific object is.

# Define how to query and persist a given model.
# Further Resource documentation: https://jsonapi-suite.github.io/jsonapi_compliable/JsonapiCompliable/Resource.html
class DayResource < ApplicationResource # Used for associating this resource with a given input. # This should match the 'type' in the corresponding serializer. type :days # Associate to a Model object so we know how to persist. model Day has_many :recipes, foreign_key: :day_id, resource: RecipeResource, scope: -> { Recipe.all }

  allow_filter :date do |scope, value|
    scope.where(date: value)
  end
end

If we plan on allowing consumers of the API to update or persist objects, we need to tell the jsonapi_suite gem which model to use for all of the ActiveRecord calls.

# Define how to query and persist a given model.
# Further Resource documentation: https://jsonapi-suite.github.io/jsonapi_compliable/JsonapiCompliable/Resource.html
class DayResource < ApplicationResource # Used for associating this resource with a given input. # This should match the 'type' in the corresponding serializer. type :days # Associate to a Model object so we know how to persist. model Day has_many :recipes, foreign_key: :day_id, resource: RecipeResource, scope: -> { Recipe.all }

  allow_filter :date do |scope, value|
    scope.where(date: value)
  end
end

Then any associations that will be exposed to the API should be specified here as well. You could have more associations in the actual model that you don't expose to the API if you like. If you do add associations you need to tell the code how to find the association and how to represent it by indicating the foreign_key and the resource attributes, as well as whether the objects to return for this association are scoped or not.

# Define how to query and persist a given model.
# Further Resource documentation: https://jsonapi-suite.github.io/jsonapi_compliable/JsonapiCompliable/Resource.html
class DayResource < ApplicationResource # Used for associating this resource with a given input. # This should match the 'type' in the corresponding serializer. type :days # Associate to a Model object so we know how to persist. model Day has_many :recipes, foreign_key: :day_id, resource: RecipeResource, scope: -> { Recipe.all }

  allow_filter :date do |scope, value|
    scope.where(date: value)
  end
end

Lastly, any filters that are available through the API should be defined here as well using the allow_filter block, which includes the current scope to apply the filter on and the value of the filter passed through the API call.

# Serializers define the rendered JSON for a model instance.
# We use jsonapi-rb, which is similar to active_model_serializers.
class SerializableDay < JSONAPI::Serializable::Resource
  type :days

  attribute :date

  has_many :recipes
end

Along with the resource class we will need the corresponding serializer. This class defines the representation of the model in the final response, so here's where you want to specify the attributes you want to include using the attribute method, and whichever associations that you added in the resource class. The type attribute should match the one used in the resource class above.

# curl "localhost:3000/recipes.json?filter%5Bdate%5D=2018-07-04"
{
  "data": [
    {
      "id": "1",
      "type": "recipes",
      "attributes": {
        "name": "Fattoush",
        "description": "Fattoush is one of the most well known Middle Eastern salads and a standard dish on the 'mezza' (small dishes) table. It's a colorful tossed salad with a lemony garlic dressing, and if you've never made a single Arabic dish, this is a delicious and healthy place to start.",
        "url": "https://www.allrecipes.com/recipe/223439/arabic-fattoush-salad/?internalSource=streams&referringId=235&referringContentType=recipe%20hub&clickId=st_trending_b"
      },
      "relationships": {
        "day": {
          "meta": {
            "included": false
          }
        }
      }
    },
    {
      "id": "2",
      "type": "recipes",
      "attributes": {
        "name": "Baked Falafel",
        "description": "The baked and healthier version of the traditional falafel. Falafel is a deep-fried ball, doughnut or patty made from ground chickpeas, fava beans, or both.",
        "url": "https://www.allrecipes.com/recipe/183947/baked-falafel/?internalSource=hub%20recipe&referringContentType=search%20results&clickId=cardslot%209"
      },
      "relationships": {
        "day": {
          "meta": {
            "included": false
          }
        }
      }
    }
  ],
  "meta": {},
  "jsonapi": {
    "version": "1.0"
  }
}

So now when we pass in the date as a filter, the response we get back has been filtered appropriately.

Note that part of the response includes relationship information between the recipe and day.

Yet the response says that the day object is not included in the response. What would it take to include it?

Actually, not that much. We've done all of the work we need already, without even realizing it. The gem we are using will take care of this next step for us.

All we needed to do is specify the include=day param in the URL...

...execute our request...

...and the information about the day is included in the response automagically!

There's something we should take special note of here. Rather than being included in each of the Recipe resources, the Day information is listed separately at the end.

Each Recipe references the included Day resource by ID.

# curl "localhost:3000/recipes.json?filter%5Bdate%5D=2018-07-04&include=day"
{
  "data": [
    {
      "id": "1",
      "type": "recipes",
      "attributes": {
        "name": "Fattoush",
        "description": "Fattoush is one of the most well known Middle Eastern salads and a standard dish on the 'mezza' (small dishes) table. It's a colorful tossed salad with a lemony garlic dressing, and if you've never made a single Arabic dish, this is a delicious and healthy place to start.",
        "url": "https://www.allrecipes.com/recipe/223439/arabic-fattoush-salad/?internalSource=streams&referringId=235&referringContentType=recipe%20hub&clickId=st_trending_b"
      },
      "relationships": {
        "day": {
          "data": {
            "type": "days",
            "id": "1"
          }
        }
      }
    },
    {
      "id": "2",
      "type": "recipes",
      "attributes": {
        "name": "Baked Falafel",
        "description": "The baked and healthier version of the traditional falafel. Falafel is a deep-fried ball, doughnut or patty made from ground chickpeas, fava beans, or both.",
        "url": "https://www.allrecipes.com/recipe/183947/baked-falafel/?internalSource=hub%20recipe&referringContentType=search%20results&clickId=cardslot%209"
      },
      "relationships": {
        "day": {
          "data": {
            "type": "days",
            "id": "1"
          }
        }
      }
    }
  ],
  "included": [
    {
      "id": "1",
      "type": "days",
      "attributes": {
        "date": "2018-07-04"
      },
      "relationships": {
        "recipes": {
          "meta": {
            "included": false
          }
        }
      }
    }
  ],
  "meta": {},
  "jsonapi": {
    "version": "1.0"
  }
}

So even though we've got 2 recipes associated to the one day with date "2018-07-04", the day object itself is included only once in the response. This minimize the data transmitted and makes our API faster. The client will have to do the work to re-associated the recipe and the day objects, but if you are using any of the JSON API libraries like jsonapi_suite, the library will handle doing all of that work for you.

This illustrates what makes this solution flexible. Without any work on the backend side, the client can specify what to include in the response and limit the amount of data transmitted to only the items required. The same is true for attributes.

# Benefits of using a spec
# 1. Decreased effort to implement changes
# 2. Flexible by default
# 3. Self documenting API
# 4. Consistency

To summarize, by following a spec we can

# Benefits of using a spec
# 1. Decreased effort to implement changes
# 2. Flexible by default
# 3. Self documenting API
# 4. Consistency

reduce the amount of work we need to do in order to support changes,

# Benefits of using a spec
# 1. Decreased effort to implement changes
# 2. Flexible by default
# 3. Self documenting API
# 4. Consistency

we are able to support flexibility out of the box,

# Benefits of using a spec
# 1. Decreased effort to implement changes
# 2. Flexible by default
# 3. Self documenting API
# 4. Consistency

we adopt a self documenting API,

# Benefits of using a spec
# 1. Decreased effort to implement changes
# 2. Flexible by default
# 3. Self documenting API
# 4. Consistency

and we promote consistency across our API making it much easier to consume on the client side and for developers to work with on the backend side.

Next time you are thinking of designing and building an API I hope you give the JSON API spec a look.

Happy Hacking!