(Part 1)

Part 2: Responsible Monkey-Patching

This time, we don’t have a natural starting place because we don’t ever pull things from the queue—that’s Sidekiq’s job! Instead of using the console, let’s go read some code on GitHub and see if we can find some clues.

ActiveJob is a gem that lives on GitHub at rails/rails. Let’s start by looking at the queue adapter for Sidekiq.

Some of this code looks familiar—we already saw the #enqueue method while we were spelunking. Aside from the enqueue methods, the only other bit is a method called perform, which makes a call to ActiveJob::Base.execute. That looks promising so let’s hop back to our console and take a look.

ActiveJob::Base.method(:execute).source.display
#       def execute(job_data) #:nodoc:
#         job = deserialize(job_data)
#         job.perform_now
#       end
# => nil

It does two things: pass job_data into a method called deserialize and then call perform_now on the job that’s returned.

Looking at deserialize, we can see the serialized values being assigned to attributes on the newly created job instance.

ActiveJob::Base.method(:deserialize).source.display
#       def deserialize(job_data)
#         job                      = job_data['job_class'].constantize.new
#         job.job_id               = job_data['job_id']
#         job.queue_name           = job_data['queue_name']
#         job.serialized_arguments = job_data['arguments']
#         job.locale               = job_data['locale'] || I18n.locale
#         job
#       end
# => nil

So, we’ve made some progress but there’s a problem. Deserialization is done by calling ActiveJob::Base.deserialize so in order to override it in our jobs, we’d need to monkey patch ActiveJob::Base. It would be much easier to extend if it worked like serialization and we could simply implement ApplicationJob#deserialize but it looks like we’ll need to take a more aggressive approach. Before we do, now is a good time to pause, take a breathe, and check Rails master. Remember, we’re running Rails 4.2 app and there’s a chance that someone else hit the same issue as us. It may have even been solved in a future version of Rails so let’s take a look at the deserialize method in Rails master.

Using source_location, we can see that it's defined in lib/active_job/core.rb so let’s start by looking there.

ActiveJob::Base.method(:deserialize).source_location
# => ["activejob-4.2.10/lib/active_job/core.rb", 27]

def deserialize(job_data)
  job = job_data["job_class"].constantize.new
  job.deserialize(job_data)
  job
end

In a wonderful bit of luck, it looks like someone solved this for us! Now, ActiveJob::Base.deserialize delegates to the job via the #deserialize instance method. Even though we’re using Rails 4.2, we can backport this behaviour using a monkey patch and start using it immediately. This helps prepare us for the future so when it comes time to upgrade, our code will already work with the newer version of Rails.

Let’s add an initializer that re-opens ActiveJob::Base and redefine the deserialize method. This is the most crude form of monkey patching but it gets the job done. Let's copy/paste the original Rails 4.2 implementation into the method and then, after the original deserialization happens, we’ll add a call to #deserialize.

class ActiveJob::Base
  def self.deserialize(job_data)
    job                      = job_data['job_class'].constantize.new
    job.job_id               = job_data['job_id']
    job.queue_name           = job_data['queue_name']
    job.serialized_arguments = job_data['arguments']
    job.locale               = job_data['locale'] || I18n.locale

    job.deserialize(job_data)

    job
  end
end

Let's try running the job, first turning on inline processing to ensure Sidekiq serializes and deserialized the job completely. Oops! There's a problem. Our jobs don't implement deserialize yet so...

require 'sidekiq/testing'; Sidekiq::Testing.inline!
MyJob.perform_later
# NoMethodError: undefined method `deserialize' for #
# Did you mean?  serialize
#   config/initializers/extend_active_job.rb:9:in `deserialize'

...let's tweak our patch to only call deserialize if the job responds to it.

class ActiveJob::Base
  def self.deserialize(job_data)
    job                      = job_data['job_class'].constantize.new
    job.job_id               = job_data['job_id']
    job.queue_name           = job_data['queue_name']
    job.serialized_arguments = job_data['arguments']
    job.locale               = job_data['locale'] || I18n.locale

    job.deserialize(job_data) if job.respond_to?(:deserialize)

    job
  end
end

require 'sidekiq/testing'; Sidekiq::Testing.inline!
MyJob.perform_later
# TRACE UUID: nil
# => nil

Better! Now we have a working backport of the Rails master behaviour...

...and we can go ahead and implement ApplicationJob#deserialize to take job_data and assign trace_uuid.

class ApplicationJob < ActiveJob::Base attr_accessor :trace_uuid def serialize super.merge("trace_uuid" => trace_uuid || Trace.uuid)
  end

  def deserialize(job_data)
    self.trace_uuid = job_data["trace_uuid"]
  end
end

require 'sidekiq/testing'; Sidekiq::Testing.inline!
Trace.uuid = "abcdef"
MyJob.perform_later
# TRACE UUID: "abcdef"
# => #

This time, it works! Our job serializes the trace UUID and then restores it during deserialization. We’re done!

Or, are we?

Remember, when we copied and pasted code into the patch? Aside from making our patch longer and more complex, it also means we take full responsibility of implementing the original behaviour. If we upgrade Rails and the implementation changes, our code won’t adapt.

What if we swap it out for a call to super instead?

class ActiveJob::Base
  def self.deserialize(job_data)
    job = super
    job.deserialize(job_data) if job.respond_to?(:deserialize)
    job
  end
end

Let’s check to make sure everything works as expected—and it does.

require 'sidekiq/testing'; Sidekiq::Testing.inline!
Trace.uuid = "abcdef"
MyJob.perform_later
# TRACE UUID: abcdef
# => #

This looks a lot better: it's small and delegates to the original implementation.

But before we call it a day, there’s one more important piece: this monkey patch has an expiry date. We only need it so long as we’re using an old version of Rails. To make sure we remember to remove it, let's raise an exception when running a new version of Rails.

First, let’s find out when the new behaviour was added by going back to GitHub and looking at the git blame for the specific commit that added the change. Once on the commit, we can see all of the tags associated with this commit and see it was first added to the Rails 5.0 beta.

And now, if we're running Rails 5.0 or newer, we can raise an exception by using the Gem::Version object to do a version comparison.

if Gem::Version.new(Rails.version) >= Gem::Version.new("5.0")
  raise "Remove this file! You don't need it anymore!"
end

class ActiveJob::Base
  def self.deserialize(job_data)
    job = super
    job.deserialize(job_data) if job.respond_to?(:deserialize)
    job
  end
end

That’s a look at how you can use spelunking along with GitHub to learn more about the gems you use and how to carefully and responsibly patch and extend code.