Explore and Extend – Part 2
In the previous episode, we learned how to extend
ActiveJob serialization to add a trace UUID to every serialized job using method introspection techniques to peek under the surface and work with the code beneath. Today we’ll be looking at how to work with code that wasn’t designed to be extended by adding the trace UUID to deserialization.
Video transcript & code
Part 2: Responsible Monkey-Patching
This time, we don’t have a natural starting place because we don’t ever pull things from the queue—that’s Sidekiq’s job! Instead of using the console, let’s go read some code on GitHub and see if we can find some clues.
Some of this code looks familiar—we already saw the
#enqueue method while we were spelunking. Aside from the enqueue methods, the only other bit is a method called
perform, which makes a call to
ActiveJob::Base.execute. That looks promising so let’s hop back to our console and take a look.
ActiveJob::Base.method(:execute).source.display # def execute(job_data) #:nodoc: # job = deserialize(job_data) # job.perform_now # end # => nil
It does two things: pass
job_data into a method called
deserialize and then call
perform_now on the job that’s returned.
deserialize, we can see the serialized values being assigned to attributes on the newly created job instance.
ActiveJob::Base.method(:deserialize).source.display # def deserialize(job_data) # job = job_data['job_class'].constantize.new # job.job_id = job_data['job_id'] # job.queue_name = job_data['queue_name'] # job.serialized_arguments = job_data['arguments'] # job.locale = job_data['locale'] || I18n.locale # job # end # => nil
So, we’ve made some progress but there’s a problem. Deserialization is done by calling
ActiveJob::Base.deserialize so in order to override it in our jobs, we’d need to monkey patch
ActiveJob::Base. It would be much easier to extend if it worked like serialization and we could simply implement
ApplicationJob#deserialize but it looks like we’ll need to take a more aggressive approach.
Before we do, now is a good time to pause, take a breathe, and check Rails master. Remember, we’re running Rails 4.2 app and there’s a chance that someone else hit the same issue as us. It may have even been solved in a future version of Rails so let’s take a look at the
deserialize method in Rails master.
source_location, we can see that it's defined in
lib/active_job/core.rb so let’s start by looking there.
ActiveJob::Base.method(:deserialize).source_location # => ["activejob-4.2.10/lib/active_job/core.rb", 27]
def deserialize(job_data) job = job_data["job_class"].constantize.new job.deserialize(job_data) job end
In a wonderful bit of luck, it looks like someone solved this for us! Now,
ActiveJob::Base.deserialize delegates to the job via the
#deserialize instance method. Even though we’re using Rails 4.2, we can backport this behaviour using a monkey patch and start using it immediately. This helps prepare us for the future so when it comes time to upgrade, our code will already work with the newer version of Rails.
Let’s add an initializer that re-opens
ActiveJob::Base and redefine the
deserialize method. This is the most crude form of monkey patching but it gets the job done. Let's copy/paste the original Rails 4.2 implementation into the method and then, after the original deserialization happens, we’ll add a call to
class ActiveJob::Base def self.deserialize(job_data) job = job_data['job_class'].constantize.new job.job_id = job_data['job_id'] job.queue_name = job_data['queue_name'] job.serialized_arguments = job_data['arguments'] job.locale = job_data['locale'] || I18n.locale job.deserialize(job_data) job end end
Let's try running the job, first turning on inline processing to ensure Sidekiq serializes and deserialized the job completely. Oops! There's a problem. Our jobs don't implement
deserialize yet so...
require 'sidekiq/testing'; Sidekiq::Testing.inline! MyJob.perform_later # NoMethodError: undefined method `deserialize' for # # Did you mean? serialize # config/initializers/extend_active_job.rb:9:in `deserialize'
...let's tweak our patch to only call
deserialize if the job responds to it.
class ActiveJob::Base def self.deserialize(job_data) job = job_data['job_class'].constantize.new job.job_id = job_data['job_id'] job.queue_name = job_data['queue_name'] job.serialized_arguments = job_data['arguments'] job.locale = job_data['locale'] || I18n.locale job.deserialize(job_data) if job.respond_to?(:deserialize) job end end
require 'sidekiq/testing'; Sidekiq::Testing.inline! MyJob.perform_later # TRACE UUID: nil # => nil
Better! Now we have a working backport of the Rails master behaviour...
...and we can go ahead and implement
ApplicationJob#deserialize to take
job_data and assign
class ApplicationJob < ActiveJob::Base attr_accessor :trace_uuid def serialize super.merge("trace_uuid" => trace_uuid || Trace.uuid) end def deserialize(job_data) self.trace_uuid = job_data["trace_uuid"] end end
require 'sidekiq/testing'; Sidekiq::Testing.inline! Trace.uuid = "abcdef" MyJob.perform_later # TRACE UUID: "abcdef" # => #
This time, it works! Our job serializes the trace UUID and then restores it during deserialization. We’re done!
Or, are we?
Remember, when we copied and pasted code into the patch? Aside from making our patch longer and more complex, it also means we take full responsibility of implementing the original behaviour. If we upgrade Rails and the implementation changes, our code won’t adapt.
What if we swap it out for a call to
class ActiveJob::Base def self.deserialize(job_data) job = super job.deserialize(job_data) if job.respond_to?(:deserialize) job end end
Let’s check to make sure everything works as expected—and it does.
require 'sidekiq/testing'; Sidekiq::Testing.inline! Trace.uuid = "abcdef" MyJob.perform_later # TRACE UUID: abcdef # => #
This looks a lot better: it's small and delegates to the original implementation.
But before we call it a day, there’s one more important piece: this monkey patch has an expiry date. We only need it so long as we’re using an old version of Rails. To make sure we remember to remove it, let's raise an exception when running a new version of Rails.
First, let’s find out when the new behaviour was added by going back to GitHub and looking at the git blame for the specific commit that added the change. Once on the commit, we can see all of the tags associated with this commit and see it was first added to the Rails 5.0 beta.
And now, if we're running Rails 5.0 or newer, we can raise an exception by using the
Gem::Version object to do a version comparison.
if Gem::Version.new(Rails.version) >= Gem::Version.new("5.0") raise "Remove this file! You don't need it anymore!" end class ActiveJob::Base def self.deserialize(job_data) job = super job.deserialize(job_data) if job.respond_to?(:deserialize) job end end