Background processing with Paperclip

I’m going to assume that you are familiar with Paperclip by now. If you’re not, you should download and implement immediately. The background daemon I’m using is delayed_job but you are of course free to use your weapon of choice. Ryan Bates recently did a nice screencast about delayed_job which I highly recommend (check out this Ascii-cast if you don’t like videos).

Ok, so on to the cool stuff, we start by making a table and all that:

class CreateImages < ActiveRecord::Migration
  def self.up
    create_table(:images) do |t|
    
      # default paperclip fields
      t.string   :data_file_name
      t.string   :data_content_type
      t.integer  :data_file_size
      t.datetime :data_updated_at
      
      # processing
      t.boolean :processing, :default => true
      
      # timestamps
      t.timestamps
      
    end
  end
  
  def self.down
    drop_table :images
  end
end

Pretty straight forward migration. The processing boolean is used to check whether a particular photo has been processed yet. It defaults to true, since we will always intercept Paperclip’s processing when a new Photo is added.

So let’s create an ImageJob, the little delayed_job gnome that will be doing all the dirty work.

class ImageJob < Struct.new(:image_id)
  def perform
    Image.find(self.image_id).perform
  end
end

This is a regular pattern for creating jobs in delayed_job. What it does is basically to (on perform) find the image in question, and call the instance method perform() on it which will do all the processing stuff. I normally add these jobs into lib/ since they’re not models per say.

In our Image model, we add the following:

before_data_post_process do |image|
  false if image.processing? # do not process if just added
end
  
after_create do |image|
  Delayed::Job.enqueue ImageJob.new(image.id) # add to queue
end

This utilizes Paperclip’s before_:attachment_post_process callback to halt the processing and just go through with the create action (save the record in db). After create, we add this Image to our delayed job queue.

Since it might take some time processing our Image, we add this little convenience method which serves two purposes; normally when using Paperclip you have to do model.attachment.url(:style) to get the url of the attachment, this method allows you to to model.url(:style). We are also able to return a default URL while processing the Image in the background.

def url(style = :original)
  if self.data && processing? && style != :original
    return data.send(:interpolate, @@default_url, "#{style}")
  end
  data.url(style)
end

So now we are ready to try out the actual processing, with this insanely advanced method on our Image model:

def perform
  self.processing = false # unlock for processing
  data.reprocess! # do the processing
  save
end

And that’s it. You can now start up your delayed_job queue listener, go into console and try out with a few files.

10.times { Image.create(:data => File.open('/home/lars/test.jpg')) }

Happy hacking :-)