Recurring Tasks With Sidetiq

7 July 2014

Update: Sidetiq is no longer maintained. I have used sidekiq-scheduler in its place.

Software often relies on recurring scheduled tasks in order to function. From status checks, to billing runs, the UNIX cron daemon has regularly been trusted with this responsibility for decades now. Cron has performed the job admirably and we’ve managed to build some really interesting tools to manage our crontabs. Whenever is a really good example of building a Ruby DSL over cron.

There’s an alternative approach however. If your application has a background worker - which in my experience is pretty much all Ruby web apps - you’ve got the opportunity to use your worker to run scheduled tasks. It’s a really good fit to the problem since it divorces your application’s execution from the server it’s running on. It becomes scalable across multiple servers by pulling scheduled jobs off a queue rather than having a single machine dedicated to running the scheduled tasks.

The Sidekiq gem makes it easy to run a background job queue in Ruby. It’s as easy as adding the gem to your Gemfile and running the sidekiq executable.

Sidekiq has a useful trick up its sleeve for running scheduled tasks. Given that you have a worker called BillingWorker you can queue it up to run at a given time using Sidekiq’s perform_at method.

BillingWorker.perform_at(1.day.from_now)

However this will only execute the job once.

Sidetiq is an addon to Sidekiq that allows for recurring jobs to be declared. I first came across it when reading the source code for Discourse, so it has earned its trust in the Ruby community.

An example BillingWorker that runs daily in Sidetiq can be declared as follows

class BillingWorker
  include Sidekiq::Worker
  include Sidetiq::Schedulable

  recurrence { daily }

  def perform
  	# implementation details...
  end
end

Sidetiq will manage the schedule to run this task daily on one of your Sidekiq workers.

There are a couple of gotchas with Sidetiq at the moment. Its dependency on ice_cube is known to cause a bit of a slow down in generating the recurrence schedule for a worker. Be sure to read up the wiki article on Known Issues to avoid these pitfalls.

Sidetiq also hooks into Sidekiq’s web dashboard. Installing it is as easy as adding require 'sidetiq/web' to the file that you’ve used to mount the Sidekiq::Web Rack app. It’s pretty useful to have this hooked in since it also lets you manually trigger the jobs if needed.

Sidetiq’s Web UI

To make this work in Rails’ development mode you’ll need to set config.eager_load = true in your development.rb. It’s usually not an issue though.

In my experience so far Sidetiq has been a far more elegant solution to the problem of recurring scheduled tasks than managing crontabs on cloud VMs, or using Heroku’s Scheduler addon.