If you need to perform any time consuming work in your Rails actions, you’ll probably want to offload this into a background job. There are many different frameworks to help with this, and the one we use is Workling. The nice thing about Workling is that it provides an abstraction layer that allows you to decouple your actual background job implementation from the background execution strategy. For example, in our development environment we are using the Spawn runner (which simply forks the Rails process for each background job), but we need a proper, queue based runner in production. Up until recently we were using the Starling runner, which worked pretty well for a small set of machines.
However, after migrating our infrastructure to Amazon EC2 and rapidly scaling up the number of app servers, we figured it would be great to take advantage of Amazon SQS (Simple Queue Service), rather than maintaining our own queue servers. Fortunately, Workling’s plugin architecture makes it very easy to implement your own clients, so writing an SQS Workling client turned out to be fairly straightforward.
If you are interested in using this in your own project, simply use my Workling fork on Github. I haven’t decided yet whether to extract this into a separate plugin that you could install alongside Workling, so let me know if you have a strong preference. I’ll also get in touch with the Workling developers to see if they might be interested in pulling this feature into the main code base. But for now, you can simply install it by following the regular Workling plugin installation instructions, except using my Workling fork:
The README includes detailed instructions on configuring the client, but it’s actually very easy:
Install the RightAws gem:
Configure Workling to use the SqsClient. Add this to your environment:
Add your AWS key id and secret key to
You can optionally override the following settings, although the defaults will likely be sufficient:
Now start the Workling Client:
There are still some caveats, such as the fact that messages are currently deleted from the queue at the beginning of processing rather than at the end (unfortunately Workling currently doesn’t provide the necessary hooks). This is good enough for us (we’re not relying on our background jobs for anything highly critical), but you probably don’t want to build your financial transaction processing on top of this… If there’s demand for this, I may try to extend Workling at some point to fix this issue.
Please leave a comment if you find this useful or have any other feedback. Also let me know if you encounter any bugs, or better yet, update my test case to reproduce the issue or send me a Github pull request with your fix. :)