Re: [NYC-rb] Ruby job queues

From: user 2.
Sent on: Thursday, May 9, 2013 12:24 PM
I've also been using Sidekiq with much success on Apartable. Highly recommended. Once you get the hang of it, it stays up forever and doesn't consume a whole lot of resources like Resque does. 

-Avishai


On Thu, May 9, 2013 at 12:14 PM, Dmitriy Likhten <[address removed]> wrote:
We use resqueue + resqueue scheduler and we scale using Hirefire.io -- if too many jobs in the queue we auto-scale more workers.

The only problems I've seen is every once in a while jobs would die on heroku (killed), and there's not a great mechanism I know of (limited knowledge) which auto-retries jobs based on certain kinds of errors. This may be true of any job system you try on heroku though.

Our background jobs per second is pretty small, probably 1 per second or so. Every once in a while we'll schedule a few hundred/thousand jobs at once so that one task is broken up. But that's about it, no real problems, using a free redis service for all our queueing needs.

-- 
Dmitriy Likhten

On Thursday, May 9, 2013 at 12:06 PM, PatrickTulskie wrote:

Hey David,

At BeenVerified, we use Resque for everything in the background.  We've used starling/workling, Delayed Job, and now we're on Resque.  We process anywhere from 8-15 background jobs per second 24/7 and so far it hasn't let us down.  The built in web interface is great and it really takes everything we've thrown at it.

We do use resque-pool and we're still on REE so we can have an enormous amount of workers on tap without as much memory bloat.

I haven't tried Backburner but Resque hasn't let us down like starling and DJ did and I think we probably drive it harder than most people.


On May 9, 2013, at 11:10 AM, David Michael <[address removed]> wrote:

Hello list

It has been a while since I have setup a new job queue for a Ruby project and I was wondering if some of yous had a favorite job queue youd like to share.

Is Resque old hat? Is Backburner the new hotness?

Thanks!
David




--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by David Michael ([address removed]) from NYC.rb.
To learn more about David Michael, visit his/her member profile
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]





--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by PatrickTulskie ([address removed]) from NYC.rb.
To learn more about PatrickTulskie, visit his/her member profile
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]





--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Dmitriy Likhten ([address removed]) from NYC.rb.
To learn more about Dmitriy Likhten, visit his/her member profile



--
Apartable: the easiest way to find an apartment.

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy