Add capability to discard duplicate jobs with concurrency configuration

[joelzwarrington]

Closes #176

Add ability to discard duplicate jobs instead of becoming blocked. When configuring concurrency_limits with at_limit: discard jobs scheduled above the concurrency limit are discarded and not executed.

Implementation Details

• adds class_attribute concurrency_at_limit and at_limit option to concurrency_limits
• adds methods to Semaphore to detect when a job would be above the concurrency limit
• adds logic to discard a job on dispatch when at concurrency limit

[joelzwarrington]

Thanks for the review @rosa ❤️, I'll review your comments more thoroughly and make some changes. 😊

[joelzwarrington]

Hi @rosa, I've made a few changes and believe to have encountered weird interactions with bulk enqueuing which might be a bug. Let me know if you'd like those changes split out into another PR.

[rajeevriitm]

I am really in need of this feature. As its not yet available , could you help me in implementing this ? I have a ContinuousSearchJob thats run with recurring.yml . Sometimes the job takes longer and I dont want another job to be added to the queue unless the existing one is finished. Could you help me with the best way I can implement this currently?

[joelzwarrington]

@rajeevriitm if you have a look at the read me you should find concurrency controls are available for you to use.

The only difference being that jobs will always be blocked, so you may get several jobs that are awaiting execution.

[joelzwarrington]

@rosa any updates on your review here?

[rynonl]

@rosa Just pinging this. Would love to see this functionality get merged!

[danlo]

@rosa +1

[rosa]

Hey, sorry everyone! I've been crazy busy at work in a bunch of unrelated things, but I'm planning for at least a day of Solid Queue work to catch up with all issues and PRs very soon. Thanks y'all for your patience and sorry for the delay!

[joelzwarrington]

Thanks for the review. Gonna have a look and implement any feedback this week

[joelzwarrington]

It'd be a good idea to update https://github.com/rails/rails/blob/c049370882474acffbc8fd5be458e6ecd1d7d1cb/guides/source/active_job_basics.md?plain=1#L346 when merging this as I've changed SolidQueue::Job::EnqueueError to inherit from ActiveJob::EnqueueError.

[joelzwarrington]

@rosa, pull request is ready for you review again! Let me know what you think.

If you're on-board with these changes, I'll also add a PR to update the RoR edge guide.