Skip to content

Structuring Queues and Jobs

This guide covers how to structure your queues and jobs for clarity, maintainability, and operational efficiency.

Queue naming conventions

Use descriptive, lowercase, hyphenated names that indicate the purpose:

GoodBadWhy
webhook-deliveryqueue1Descriptive
email-notificationsemailsSpecific purpose
stripe-webhookswebhooksIdentifies the target service
order-processing-prodordersIncludes environment context

One queue per concern

Group jobs by their target endpoint or purpose, not by source. This lets you tune retry delays, timeouts, and parallelism per destination:

webhook-delivery → retries: [30, 300, 3600], parallelism: 10
email-notifications → retries: [60, 600], parallelism: 5
stripe-webhooks → retries: [60, 1800, 10800], parallelism: 3
analytics-events → retries: disabled, parallelism: unlimited

When to share a queue

Share a queue when jobs have the same destination, timeout, and retry requirements. For example, all Stripe webhooks can share a queue because they go to the same endpoint with the same reliability expectations.

When to separate queues

Create separate queues when:

  • Jobs go to different endpoints with different SLAs
  • One endpoint needs a longer timeout than another
  • One endpoint is rate-limited and needs parallelism limits
  • You want separate DLQ and alerting behavior

Job granularity

Keep jobs atomic — one job per action:

GoodBad
One job per webhook to deliverOne job that loops through 100 webhooks
One job per email to sendOne job that sends a batch of emails
One job per API callOne job that chains 5 API calls

Atomic jobs are easier to retry individually, inspect in the DLQ, and reason about failure causes.

Environment separation

Use separate queues or separate teams for staging and production:

Option A: Separate queues — Prefix queue names with the environment:

staging-webhook-delivery
production-webhook-delivery

Option B: Separate teams — Create a “Staging” team and a “Production” team. Each team has its own queues, jobs, usage limits, and API tokens.

Option B is recommended because it provides complete isolation, including separate alerting and billing.

Next steps