2026-09-12 · 11 min
Background Job Queue with BullMQ and Redis: A Production Setup for Node.js
A worker that looks perfectly healthy in pm2 list but has quietly stopped pulling jobs is the most expensive silence in a Node.js backend. No error, no crash, no restart — just a queue that grows while your dashboard stays green. Nine times out of ten the cause is one missing line in the Redis connection config, and it is the first thing I check when someone tells me their background job queue with BullMQ “randomly stops.” This is the production setup I actually use, including the parts the getting-started guides skip.
BullMQ is the Redis-backed job queue for Node.js that replaced the older Bull library. It gives you retries with backoff, concurrency control, rate limiting, delayed and scheduled jobs, and job dependencies — all coordinated through Redis so multiple worker processes can share the load safely. What it does not give you, and what this post is really about, is a safe default configuration. You have to build that.
The connection config that bites everyone
Start here, because getting it wrong wastes an afternoon. BullMQ workers wait for jobs using Redis blocking commands rather than polling in a loop. ioredis, by default, gives up on a request after 20 retries and throws. A blocking command that legitimately waits ten seconds for the next job trips that limit, the connection dies, and your worker stops consuming — without logging anything useful.
// redis.ts — one connection factory, reused everywhere
import { Redis } from "ioredis";
export function makeConnection() {
return new Redis(process.env.REDIS_URL!, {
// Non-negotiable for BullMQ. Blocking commands must never be retried away.
maxRetriesPerRequest: null,
// BullMQ manages its own readiness; skip ioredis's INFO check.
enableReadyCheck: false,
});
}
BullMQ 6 needs Redis 5.0 or newer, but run at least 6.2 — earlier versions miss commands the scheduler relies on, and in production I default to Redis 7.x with AOF persistence turned on. If Redis loses its dataset on restart, you lose queued jobs. appendonly yes in redis.conf is not optional for a queue that matters.
Queue and Worker, separated
A common mistake is treating BullMQ like a function call. It is not. The producer (your API) and the consumer (your worker) are different processes, and they should be different files, ideally different deployments.
// queue.ts — producer side, imported by your API
import { Queue } from "bullmq";
import { makeConnection } from "./redis";
export const emailQueue = new Queue("email", {
connection: makeConnection(),
defaultJobOptions: {
attempts: 5,
backoff: { type: "exponential", delay: 2000 },
removeOnComplete: { age: 3600, count: 1000 }, // keep 1h or last 1000
removeOnFail: { age: 24 * 3600 }, // keep failures a day
},
});
That removeOnComplete matters more than it looks. BullMQ keeps completed and failed jobs in Redis so you can inspect them. Leave it unbounded and your Redis memory climbs until an OOM kill takes the whole thing down at the worst possible moment. I have seen a completed-jobs set eat 4GB on a queue nobody thought was busy. Bound it by age and count from day one.
The worker lives on its own:
// worker.ts — consumer side, runs as a separate process
import { Worker } from "bullmq";
import { makeConnection } from "./redis";
const worker = new Worker(
"email",
async (job) => {
// job.data is whatever you enqueued. Validate it — never trust it blindly.
const { to, template, payload } = job.data;
await sendEmail(to, template, payload);
},
{
connection: makeConnection(),
concurrency: 20, // 20 jobs in flight per worker process
limiter: { max: 50, duration: 1000 }, // queue-wide cap: 50 jobs/sec
},
);
Two knobs here do very different things and people conflate them. concurrency is per-process parallelism — how many jobs this one worker runs at once. limiter is a global throttle enforced across every worker on the queue through Redis. Set max: 50 and run four worker pods, and you still get 50 jobs per second total, not 200. That distinction is exactly why you throttle at the queue, not in the handler: the limit usually belongs to a downstream dependency, like an SMTP provider or a payment gateway, and it should hold no matter how you scale.
Idempotency, because at-least-once is a promise you keep, not one you get
BullMQ delivers each job at least once. If a worker does the work, then dies before acknowledging, the job comes back. So any job with a real side effect — charging a card, sending money, posting to a ledger — must be safe to run twice. This is not paranoia. A fintech client of mine runs disbursements out of a small office near Sinpasa Summarecon Serpong, and early on a redeploy mid-batch double-sent three transfers because the handler assumed exactly-once. It does not exist. Design for repeats.
Two layers. First, collapse duplicate enqueues with a deterministic job id:
// Same logical work → same job. BullMQ dedupes on jobId.
await payoutQueue.add(
"disburse",
{ payoutId },
{ jobId: `payout:${payoutId}` },
);
Second, make the handler itself idempotent — belt and suspenders, because the job id only guards enqueue time, not a retry of an already-partially-run job:
async (job) => {
const { payoutId } = job.data;
// Claim the work atomically. If another attempt already did it, bail.
const claimed = await db.query(
`INSERT INTO processed_payouts (payout_id) VALUES ($1)
ON CONFLICT (payout_id) DO NOTHING RETURNING payout_id`,
[payoutId],
);
if (claimed.rowCount === 0) return; // already done, harmless no-op
await gateway.disburse(payoutId); // pass an idempotency key too if it supports one
}
The ON CONFLICT DO NOTHING is doing the heavy lifting. The database, not the queue, is your source of truth for “did this happen.”
Failures: retry the transient, quarantine the poison
Not every failure deserves a retry. A 503 from a provider is worth retrying with backoff. A malformed payload will fail identically five times and just burn attempts. Distinguish them by throwing differently.
import { UnrecoverableError } from "bullmq";
async (job) => {
if (!isValid(job.data)) {
// Skip remaining attempts. This will never succeed by retrying.
throw new UnrecoverableError("invalid payload: " + JSON.stringify(job.data));
}
await doWork(job.data); // transient errors bubble up and retry per backoff
}
Jobs that exhaust all attempts land in the failed set. That set is a dead-letter queue you should actually look at. Wire a QueueEvents listener to alert when it grows, and give yourself a way to re-drive failures after you ship a fix, rather than losing them.
import { QueueEvents } from "bullmq";
const events = new QueueEvents("payout", { connection: makeConnection() });
events.on("failed", ({ jobId, failedReason }) => {
logger.error({ jobId, failedReason }, "payout job failed permanently");
// push a metric; alert if the failed set grows beyond a threshold
});
Graceful shutdown, or you will lose in-flight work
This is the part almost every tutorial omits, and it is the difference between a clean deploy and half-finished jobs. When Kubernetes or systemd sends SIGTERM, the default behavior kills the process while jobs are mid-flight. Those jobs get marked stalled and retried — which, if your idempotency is shaky, means double execution.
async function shutdown() {
// Stop accepting new jobs, let running ones finish (with a grace window).
await worker.close();
await events.close();
process.exit(0);
}
process.on("SIGTERM", shutdown);
process.on("SIGINT", shutdown);
worker.close() stops fetching new jobs and waits for active ones to complete. Pair it with a terminationGracePeriodSeconds in your pod spec that is comfortably longer than your slowest job. If your longest job runs 30 seconds, a 10-second grace period guarantees truncation.
The scheduler question, briefly
For periodic work — nightly aggregation, hourly syncs — BullMQ’s Job Schedulers put the schedule in Redis so exactly one instance produces the job, even across a fleet of workers. That kills the classic bug where every replica fired the same cron and you sent the daily digest ten times.
// Runs once across the whole cluster, at 01:00 Asia/Jakarta.
await reportQueue.upsertJobScheduler(
"daily-txn-rollup",
{ pattern: "0 1 * * *", tz: "Asia/Jakarta" },
{ name: "rollup", data: {} },
);
I still keep genuinely critical timing — the kind that must fire even if the Node.js fleet is down — on system cron or a dedicated scheduler, because I do not want it coupled to Redis availability. But for ordinary app-level periodic jobs, this is less to operate than a separate cron box.
What I would actually ship
Separate the producer from the worker as distinct deployments so you can scale and restart them independently. Bound removeOnComplete and removeOnFail before anything else, because unbounded job retention is a slow-motion OOM. Set maxRetriesPerRequest: null and stop chasing the phantom “worker stopped” bug. Assume every job runs at least twice and make the handler safe under that assumption. Handle SIGTERM so deploys drain instead of truncate. And put a metric on the failed set — a dead-letter queue you never read is just a place where problems go to hide.
BullMQ is not complicated. The defaults just are not production defaults, and the gap between “it works on my machine” and “it holds at 2am under load” is entirely in the six things above. Get those right and it will run for months without you thinking about it — which, for a queue, is the whole point.
Ditulis oleh Reza Pradipta