Redis Queues Explained: Lists vs Streams vs Pub/Sub

Redis Queues Explained: Lists vs Streams vs Pub/Sub

Redis gives you several ways to move work or messages between processes, but they do not provide the same delivery semantics.

The three options developers most often confuse are:

• Lists — simple producer/consumer queues;
• Streams — persisted ordered messages with consumer tracking and acknowledgments;
• Pub/Sub — live broadcast to subscribers that are connected right now.

Choosing the right primitive matters more than the syntax. A queue that is fine for disposable work can be dangerous for payments, emails, imports, or other tasks that must survive worker failures.

The quick decision table

A practical rule:

> Use a List for simple queues, a Stream when delivery state and recovery matter, and Pub/Sub when losing a message for an offline subscriber is acceptable.

Option 1: Redis Lists for simple work queues

A Redis List can behave like a queue by pushing on one side and popping from the other.

Producer:

LPUSH jobs:thumbnail '{"image_id":8172}'

Worker:

BRPOP jobs:thumbnail 0

BRPOP blocks instead of polling when the list is empty, which makes the pattern simple and efficient.

The Redis Lists documentation describes blocking list operations specifically for producer/consumer patterns.

The danger of a simple pop

A plain blocking pop removes the item before your application has finished processing it.

Imagine:

1. BRPOP returns job A
2. Redis removes job A from the list
3. worker crashes
4. job A was never completed

Redis' BLPOP documentation explicitly calls out this failure mode.

For work that can safely disappear—perhaps best-effort cleanup—that may be acceptable. For important jobs, it is not.

A safer List pattern: pending -> processing

Redis supports atomic movement between lists with commands such as LMOVE and the blocking BLMOVE.

Instead of deleting a job when a worker claims it, move it atomically:

jobs:pending -> jobs:processing

Conceptually:

BLMOVE jobs:pending jobs:processing RIGHT LEFT 0

After successful processing, remove the entry from the processing list.

The official LMOVE documentation describes this as a reliable queue pattern: a monitor can detect items left too long in the processing list and return them to the pending queue.

That is useful, but notice what you now have to build yourself:

• processing ownership;
• visibility timeout / stale-job detection;
• retries;
• attempt counts;
• delayed retries;
• dead-letter handling;
• monitoring.

For a small queue this may be perfectly reasonable. As those requirements grow, Streams become more attractive.

Option 2: Redis Streams for tracked delivery

A Redis Stream is an append-oriented data structure with ordered entry IDs and richer consumption semantics.

Producer:

XADD jobs:email * job_id 42 template welcome

A worker pool can use a consumer group:

XREADGROUP GROUP email-workers worker-1 \
  BLOCK 5000 COUNT 10 \
  STREAMS jobs:email >

When an entry is delivered through a consumer group, Redis tracks it as pending until the consumer acknowledges it:

XACK jobs:email email-workers 1726123456789-0

This gives Streams important queue-like capabilities:

• consumers in one group divide the work;
• delivered-but-unfinished entries are tracked;
• crashed-worker messages can be claimed by another consumer;
• multiple consumer groups can independently process the same stream;
• entries can be replayed while retained;
• the stream can be trimmed to bound memory.

See the official Redis Streams documentation for the full model.

Streams are not “exactly once” by default

If a worker completes a side effect and crashes before XACK, the message may later be processed again.

That means worker handlers still need idempotency for important side effects.

Streams give you better delivery tracking; they do not remove the distributed-systems problem between “the business action completed” and “the broker learned that it completed.”

Option 3: Redis Pub/Sub is broadcast, not a durable queue

Redis Pub/Sub is excellent when you want to push a live message to every currently connected subscriber.

Publisher:

PUBLISH notifications:user:42 '{"type":"typing"}'

Subscriber:

SUBSCRIBE notifications:user:42

But Redis' Pub/Sub documentation is explicit about the delivery model: Pub/Sub is at-most-once. If a subscriber is disconnected or cannot handle the message, Redis does not retain it for later replay.

That makes Pub/Sub a good fit for things such as:

• typing indicators;
• transient presence signals;
• cache-invalidation hints where another recovery path exists;
• live process-to-process signals;
• UI updates that can be reconstructed from authoritative state.

It is usually a poor fit for:

• orders that must be processed;
• emails that must eventually be sent;
• payment events that cannot silently disappear;
• jobs that need retry or acknowledgment;
• consumers that may be offline and catch up later.

Calling Pub/Sub a “queue” hides the most important difference: there is no queue of missed messages waiting for the subscriber.

Broadcast vs work sharing

This distinction is worth visualizing:

A worker queue normally wants one processor per job.

A broadcast system wants every interested live subscriber to see the event.

Redis Streams can also support multiple independent consumer groups, which gives a third pattern: every group can receive the event, while workers inside a group divide that group's work.

What about delayed jobs and priorities?

Neither a basic List nor a basic Stream is automatically a full scheduler.

A common Redis primitive for scheduled work is a sorted set, where the score represents time:

ZADD jobs:scheduled 1789194600 job-42

A scheduler can fetch entries whose score is due and move them into the runnable queue or stream.

Sorted sets can also represent priority, depending on your design.

The hard part is making the transition crash-safe and duplicate-safe. If delayed jobs, retries, repeat schedules, priorities, rate limits, and job state are central to your product, using a mature Redis-backed queue library can be safer than implementing a job framework from raw commands.

Queue durability depends on Redis durability too

Even a perfectly designed consumer algorithm cannot recover messages that disappear with the Redis deployment itself.

If queued work is important, decide deliberately:

• whether Redis persistence is enabled;
• what persistence mode matches the workload;
• whether replicas and automatic failover are required;
• what amount of acknowledged data loss is acceptable;
• whether queue data is included in backup/recovery planning.

A queue of disposable cache-refresh tasks has very different durability requirements from a queue containing customer-facing business operations.

Backpressure: the reason queues exist

Queues are not only about asynchronous code. They also buffer work when producers and consumers operate at different speeds.

Suppose an API accepts 500 export requests quickly but the export service can safely run only 20 at a time.

A queue lets the API accept work without launching 500 expensive jobs simultaneously:

But the queue does not eliminate overload; it moves overload into backlog.

Monitor:

• queue/stream depth;
• oldest waiting item;
• enqueue-to-start latency;
• processing latency;
• retry/failure rate;
• worker utilization;
• downstream saturation.

If backlog grows continuously, more queue capacity only hides the real throughput mismatch.

Choosing between Lists, Streams, and Pub/Sub

Choose a Redis List when

• the workflow is simple;
• one worker should handle each item;
• minimal overhead matters;
• you can tolerate simple delivery semantics or are comfortable implementing a processing-list recovery pattern;
• you do not need replay or multiple independent consumer groups.

Choose Redis Streams when

• messages should remain available while retained;
• acknowledgments matter;
• workers can fail after receiving work;
• you need pending-message inspection/recovery;
• multiple independent consumer groups need the same event stream;
• replay and consumer lag are valuable.

Choose Redis Pub/Sub when

• the message is useful only to subscribers connected now;
• fan-out is the goal;
• missed messages can safely be lost or reconstructed;
• you do not need acknowledgment, history, or replay.

Common mistakes

Using Pub/Sub for must-not-lose jobs

Offline subscribers miss messages. Use a durable model instead.

Using BLPOP for important work without recovery

The item is removed when delivered. A worker crash can lose it.

Choosing Streams but ignoring duplicate processing

Acknowledgment-based recovery implies that redelivery can happen. Make business effects idempotent.

Building a full job platform from primitives unnecessarily

Retries, schedules, priorities, concurrency, metrics, cleanup, and failure handling add up quickly. Raw Redis commands are building blocks, not automatically a complete queue product.

Scaling workers without protecting dependencies

More consumers can overload the database or external API behind the queue. Concurrency limits are part of queue design.

The main takeaway

Redis has several messaging primitives because they solve different problems.

• Lists are the simplest queue.
• Streams are the stronger choice when you need tracked processing, acknowledgments, recovery, replay, or multiple consumer groups.
• Pub/Sub is for ephemeral live fan-out, not durable background work.

Choose based on the failure behavior you need—not on which command takes the fewest lines of code.