Why Message Queues

Decouple services.

Async processing.

Handle bursts.

Reliable delivery.

Retry on failure.

RabbitMQ

Traditional message broker.

AMQP protocol.

Complex routing rules.

Task queues.

Widely deployed.

Kafka

Distributed streaming platform.

High throughput.

Long-term storage of events.

Event sourcing.

Analytics pipelines.

AWS SQS

Managed queue.

Simple.

Pay per use.

Standard vs FIFO options.

Zero ops.

Redis Pub/Sub

Simple, fast.

No persistence.

Fire and forget.

Not for critical messages.

Redis Streams

Redis’s answer to Kafka.

Persistent.

Consumer groups.

Cheaper than Kafka for smaller workloads.

BullMQ / Sidekiq

Redis-backed job queues.

Simple integration with Node/Ruby.

Good UI.

Perfect for job processing.

Google Pub/Sub

Google’s managed messaging.

Auto-scaling.

Global.

Good for GCP-heavy setups.

Use Cases

Job processing: BullMQ, Sidekiq, SQS.

Event streaming: Kafka.

Microservices: RabbitMQ, Kafka.

Notifications: SQS, Pub/Sub.

Delivery Guarantees

At most once: might be lost.

At least once: might be duplicate.

Exactly once: hardest, expensive.

Choose based on requirement.

Ordering

Kafka: ordered per partition.

RabbitMQ: ordered per queue.

SQS FIFO: ordered.

SQS Standard: not guaranteed.

Throughput

Kafka: 100K+ msg/sec.

RabbitMQ: 20-50K msg/sec.

SQS: 3K/sec per queue (unlimited queues).

Redis Streams: 100K+ msg/sec.

Persistence

Kafka: long-term retention.

RabbitMQ: until consumed.

SQS: 14 days max.

Choose based on needs.

Cost

RabbitMQ: server cost.

Kafka: server cost + storage.

SQS: per request.

Managed vs self-hosted trade-off.

Operational Complexity

SQS: zero.

Redis: simple.

RabbitMQ: moderate.

Kafka: high.

Match complexity to needs.

Dead Letter Queues

Failed messages go here.

For investigation.

All modern queues support.

Alerting on DLQ growth.

Idempotency

Handle duplicate messages gracefully.

Message ID tracking.

Idempotent operations.

Critical for at-least-once.

Common Mistakes

Using Kafka when RabbitMQ or SQS enough.

Not handling failures.

Ignoring DLQs.

Small messages when should batch.

Wrong delivery guarantee for use case.

Our Recommendations

Simple job queues: BullMQ with Redis.

AWS-native: SQS.

High-throughput events: Kafka.

Complex routing: RabbitMQ.

Start simple, evolve as needed.

Based on Real Projects

This guide is based on our work with:

• Platforma365
• Nasca

Further Reading

If this guide helped you, you might also want to read our comprehensive guide on Custom SaaS Development.