Amazon SQS in Production: Standard vs FIFO, Visibility Timeouts, DLQs, and Worker Scaling

Amazon SQS in Production: Standard vs FIFO, Visibility Timeouts, DLQs, and Worker Scaling

Amazon SQS is a managed message queue for separating producers from background consumers without operating your own broker cluster.

Standard vs FIFO

Standard queues provide very high throughput with at-least-once delivery and best-effort ordering. Consumers must handle duplicates safely.

FIFO queues provide ordered processing within message groups and deduplication features. Use multiple message groups when you need parallelism while preserving order per entity/workflow.

Choose FIFO because the business requires ordering/deduplication semantics—not simply because it sounds safer.

Understand Visibility Timeout

When a consumer receives a message, SQS hides it for the visibility timeout. The worker must finish and delete the message before that period ends.

If processing takes longer, extend visibility deliberately; if a worker crashes, the message eventually becomes visible for another attempt.

Set the timeout from real processing duration rather than one arbitrary global value.

Consumers Must Be Idempotent

Standard SQS can deliver a message more than once, and retries happen in failure scenarios. Use a stable business/message ID, unique constraints, or processed-operation state so duplicate delivery does not repeat side effects.

Use Long Polling

Long polling waits briefly for messages instead of repeatedly returning empty responses. It reduces empty receives and unnecessary API calls for normal worker workloads.

Dead-Letter Queues

Configure a redrive policy so messages that repeatedly fail move to a DLQ instead of retrying forever.

DLQs need their own monitoring and retention. AWS recommends setting DLQ retention longer than the source queue in common Standard-queue designs.

Batch Where It Helps

SQS supports batch send/receive/delete operations. Batching reduces API overhead, but worker failure handling should still track which individual messages succeeded or failed.

Scale Workers From Backlog

Useful metrics include:

ApproximateNumberOfMessagesVisible
ApproximateAgeOfOldestMessage
messages in flight
processing success/failure rate

Oldest-message age often maps more directly to user/job delay than raw queue depth.

Do not scale workers beyond the database or third-party API capacity they consume.

Final Takeaway

SQS is strongest when workers are idempotent, visibility timeouts match processing time, retries are bounded by a DLQ, and scaling follows backlog age rather than guesswork. Use Standard for high-throughput at-least-once work and FIFO when ordering within message groups is a real requirement.

References

• Amazon SQS Queue Types
• Amazon SQS Dead-Letter Queues
• Amazon SQS Management Guide