Lambda behind a queue: event source mappings, partial batches and destinations
The most common serverless shape in real systems is not an API - it is a queue with a function behind it. Orders, emails, thumbnails, webhooks: the producer drops a message in SQS and Lambda works through the backlog. Lambda does the polling for you through an event source mapping, and almost every production problem with this shape comes from three settings: the visibility timeout, how a batch fails, and where failures end up.
Need to know: an event source mapping polls the queue, receives up to BatchSize messages (10 by default for SQS, up to 10,000 with a batching window) and invokes the function once per batch with event.Records. If the function returns normally, the mapping deletes the whole batch; if it throws, nothing is deleted and the whole batch comes back after the visibility timeout. With ReportBatchItemFailures the function returns the ids that failed and only those come back. The queue's redrive policy still decides when a message gives up and goes to the DLQ.
Connect the queue
try-orders has a DLQ (maxReceiveCount 2) and a visibility timeout of 10 seconds; try-worker has a timeout of 30 seconds:
$ cd ~/oncall-lab/labs/aws/try6
$ cat worker/index.mjs
export const handler = async (event) => {
if (!event.Records) throw new Error('not an SQS event: ' + JSON.stringify(event));
const batchItemFailures = [];
for (const r of event.Records) {
const order = JSON.parse(r.body);
if (order.bad) {
console.log('cannot process order', order.id);
if (process.env.PARTIAL === 'on') { batchItemFailures.push({ itemIdentifier: r.messageId }); continue; }
throw new Error('order ' + order.id + ' is malformed');
}
console.log('processed order', order.id);
}
return { batchItemFailures };
};
$ aws lambda create-event-source-mapping --function-name try-worker --event-source-arn arn:aws:sqs:eu-central-1:111122223333:try-orders --batch-size 10
aws: [ERROR]: An error occurred (InvalidParameterValueException) when calling the CreateEventSourceMapping operation: Queue visibility timeout: 10 seconds is less than Function timeout: 30 seconds
Lambda refuses: while the function may still be working on a batch, the messages must stay invisible, or a second poller would hand them to another copy of the function. AWS recommends a visibility timeout of at least six times the function timeout (room for retries of a throttled batch). Fix the queue, not the function:
$ Q=$(aws sqs get-queue-url --queue-name try-orders --query QueueUrl --output text)
$ aws sqs set-queue-attributes --queue-url $Q --attributes VisibilityTimeout=180
$ aws lambda create-event-source-mapping --function-name try-worker --event-source-arn arn:aws:sqs:eu-central-1:111122223333:try-orders --batch-size 10 --query '[UUID,State,BatchSize]' --output text
d513291c-f36f-4168-bd5e-597c624f59ef Creating 10
$ sleep 5
$ aws lambda list-event-source-mappings --function-name try-worker --query 'EventSourceMappings[0].[State,StateTransitionReason]' --output text
Enabled USER_INITIATED
The execution role needs sqs:ReceiveMessage, sqs:DeleteMessage and sqs:GetQueueAttributes on the queue - the managed policy AWSLambdaSQSQueueExecutionRole has them. Without them the mapping is created and then reports the error in StateTransitionReason and LastProcessingResult, not in your logs.
One bad message, a whole batch back
Three orders, the second one malformed:
$ for i in 1 2 3; do aws sqs send-message --queue-url $Q --message-body "{\"id\":$i,\"bad\":$([ $i = 2 ] && echo true || echo false)}" --query MessageId --output text; done
c0ae4440-54a0-4af4-821f-1f9763cb060f
94983b61-586f-4668-a2a5-6695341f228b
94fc4d19-5377-43b3-a607-33d378313773
$ sleep 10
$ aws logs tail /aws/lambda/try-worker --format short | grep -v 'START\|END\|REPORT'
2026-09-22T20:00:11 2026-09-22T20:00:11.157Z 52397999-4959-4861-b858-1384653817bd INFO processed order 1
2026-09-22T20:00:11 2026-09-22T20:00:11.157Z 52397999-4959-4861-b858-1384653817bd INFO cannot process order 2
2026-09-22T20:00:11 2026-09-22T20:00:11.159Z 52397999-4959-4861-b858-1384653817bd ERROR Invoke Error {"errorType":"Error","errorMessage":"order 2 is malformed","stack":["Error: order 2 is malformed"," at Runtime.handler (file:///var/task/index.mjs:9:13)"," at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1206:29)"]}
$ aws sqs get-queue-attributes --queue-url $Q --attribute-names ApproximateNumberOfMessages ApproximateNumberOfMessagesNotVisible --query 'Attributes' --output text
0 3
Order 1 was processed, then the function threw on order 2 (order 3 never ran) - and all three are back in flight. In 180 seconds the batch is processed again: order 1 twice, a customer charged twice if the handler is not idempotent. Turn on partial batch responses: the mapping has to know about them (--function-response-types) and the code has to return them (here an environment variable switches the handler to that mode):
$ M=$(aws lambda list-event-source-mappings --function-name try-worker --query 'EventSourceMappings[0].UUID' --output text)
$ aws lambda update-event-source-mapping --uuid $M --function-response-types ReportBatchItemFailures --query FunctionResponseTypes
[
"ReportBatchItemFailures"
]
$ aws lambda update-function-configuration --function-name try-worker --environment 'Variables={PARTIAL=on}' --query Environment
{
"Variables": {
"PARTIAL": "on"
}
}
$ aws lambda wait function-updated-v2 --function-name try-worker
$ sleep 200
$ aws logs tail /aws/lambda/try-worker --since 4m --format short | grep -v 'START\|END\|REPORT'
2026-09-22T20:00:11 2026-09-22T20:00:11.157Z 52397999-4959-4861-b858-1384653817bd INFO processed order 1
2026-09-22T20:00:11 2026-09-22T20:00:11.157Z 52397999-4959-4861-b858-1384653817bd INFO cannot process order 2
2026-09-22T20:00:11 2026-09-22T20:00:11.159Z 52397999-4959-4861-b858-1384653817bd ERROR Invoke Error {"errorType":"Error","errorMessage":"order 2 is malformed","stack":["Error: order 2 is malformed"," at Runtime.handler (file:///var/task/index.mjs:9:13)"," at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1206:29)"]}
2026-09-22T20:03:11 2026-09-22T20:03:11.179Z 217636f4-06f9-4b76-95aa-0774149adbe3 INFO processed order 1
2026-09-22T20:03:11 2026-09-22T20:03:11.179Z 217636f4-06f9-4b76-95aa-0774149adbe3 INFO cannot process order 2
2026-09-22T20:03:11 2026-09-22T20:03:11.179Z 217636f4-06f9-4b76-95aa-0774149adbe3 INFO processed order 3
$ aws sqs get-queue-attributes --queue-url $Q --attribute-names ApproximateNumberOfMessages ApproximateNumberOfMessagesNotVisible --query 'Attributes' --output text
0 1
The redelivered batch ran again (order 1 a second time - the cost of the first failure); this time order 3 went through too and only order 2 was reported and stayed. It has now been received twice, so the next receive moves it to the DLQ:
$ sleep 200
$ D=$(aws sqs get-queue-url --queue-name try-orders-dlq --query QueueUrl --output text)
$ aws sqs receive-message --queue-url $D --query 'Messages[].Body' --output text
{"id":2,"bad":true}
Two rules for the handler: report failures with itemIdentifier = the record's messageId, and never both throw and report (a thrown error fails the whole batch again). An empty or invalid response with ReportBatchItemFailures on counts as complete success or complete failure, so test it.
Scaling and throttling a mapping
The mapping scales its pollers with the backlog - up to 1,000 concurrent invocations for one queue - and every batch is an invocation, so it competes for the function's concurrency. Two ways to protect what is behind the function (a database with 50 connections):
--scaling-config MaximumConcurrency=Non the mapping (2 to 1,000): the mapping never runs more than N batches at once. Excess messages simply wait in the queue - no throttling errors.- reserved concurrency on the function: a hard cap for every trigger. Batches beyond it are throttled; the messages go back to the queue and count as a receive, so a long throttle can push healthy messages into the DLQ. Prefer
MaximumConcurrencyfor queues.
Watch ApproximateAgeOfOldestMessage (how far behind you are) and ApproximateNumberOfMessagesVisible on the queue, and Throttles and ConcurrentExecutions on the function. A backlog that grows while the function is idle at a low concurrency is a cap you set; a backlog with errors is a bad message or a broken dependency.
Asynchronous invocations: retries and destinations
When S3, SNS or EventBridge invoke a function they use the asynchronous path: Lambda queues the event and, on an error, retries twice (after about 1 and 2 minutes). Then the event is gone
- unless you configure where it goes. An on-failure destination (SQS, SNS, Lambda,
EventBridge, S3) gets a record with the event and the error; an on-success destination gets the result. The older dead-letter queue setting on a function gets only the event.
$ aws lambda put-function-event-invoke-config --function-name try-worker --maximum-retry-attempts 1 --destination-config '{"OnFailure":{"Destination":"arn:aws:sqs:eu-central-1:111122223333:try-failed"}}' --query '[MaximumRetryAttempts,DestinationConfig.OnFailure.Destination]' --output text
1 arn:aws:sqs:eu-central-1:111122223333:try-failed
$ aws lambda invoke --function-name try-worker --invocation-type Event --payload '{"source":"manual"}' /dev/null
{
"StatusCode": 202
}
$ sleep 90
$ aws sqs receive-message --queue-url $(aws sqs get-queue-url --queue-name try-failed --query QueueUrl --output text) --query 'Messages[0].Body' --output text | jq '{condition: .requestContext.condition, attempts: .requestContext.approximateInvokeCount, error: .responsePayload.errorMessage, event: .requestPayload}'
{
"condition": "RetriesExhausted",
"attempts": 2,
"error": "not an SQS event: {\"source\":\"manual\"}",
"event": {
"source": "manual"
}
}
Destinations are for asynchronous invocations (and, as on-failure destinations, for Kinesis and DynamoDB stream mappings). For SQS mappings the failure path is the queue's own DLQ - a DLQ or destination on the function is ignored for them.
On the exam (SAA-C03 2.1, 3.3; SOA-C03 1.2 and 2.1): Lambda + SQS to decouple and absorb bursts; visibility timeout vs function timeout (6x); partial batch responses; a DLQ on the source queue for poison messages; MaximumConcurrency or reserved concurrency to protect a database; async retries, destinations vs DLQ; "messages are processed more than once" = visibility timeout too short or no idempotency.
In an interview: "A message keeps getting processed again and again. Why?" - "Either it is never deleted - the function throws, or the batch fails because of another message and there is no partial batch response - or the visibility timeout is shorter than the processing, so it reappears while it is still being worked on. I'd check the function's errors and duration against the queue's visibility timeout, add ReportBatchItemFailures and a DLQ, and make the handler idempotent."
You can now: connect a queue to a function with the right visibility timeout and role, explain why a thrown error redelivers a whole batch, turn on and return partial batch responses, follow a poison message to the DLQ, cap a mapping with MaximumConcurrency instead of throttling it, and route failed asynchronous events to an on-failure destination with the error attached.