AWS SQS – failed message clogs queue
I have a long running-task (10 minutes) executed via a lambda function. The visibility timeout for my SQS queue is set to 15 minutes and the lambda event source for the queue has a max concurrency of 2, no retries. The batch size is set to 1.
Trigger lambda only after receiving message in SQS from all datacentres
I have say 4 DC and each generates a message independently and writes to SQS queue. Each DC may have its own delay and may not write at same time as other. DC’s can have difference of as large as 5-10 min in writing to SQS.
How can I trigger a lambda after all DC have written the message.