Lambda & Messaging

Lambda & Messaging Quick Notes

AWS Lambda

• Run code without managing servers (~15-minute max duration class limits).
• Default network: AWS-owned network with internet access.
• VPC-attached: Only when you must reach private ENIs (plan NAT/endpoints).
• Reuse code: Lambda Layers.
• Cross-account S3: Execution role permissions and bucket policy allowing that role.
• RDS at scale: Put RDS Proxy in front; prefer IAM DB auth over embedded passwords.
• Concurrency: Account/Region quota is shared — SNS → Lambda floods can hit it (raise via Support).

Amazon SQS

Standard, FIFO

Order: Best-effort, Strict

Delivery: At-least-once, Exactly-once processing

Throughput: Nearly unlimited, ~300 msg/s, or ~3,000 with batching

Name: Any, Must end with .fifo

• Visibility timeout: Hidden while consumer works; reappears if not deleted in time.
• Delay queue: Postpone new messages (0–15 min). Not the same as visibility timeout.
• DLQ: After ReceiveCount exceeds max — poison messages.
• Message Group ID (FIFO): Same group = ordered; different groups = parallel consumers.
• Long polling: Cost-efficient receives (WaitTimeSeconds > 0).
• Migrate Standard → FIFO: Cannot convert in place — delete and recreate with .fifo.
• Temporary queues: Short-lived request/response without permanent queue sprawl.

Amazon SNS

• Pub/sub fan-out; ≤256 KB payloads.
• One event → many workflows: SNS → multiple SQS queues (or EventBridge).

Amazon EventBridge

• Match event patterns or schedules → many targets.
• Weekly/monthly jobs: EventBridge cron → Lambda.

Step Functions

• Standard: Long-running workflows (up to ~1 year class).
• Express: High-volume short work.
• Pair with AWS Batch for large batch/image fleets.

Kinesis Family

Service: Idea

Data Streams: Real-time; multi-consumer; rolling retention window

Firehose: Near-real-time load to S3/OpenSearch/etc.; optional Lambda transform

Data Analytics: SQL on the stream

Video Streams: Live media + CV integrations

• SQS vs Streams: SQS ≈ one consumer group; Streams ≈ many consumers + retained window.
• Firehose source trap: Kinesis Agent cannot write to a Firehose whose source is already Data Streams.
• Firehose destinations: Do not include DynamoDB.
• Serverless sensor path: SQS → Lambda (batch) → DynamoDB.

API Gateway (quick)

• REST: Stateless. WebSocket: Stateful, full-duplex.
• Throttle buffer trio: API Gateway + SQS + Kinesis.
• Caching: Stage cache cuts repeated backend hits.
• Built-in user directory for APIs → Cognito User Pools.