Anuj Tyagi

Monday AWS Misconception: Many engineers new to AWS think SNS (Simple Notification Service) and SQS (Simple Queue Service) can be used interchangeably for message passing or notifications—but they solve fundamentally different problems! 👉 Misconception 1: SQS pushes messages to consumers like SNS does. 💡 Reality: SQS is pull-based—consumers must poll the queue to receive messages. SNS is push-based, instantly delivering messages to all subscribers (e.g., Lambda, HTTP endpoints) as soon as they’re published.​ 👉 Misconception 2: Both services are just for internal service-to-service communication. 💡 Reality: While SQS is designed for reliable, decoupled service-to-service (A2A) messaging, SNS can handle both A2A (e.g., pushing events to Lambda, SQS, or HTTP) and A2P (application-to-person) use cases like SMS, email, or mobile notifications.​ 👉 Misconception 3: Messages are always durable with SNS. 💡 Reality: SNS does not retain messages—if no subscriber is available at delivery time, the message can be lost. SQS stores messages until they're processed (from 1 minute up to 14 days), giving services time to catch up and providing durability and retry options.​ 👉 Misconception 4: You can't use SNS and SQS together. 💡 Reality: The fan-out pattern (SNS + SQS) is a common best practice: publish a message to an SNS topic, and have multiple SQS queues as subscribers for reliable delivery and independent, decoupled processing. This allows both real-time notifications and durable, scalable message processing.​ 📝 Tips: 🖋️ Use SNS for real-time pub/sub—events, alarms, notifications. 🖋️ Use SQS for asynchronous task processing and reliable decoupling. 🖋️ Combine them for scalable, durable event-driven architectures. What other AWS messaging misconceptions have you encountered out there? Share your thoughts and help me discover more! …and here’s a fantastic article I came across by AWS Fundamentals https://lnkd.in/ekMWAJZE. It clearly explains SQS fundamentals beautifully and highlights key misconceptions about how SQS works in event-driven architectures, especially around polling versus pushing and visibility timeouts. Highly recommended for anyone looking to deepen their AWS messaging knowledge! #aws #cloud #sns #sqs #cloudcomputing #devops

22

💡 Lessons from the AWS DynamoDB Outage This week’s AWS outage — triggered by a DNS issue in US-EAST-1 that cascaded into DynamoDB and other core services — was a strong reminder of a simple truth: Even the most reliable platforms have failure modes. Many businesses across the globe experienced ripple effects. Some degraded gracefully. Others went dark. The difference came down to architecture resilience — and, more importantly, organizational readiness. A few leadership reflections: 🔹 Operational resilience is a leadership choice. Designing for failure isn’t only a technical decision. It’s about where we invest time, complexity, and cost before incidents happen. 🔹 Dependencies compound risk. We often celebrate velocity and abstraction in cloud architecture — until a core dependency like DynamoDB falters. Leaders need to routinely ask: “What’s our plan if this dependency fails at 2 a.m.?” 🔹 Communication beats perfection. In outages, your team’s clarity matters more than the system’s uptime. Teams that have practiced incident response — with clear ownership and psychological safety — recover faster. 🔹 Postmortems are gold, if we treat them that way. When AWS publishes their root cause analysis, resist the urge to just skim it. Map their failure pattern to your own systems. Every incident, even external, is a free rehearsal. We can’t eliminate outages. But we can choose how prepared, transparent, and adaptive our teams are when they arrive. The real test of reliability isn’t when things are working — it’s how we behave when they don’t. #Leadership #ReliabilityEngineering #AWS #DynamoDB #EngineeringManagement #IncidentResponse #SystemDesign

5

I have a goal to run all full-blown Treatwise 5 microservices written in Go on AWS under $100/month. While it's very possible, I have done some insane infrastructure optimizations: ✅ VPC Optimization — Cut down VPC to provision only in 2 AZs ✅ Network Cost Reduction — Leverage VPC endpoints for most intra-VPC network connectivity without traversing the public internet to save on egress and NAT costs ✅ Database Strategy — Using RDS Serverless V2. RDS Proxy added more cost and maintained a persistent connection pool which prevents true scale to zero. I removed it and managed database pooling at the application layer. ✅ Compute Optimization — Scheduled all services on ECS Fargate with minimum resources and scale to zero for non-critical workloads. Only Identity service can't tolerate cold-start, scheduled as always-on. ❌ The Culprit? I thought serverless was going to help me with ElastiCache for Redis too until I realized the base minimum cost for 1000 EPU is ~$90/month, while my previous cluster-mode with one node was at $40/month. 💡 Lesson: Always review costs and thinking serverless is going to save you all the time. All the cost optimizations I implemented were with the intention of not affecting reliability, performance and high availability. ⚠️ NOTE: For any serverless offering, if it doesn't offer scale-to-zero with minimal cost on startup, probably not worth it. #cloudengineering #cloudcomputing #softwarengineering #awscloud #serverless

63

7 Comments

Kubernetes just dropped something useful for anyone running non-trivial clusters — Node Readiness Controller 👇 Traditionally, Kubernetes decides if a node can run workloads using a single “Ready” flag. That’s simple, but in real clusters you often have extra pieces — network agents, storage drivers, GPUs, custom checks — that must be healthy before a node truly should accept pods. The new Node Readiness Controller lets you define fine-grained, declarative readiness conditions so nodes only accept workloads once all required components are actually ready. This helps with bootstrapping complex node setups, enforcing health requirements, and improving reliability. In other words: >> You can have custom readiness gates beyond the built-in Ready condition >> You get more control over heterogeneous nodes (eg GPU vs general workloads) >> It fits better with real-world infra where services must be healthy before scheduling starts If you’re working with clusters that need more than the default “ready/not ready”, this controller is worth a look. https://lnkd.in/gVzbeevy #Kubernetes #cloudnative #containers #devops #infrastructure

28

🎉 NVIDIA Dynamo V0.2 now delivers GPU autoscaling, Kubernetes support automation, and networking optimizations. The latest release includes: ✅ A planner for prefill and decode GPU autoscaling ✅ Kubernetes automation for large-scale Dynamo deployments ✅ Support for Amazon EFA for internode data transfers on Amazon Web Services (AWS) Dynamo is a high-throughput, low-latency #opensource inference serving framework for deploying generative AI and reasoning models in large-scale distributed environments. #COMPUTEX

3