Deep dive: On-call rotation best practices to prevent burnout

Donald White · 2024-12-22T19:21:13Z

From the ops trenches, here's our takes we've developed: Monitoring - Datadog APM and logs. Alerting - custom Slack integration. Documentation - Notion for team wikis. Training - certification programs. These have helped us maintain fast deployments while still moving fast on new features. For context, we're using Elasticsearch, Fluentd, and Kibana. Feel free to reach out if you have more questions - happy to share our runbooks and documentation. Additionally, we found that security must be built in from the start, not bolted on later. The end result was 3x increase in deployment frequency. The end result was 90% decrease in manual toil. For context, we're using Istio, Linkerd, and Envoy. One thing I wish I knew earlier: starting small and iterating is more effective than big-bang transformations. Would have saved us a lot of time.

✦ Summarize Topic

Page 2 / 2 Prev

AWS Cloud

Last Post by Michelle Ross 1 year ago

16 Posts

15 Users

0 Reactions

383 Views

RSS

Michelle Ross

(@michelle.ross286)

Posts: 0

Translate ▼

Some implementation details worth sharing from our implementation. Architecture: hybrid cloud setup. Tools used: Istio, Linkerd, and Envoy. Configuration highlights: CI/CD with GitHub Actions workflows. Performance benchmarks showed 99.99% availability. Security considerations: container scanning in CI. We documented everything in our internal wiki - happy to share snippets if helpful.

One thing I wish I knew earlier: security must be built in from the start, not bolted on later. Would have saved us a lot of time.

Posted : 06/01/2025 11:07 pm

Page 2 / 2 Prev

11 Forums
309 Topics
4,684 Posts
0 Online
109 Members

Forum Icons: Forum contains no unread posts Forum contains unread posts

Topic Icons: Not Replied Replied Active Hot Sticky Unapproved Solved Private Closed