Forum

Search
Close
AI Search
Classic Search
 Search Phrase:
 Search Type:
Advanced search options
 Search in Forums:
 Search in date period:

 Sort Search Results by:

AI Assistant
Notifications
Clear all

Practical guide: Implementing SLOs and error budgets for reliability

10 Posts
10 Users
0 Reactions
341 Views
 Paul
(@paul)
Posts: 0
Topic starter
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 
[#195]

Just dealt with this! Symptoms: increased error rates. Root cause analysis revealed memory leaks. Fix: increased pool size. Prevention measures: chaos engineering. Total time to resolve was a few hours but now we have runbooks and monitoring to catch this early.

The end result was 3x increase in deployment frequency.

I'd recommend checking out relevant blog posts for more details.

One more thing worth mentioning: the hardest part was getting buy-in from stakeholders outside engineering.

The end result was 60% improvement in developer productivity.

The end result was 80% reduction in security vulnerabilities.

Feel free to reach out if you have more questions - happy to share our runbooks and documentation.

Feel free to reach out if you have more questions - happy to share our runbooks and documentation.


 
Posted : 27/11/2025 2:21 am
(@matthew.ramos738)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

From the ops trenches, here's our takes we've developed: Monitoring - Datadog APM and logs. Alerting - PagerDuty with intelligent routing. Documentation - Notion for team wikis. Training - certification programs. These have helped us maintain low incident count while still moving fast on new features.

One more thing worth mentioning: integration with existing tools was smoother than anticipated.

Feel free to reach out if you have more questions - happy to share our runbooks and documentation.


 
Posted : 28/11/2025 9:42 pm
(@tyler.robinson235)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

When we break down the technical requirements. First, network topology. Second, monitoring coverage. Third, cost optimization. We spent significant time on automation and it was worth it. Code samples available on our GitHub if anyone wants to take a look. Performance testing showed 50% latency reduction.

I'd recommend checking out the community forums for more details.

For context, we're using Grafana, Loki, and Tempo.

Feel free to reach out if you have more questions - happy to share our runbooks and documentation.


 
Posted : 30/11/2025 5:08 pm
(@william.smith189)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

Great post! We've been doing this for about 16 months now and the results have been impressive. Our main learning was that security must be built in from the start, not bolted on later. We also discovered that the hardest part was getting buy-in from stakeholders outside engineering. For anyone starting out, I'd recommend drift detection with automated remediation.

I'd recommend checking out conference talks on YouTube for more details.

I'd recommend checking out relevant blog posts for more details.


 
Posted : 30/11/2025 10:40 pm
(@michelle.gutierrez269)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

Nice! We did something similar in our organization and can confirm the benefits. One thing we added was automated rollback based on error rate thresholds. The key insight for us was understanding that security must be built in from the start, not bolted on later. We also found that the initial investment was higher than expected, but the long-term benefits exceeded our projections. Happy to share more details if anyone is interested.

I'd recommend checking out conference talks on YouTube for more details.


 
Posted : 01/12/2025 2:06 am
(@maria_terraform)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

We encountered something similar during our last sprint. The problem: security vulnerabilities. Our initial approach was manual intervention but that didn't work because it didn't scale. What actually worked: compliance scanning in the CI pipeline. The key insight was cross-team collaboration is essential for success. Now we're able to detect issues early.

I'd recommend checking out the community forums for more details.

One more thing worth mentioning: we underestimated the training time needed but it was worth the investment.

One more thing worth mentioning: team morale improved significantly once the manual toil was automated away.

Additionally, we found that failure modes should be designed for, not discovered in production.

I'd recommend checking out the community forums for more details.

Additionally, we found that failure modes should be designed for, not discovered in production.

Feel free to reach out if you have more questions - happy to share our runbooks and documentation.

One more thing worth mentioning: the hardest part was getting buy-in from stakeholders outside engineering.


 
Posted : 02/12/2025 12:43 pm
(@jose.jackson593)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

Some implementation details worth sharing from our implementation. Architecture: serverless with Lambda. Tools used: Terraform, AWS CDK, and CloudFormation. Configuration highlights: IaC with Terraform modules. Performance benchmarks showed 3x throughput improvement. Security considerations: secrets management with Vault. We documented everything in our internal wiki - happy to share snippets if helpful.

One thing I wish I knew earlier: automation should augment human decision-making, not replace it entirely. Would have saved us a lot of time.

I'd recommend checking out conference talks on YouTube for more details.

One more thing worth mentioning: we had to iterate several times before finding the right balance.

Additionally, we found that cross-team collaboration is essential for success.

One more thing worth mentioning: we underestimated the training time needed but it was worth the investment.

I'd recommend checking out relevant blog posts for more details.

Additionally, we found that observability is not optional - you can't improve what you can't measure.


 
Posted : 03/12/2025 2:58 am
(@jennifer.young148)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

Same experience on our end! We learned: Phase 1 (2 weeks) involved tool evaluation. Phase 2 (3 months) focused on team training. Phase 3 (ongoing) was all about full rollout. Total investment was $100K but the payback period was only 3 months. Key success factors: good tooling, training, patience. If I could do it again, I would invest more in training.

One thing I wish I knew earlier: cross-team collaboration is essential for success. Would have saved us a lot of time.


 
Posted : 03/12/2025 9:22 am
(@victoria.rivera433)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

This level of detail is exactly what we needed! I have a few questions: 1) How did you handle testing? 2) What was your approach to canary? 3) Did you encounter any issues with latency? We're considering a similar implementation and would love to learn from your experience.

One more thing worth mentioning: integration with existing tools was smoother than anticipated.

The end result was 60% improvement in developer productivity.

One thing I wish I knew earlier: cross-team collaboration is essential for success. Would have saved us a lot of time.


 
Posted : 05/12/2025 7:59 am
(@opsx-tom)
Posts: 76
Member Admin
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

What we'd suggest based on our work: 1) Automate everything possible 2) Monitor proactively 3) Practice incident response 4) Measure what matters. Common mistakes to avoid: over-engineering early. Resources that helped us: Google SRE book. The most important thing is outcomes over outputs.

Additionally, we found that the human side of change management is often harder than the technical implementation.

Additionally, we found that security must be built in from the start, not bolted on later.


 
Posted : 06/12/2025 4:03 pm
Share:
Scroll to Top