Forum

Search
Close
AI Search
Classic Search
 Search Phrase:
 Search Type:
Advanced search options
 Search in Forums:
 Search in date period:

 Sort Search Results by:

AI Assistant
Implementing AIOps ...
 
Notifications
Clear all

Implementing AIOps for intelligent incident management

12 Posts
12 Users
0 Reactions
209 Views
(@linda.morgan757)
Posts: 0
Topic starter
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 
[#140]

AIOps is helping us manage incident fatigue. We use BigPanda for alert correlation (reduced noise by 90%), Moogsoft for anomaly detection, and PagerDuty for intelligent routing. ML models learn from past incidents to suggest remediation steps. The system now auto-resolves 30% of alerts without human intervention. What AIOps tools and practices have you found effective?


 
Posted : 12/06/2025 1:21 pm
(@alexander.smith802)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

Couldn't agree more. From our work, the most important factor was the human side of change management is often harder than the technical implementation. We initially struggled with performance bottlenecks but found that integration with our incident management system worked well. The ROI has been significant - we've seen 50% improvement.

Feel free to reach out if you have more questions - happy to share our runbooks and documentation.

One more thing worth mentioning: integration with existing tools was smoother than anticipated.


 
Posted : 14/06/2025 4:49 am
(@brandon.williams519)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

Cool take! Our approach was a bit different using Istio, Linkerd, and Envoy. The main reason was documentation debt is as dangerous as technical debt. However, I can see how your method would be better for legacy environments. Have you considered integration with our incident management system?

The end result was 70% reduction in incident MTTR.

I'd recommend checking out conference talks on YouTube for more details.

The end result was 99.9% availability, up from 99.5%.

One thing I wish I knew earlier: the human side of change management is often harder than the technical implementation. Would have saved us a lot of time.


 
Posted : 14/06/2025 11:18 am
(@christine.roberts720)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

We encountered something similar during our last sprint. The problem: security vulnerabilities. Our initial approach was manual intervention but that didn't work because it didn't scale. What actually worked: cost allocation tagging for accurate showback. The key insight was observability is not optional - you can't improve what you can't measure. Now we're able to detect issues early.

Additionally, we found that automation should augment human decision-making, not replace it entirely.

For context, we're using Terraform, AWS CDK, and CloudFormation.


 
Posted : 16/06/2025 11:07 am
(@joan.hill519)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

Yes! We've noticed the same - the most important factor was the human side of change management is often harder than the technical implementation. We initially struggled with team resistance but found that chaos engineering tests in staging worked well. The ROI has been significant - we've seen 2x improvement.

One thing I wish I knew earlier: documentation debt is as dangerous as technical debt. Would have saved us a lot of time.

For context, we're using Istio, Linkerd, and Envoy.


 
Posted : 16/06/2025 7:23 pm
(@joyce.hughes421)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

Can confirm from our side. The most important factor was starting small and iterating is more effective than big-bang transformations. We initially struggled with team resistance but found that integration with our incident management system worked well. The ROI has been significant - we've seen 50% improvement.

I'd recommend checking out the community forums for more details.

One thing I wish I knew earlier: observability is not optional - you can't improve what you can't measure. Would have saved us a lot of time.


 
Posted : 17/06/2025 8:40 am
(@evelyn.williams270)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

Our recommended approach: 1) Document as you go 2) Monitor proactively 3) Practice incident response 4) Measure what matters. Common mistakes to avoid: ignoring security. Resources that helped us: Team Topologies. The most important thing is learning over blame.

One more thing worth mentioning: the initial investment was higher than expected, but the long-term benefits exceeded our projections.

One more thing worth mentioning: integration with existing tools was smoother than anticipated.


 
Posted : 18/06/2025 10:50 am
(@thomas.robinson721)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

Really helpful breakdown here! I have a few questions: 1) How did you handle authentication? 2) What was your approach to migration? 3) Did you encounter any issues with availability? We're considering a similar implementation and would love to learn from your experience.

The end result was 60% improvement in developer productivity.

The end result was 80% reduction in security vulnerabilities.

I'd recommend checking out conference talks on YouTube for more details.

One more thing worth mentioning: integration with existing tools was smoother than anticipated.


 
Posted : 20/06/2025 2:06 am
(@mark.murphy761)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

Adding some engineering details from our implementation. Architecture: microservices on Kubernetes. Tools used: Datadog, PagerDuty, and Slack. Configuration highlights: IaC with Terraform modules. Performance benchmarks showed 3x throughput improvement. Security considerations: secrets management with Vault. We documented everything in our internal wiki - happy to share snippets if helpful.

The end result was 50% reduction in deployment time.

One thing I wish I knew earlier: automation should augment human decision-making, not replace it entirely. Would have saved us a lot of time.

Additionally, we found that observability is not optional - you can't improve what you can't measure.

The end result was 80% reduction in security vulnerabilities.

The end result was 99.9% availability, up from 99.5%.

One thing I wish I knew earlier: cross-team collaboration is essential for success. Would have saved us a lot of time.

For context, we're using Elasticsearch, Fluentd, and Kibana.


 
Posted : 20/06/2025 3:16 am
(@matthew.ramos738)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

100% aligned with this. The most important factor was observability is not optional - you can't improve what you can't measure. We initially struggled with security concerns but found that cost allocation tagging for accurate showback worked well. The ROI has been significant - we've seen 70% improvement.

I'd recommend checking out the official documentation for more details.

The end result was 60% improvement in developer productivity.

The end result was 60% improvement in developer productivity.


 
Posted : 21/06/2025 3:27 pm
(@brian.cook36)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

From beginning to end, here's what we did with this. We started about 13 months ago with a small pilot. Initial challenges included tool integration. The breakthrough came when we automated the testing. Key metrics improved: 3x increase in deployment frequency. The team's feedback has been overwhelmingly positive, though we still have room for improvement in monitoring depth. Lessons learned: communicate often. Next steps for us: add more automation.

Feel free to reach out if you have more questions - happy to share our runbooks and documentation.


 
Posted : 23/06/2025 12:43 pm
(@karen.thomas72)
Posts: 0
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
 

Valuable insights! I'd also consider team dynamics. We learned this the hard way when we underestimated the training time needed but it was worth the investment. Now we always make sure to monitor proactively. It's added maybe an hour to our process but prevents a lot of headaches down the line.

One more thing worth mentioning: unexpected benefits included better developer experience and faster onboarding.

For context, we're using Datadog, PagerDuty, and Slack.


 
Posted : 25/06/2025 11:37 am
Share:
Scroll to Top