1 views collected around this technical thread.
The June 2025 Google Cloud outage, caused by an untested code change that triggered a NullPointerException, crippled over 70 core services worldwide, prompting a rapid technical fix, public apology, and industry‑wide reflections on cloud stability, fault tolerance, and deployment practices.
This article explains the purpose and key features of OnCall platforms, compares popular solutions like PagerDuty, Opsgenie, Grafana OnCall and Alibaba Cloud ARMS, clarifies webhooks with a simple analogy, and summarizes how centralized on‑call management boosts operational efficiency while minimizing manual intervention.
A coordinated overseas cyber‑attack breached a Guangzhou tech firm's self‑service equipment backend, causing hours of service outage, data leakage, and significant losses, prompting swift police investigation, evidence preservation, and a detailed technical analysis of the attackers' methods.
This article explains the importance, pain points, architecture, core capabilities, and future roadmap of the 360 Zhihui Cloud "Yunzhou" unified alert service, showing how it improves observability, reduces alert noise, and accelerates incident response for modern cloud‑native systems.
A recent case where a maintenance worker exploited device‑management flaws to steal confidential files for foreign spies highlights the need for heightened vigilance, strict self‑discipline, and prompt reporting, offering practical steps to safeguard against similar security breaches.
This article presents a detailed walkthrough of constructing a robust security architecture, covering single‑person security team strategies, risk perception and quantification, rapid incident response, automated detection, precise strike mechanisms, deterrence tactics, and forward‑looking plans for intelligent, data‑driven risk management.
This article details a real‑world Linux server breach, describing the symptoms, investigative commands, log analysis, malicious script removal, file attribute unlocking, and practical remediation steps, while highlighting key lessons and preventive measures for future security.
This article documents a step‑by‑step investigation of a compromised Linux server that exhibited 100% CPU usage, detailing process, network, and startup‑service analysis, the discovery of a cryptomining malware, and the complete removal procedure.
This article outlines a comprehensive approach to online stability, covering preventive measures, service governance, capacity planning, incident detection, multi‑dimensional monitoring, alerting, R&D efficiency improvements, team building, and practical guidelines for simplifying, standardizing, automating, and scaling stability initiatives across an organization.
This article describes the design and implementation of a lightweight cloud‑native alert platform, outlining its core features, future enhancements, system architecture, and demo screenshots, offering practical insights for SREs and operations teams handling growing monitoring workloads.
A disgruntled former infrastructure engineer at a U.S. industrial firm deleted backups, locked administrators, and demanded $750,000 in Bitcoin, leading to his arrest and highlighting the severe risks, legal consequences, and mitigation strategies associated with insider ransomware threats.
This article recounts how a solo developer built a Django‑based Docker image signing service to meet PCI DSS requirements, faced two severe incidents—including a 17.5‑hour P0 outage caused by concurrency limits and a misconfigured Rekor service—and shares the operational lessons learned for reliable SRE practice.
This article outlines comprehensive system stability strategies for backend development, covering technical design reviews, key reliability techniques such as rate limiting, circuit breaking, timeout handling, isolation, and deployment safeguards like monitoring, gray releases, and rollback, aiming to reduce incidents and improve operational resilience.
This article demystifies Site Reliability Engineering (SRE) by explaining its origins, core responsibilities, essential skill sets, and key practices such as observability, incident response, testing, capacity planning, automation, user support, on‑call duties, and the definition of SLI/SLO/SLA, providing a comprehensive guide for modern operations teams.
On April 8, a Tencent Cloud API outage caused console login failures for nearly 2,000 customers, affecting several dependent services for 87 minutes, and the detailed root‑cause analysis and subsequent improvement actions are presented to enhance system resilience and change management.
This report details a network operations incident where a BGP routing change caused an EBGP neighbor to go idle, outlines the step‑by‑step troubleshooting, analysis of the root cause, and the implemented solution involving a new L3 node and redundant EBGP peers.
This guide walks ops teams through real‑world incident scenarios, from quick symptom identification and emergency recovery to improving monitoring, crafting concise emergency plans, and leveraging automation for smarter fault handling, helping organizations restore services faster and reduce downtime.
The author recounts a 2018 incident where a cloud disk snapshot was unintentionally made public, exposing customer data, and shares a detailed reflection on the operational mistakes, risk management failures, and recommended safeguards for high‑risk cloud operations.
This article outlines the red‑blue adversarial testing process for a big‑data platform during the Double‑Eleven promotion, detailing its purpose, benefits, step‑by‑step execution, common issues, and recommendations to improve system reliability and security.
The article explains how a configurable yellow banner system can be used on web interfaces to promptly inform users about service disruptions, guide their actions, increase transparency, improve experience, and outline implementation considerations such as configurability, persistence, and independent deployment.