DOS which means Denial of Service is a security event that denies access to your service by sending in multiple requests to your server in a very short period of time with the intent to flood your server with traffic that it can not possibly handle thereby forcing it to have a downtime.
This is a postmortem report on the event of a DDOS attack on BaseBox Software Solutions inc. on the 23rd of October, 2023.
Issue Summary:
Duration: The outage occurred on the 23rd of October, 2023 between 1:30 AM and 2:15 AM WAT and lasted for 45 minutes.
Impact: The incident affected the managed DNS service. Approximately 60% of users were affected, experiencing slow response times and in some cases inability to access certain features.
Root Cause: The root cause of the outage was a DDOS attack believed to have been carried out by malicious hackers with the intent to distort the managed DNS infrastructure thereby impacting service delivery and sales.
Timeline:
Detection: The issue was detected on the 23rd of October, 2023 between 1:30 AM and 2:15 AM WAT when our Site Reliability Engineers received a monitoring alert through the monitoring system.
Actions Taken: Immediate actions included initiating investigation and implementing mitigation techniques such as setting up a firewall to block the IP's from where the attack came and activating IP Geolocation blocking. The SRE team investigated the cloud managed DNS infrastructure and assumed the root cause to be a large Distributed Denial of Service on our US-East region service.
Misleading Paths: Misleading investigation paths were taken when the team restarted some of our third party services thinking those could have led to a downtime on the service.
Escalation: The incident was discovered by the Networking Team and escalated to the Software Reliability and Security Teams.
Resolution: The incident was resolved by activating IP Geolocation blocking on the Network to block the geographical region where the malicious attacks emanated from.
Root Cause and Resolution:
Root Cause: The root cause of the outage was a large Distributed Denial of Service on our US-East region service.
Resolution: To resolve the issue, IP Geolocation blocking was activated blocking the geographical region from accessing the network.
Corrective and Preventative Measures:
Improvements/Fixes:
Strengthen DDoS protection mechanisms.
Enhance monitoring and alerting systems for quicker anomaly detection.
Evaluate and improve infrastructure scalability.
Tasks:
Implement advanced DDoS mitigation solutions.
Enhance monitoring systems to provide timely alerts.
Conduct a scalability assessment and implement necessary improvements.
This concise post-mortem report provides an executive summary, a brief timeline of events, detailed information about the root cause and resolution, and specific corrective and preventative measures taken to mitigate this occurrence.
Top comments (0)