In Review: DDoS against HEXONET Aug 29 - Sep 03, 2021
Issue Summary
In follow up to our last Update: HEXONET DDoS Attack (Sept 3. 2021) and regular updates on status.hexonet.net, we have captured the following details to provide further insights to our dedicated and supportive customers. Thank you again for your support during this large attack.
Commencing on the evening of Sunday Aug 08, 2021 CEST, DDoS attacks against HEXONET began, first targeted against one of our websites https://1api.net.
In the course of the following week, HEXONET was hit by several DDoS attack waves, resulting in limited or no availability and reachability of our systems. Mitigation strategies have been performed and counter measures implemented. Ultimately, the additional protections in place are working and have added ongoing strength to our overall systems.
The DDoS attacks have stopped and our monitoring does not record any further attacks since Friday Sep 03, 2021.
Root Cause
The DDoS attack was aimed against HEXONET.net; however, the reason was that we happened to be the registrar of record for a single domain. As a domain registrar, having many reseller and direct clients, it can happen that you sometimes find yourself in the crossfire. We were in such a situation last week.
Remediation and Prevention
We activated our crisis management to cover our communication channels to customers e.g.:
- Taking care of all inquiries by our Support Team
- Informing Key Accounts via their Sales Manager
- Setting up an automatic reply for our Support Ticket System Zendesk
- Sending out notifications to the entire HEXONET client base
Performed Technical Measures:
- Blackholing singular systems under direct attack
- Setting up additional firewalls
- Whitelisting customer IPs to provide access to our API gateways
- Moving singular websites to CloudFlare
- Finalising all automatic renewals 24 hours earlier
- Installing a scrubbing service to filter bad traffic
Timeline and Detailed Description of Impact
Sunday 2021-08-29
16:35 UTC
First DDoS attack wave on https://1api.net/ As a measure we blackholed https://1api.net/ to protect other systems.
Monday 2021-08-30
10:20 UTC
Blackholing lifted for https://1api.net/ DDoS attacks resumed shortly after. So we blackholed https://1api.net/ again.
16:05 until 18:15 UTC
DDoS attack wave on multiple HEXONET services; our availability for customers was severely limited and barely given. This attack lasted about 2 hours.
Tuesday, 2021-08-31
No DDoS attack wave.
Wednesday, 2021-09-01
03:51 UTC
Next DDoS wave, HEXONET is periodically offline. Customer access to our systems is only very sparsely available. This attack wave is very continuous and ongoing until the next day.
8:50 UTC
We have adjusted the HEXONET renewal system and now finalise all automatic renewals 24 hours earlier, effectively renewing 48h prior to expiration. This affects all Domains set to AUTORENEW.
13:00 UTC
After discussions with our datacenter provider and our uplink provider, we placed an additional device before our firewalls. This additional filtering supported the strength and services were slowly recovering. To provide access to HEXONET's API gateways we began whitelisting IP addresses of our clients.
Thursday, 2021-09-02
DDoS attacks are ongoing throughout the day though, during the afternoon, HEXONET's systems slowly became available to our clients again. Later in the evening on this day the DDoS attack came to an end.
In an effort to provide more transparency on our system status we set up a first iteration of status.hexonet.net which is publicly available.
Friday, 2021-09-03
07:00 UTC
We applied further network changes including implementation of a scrubbing service. No more DDoS attack waves recorded.
As always, we remain dedicated to your success and hope this background information supports your understanding for all that transpired. If you have any questions or would like to connect with our team, please always contact us at [email protected].
Your HEXONET Team