We had an interesting (but not terribly surprising) situation occur with one of our DR as a Service customers over the holiday weekend, and it warrants a discussion on two fronts: The customer’s situation itself (the use case) and the service response provided by our VeriStor Cloud Services team.
Last Friday before the Memorial Day holiday weekend our ops center received a ticket from a customer stating that they may need to declare (fail-over). Actually it was probably a phone call, as this was a long-standing VeriStor customer with contacts up the support chain…and as much as we all prefer portal-based tickets, the ability to pick up the phone and talk to a team of DR/Continuity experts who understand the specific environment is something we have always prided ourselves in.
The issue had absolutely nothing to do with customer’s own environment. It just happens that they are colocated in a well-known shared datacenter that had been experiencing intermittent extreme DDOS attacks against one of the other tenants. For whatever reason the provider wasn’t able to mitigate the attacks to the point where our customer was confident that their critical operations were going to be dependable over the weekend (and heading into a critical set of scheduled processes). While there are certainly other ways to deal with these types of attacks, sometimes it’s out of your hands…and other times it’s just really bad timing. This one was a case of both. So another great use-case for an effective (and cost-effective) DR/Continuity as a Service Offering.
And for the record, the average RPO for this customer’s systems runs around 10 seconds (no, that’s not a typo) and we have an 2 hour RTO SLA in place. Those of you who understand DR/Continuity will know that these types of numbers were unheard of and/or prohibitively expensive until very recently. These days, with the right platforms in place, they are the new norm… and very affordable.
Service Response: While we work hard to make all of this as transparent to the customer as possible, the reality is that smart people need to be involved, up-front and through the process. In this case, there were multiple engineers engaged and standing by. Not just the on-call level-1 NOC guys…but level-2 and even level-3 engineers who were ready to give up a part of their holiday weekend to make sure this customer’s failover went perfectly. THAT is the SERVICE in DR as a Service. Technology is great (and getting better everyday), but talented and committed people still make the difference.
(Photo: VeriStor Cloud Engineers Marco Antinarella, Demaris Hickinbottom, and Garret DeWulf on call with the customer.)
In the end, the DDOS attacks subsided, for whatever reason, and the customer did not need to failover. But I have to believe that it made life for them a whole lot better knowing that their business was going to operate regardless, and that was a state-of the art platform and team of experts standing by to make sure that happened.
So for me, this was not only an interesting use-case example of the value this type of service and the power of the platform we have in place, but more importantly yet more validation of quality and integrity of the team we have in place to provide it. It made me tremendously proud of our people and our service.
(Which is why I stopped by to listen to the call and decided to take a photo.)