Chennai Flood Exposes Flaws in Business Continuity Plans

California-based technology firm Zoho found itself drowning in a flood of complaints from its web service customers as the water level rose on Chennai streets.

Relentless rain turned major roads into rivers, forcing people to use boats to move around.

As the Indian city of Chennai went under floodwaters following heavy raining, Cognizant asked some of its employees to remain in the office to support critical services. As the rain continued, the company, which has its largest office in Chennai with 60,000 people across 11 facilities, ferried some employees to other cities in a desperate attempt to avoid service outages.

But some companies looked on helplessly as systems went down. California-based technology firm Zoho Corporation, for example, found itself drowning in a flood of complaints from its mail service customers as the water level rose on Chennai streets.

Many enterprises believe that they can survive any disaster if they put in place a robust physical infrastructure and technical expertise. But floods in Chennai confirmed that technology companies need to do more to escape the ill-effects of natural disasters.Their business continuity plan (BCP) may not be complete if they just make sure that their data centers will never shut down and that their data centers are distributed across the world. Putting in place enough human resources is also just as important.

That’s precisely what happened in the case of Zoho. The technology firm has data centers spread across the world, but not the technical experts who investigate customer complaints. By the time  the flood struck Chennai, Zoho had just been recovering from a devastating distributed-denial-of- service (DDOS) attack that slows down movement of mails through servers. “It was our complete bad luck that this bug showed up at the same time as the disaster. We are not using the natural disaster to mask another underlying cause,” explained the company in a blog post.

Then whose mistake was this? It was in fact a loophole in Zoho’s business continuity plan (BCP), because most of the technical staff and part of the company’s mail development team, is based in Chennai. As heavy rains turned Chennai into a pond, the city’s transportation system came to a standstill, forcing employees to stay back at home. But Zoho was badly in need of the technical expertise of the same team to respond to the bug’s unique aftereffects on the customer experience. They were up against a different struggle, while Zoho was up against the software bug.

“Now if that R&D team is unavailable for any reason, the issue remains unresolved until they return, and the customer unfortunately suffers,” went on Zoho in its blog post. Frankly, relying on a single technical staff in a single location was Zoho’s drawback.

Realizing that Chennai might take a longer time to recover, Zoho opened up customer support centers in Austin (USA) and Tenkasi (India). Furthermore, it moved about 50 technical staff to Bangalore, India’s Silicon city about 350 kilometerss west of Chennai, to set up another emergency customer service and support center. At last, Zoho normalized the service, but it was a bit struggle for its employees to console customers. “Disasters and software bugs are not new to technology companies. They must anticipate these disasters well in advance and prepare a foolproof contingency plan,” says Abhishek Gupta, an expert in BCP and Disaster Recovery in Bangalore.

The effects of a lack of planning can be devastating for small companies, say analysts. As many as 40 to 60 percent of small businesses never recover after being hit by natural disasters, according to the Federal Emergency Management Agency. Considering the  2015 Travelers Business Risk Index, only 21 percent of small businesses have continuity plans.

“Some firms can survive service outages for days on end, but firms in some industries cannot survive a outage even for a minute. Banks and stock exchanges will lose millions of dollars even though they suffer an outage for few minutes,” Gupta added.

Thanks to strong contingency plans, most of the large technology firms escaped from the ill-effects of Chennai floods. “We activated the Business Continuity Plan and other centers took over the jobs we handled. This is a usual practice during emergencies,” said Ravi Viswanathan, head of Chennai operations at TCS, according to The Hindu Business Line.

Sign up for our Nearshore Americas newsletter:

When smaller enterprises get caught in a disaster, they should immediately email customers and use a work-from-home policy for its employees, suggest some analysts. “They typically rely on a single location, which makes them especially vulnerable. Even relying on insurance coverage to replace lost income cannot replace clients,” said manjunath Hegde, a independent IT analyst in Bangalore.

Small enterprises can no longer ignore business continuity plans. Some big companies now even require small businesses to have a continuity plan in place before they will work with them.

 

Tags