Rothstein Home: Your Source for Disaster Recovery, Business Continuity Books, Service Level Agreements & More Rothstein: Management Consulting Services Rothstein: Business Survival Newsletter Rothstein: Original Feature Articles Rothstein: Disaster Recovery Forum Rothstein: Today's Industry News Rothstein: Links to Industry Web Sites Contact Rothstein Associates

When is a Virus More Than a Virus Disaster Recovery & Business Continuity & Contingency Planning & Disaster Prevention Bookstore
Service Level Management & Service Level Agreements Bookstore

by  Philp Jan Rothstein, FBCI

A case study of a computer virus infestation causing a
corporate disaster.

As appears in the March/April, 1996 issue of
InfoSecurity News Magazine.

A consulting client recently experienced an ugly encounter with one of the less pleasant computer viruses. What started out as a nuisance impacting a handful of non-critical workstations rapidly escalated into a near-disaster impacting mission-critical business operations. This was not a naive or ill-prepared organization: their policies, standards, controls, and tools were in place to address just such an occurrence - or so they believed.

      It all began one Friday in early autumn (isn't it interesting how these incidents always pop up just before the weekend?) when a staff person in this medium-sized service organization who was processing routine transactions noticed that his workstation was suddenly acting 'flaky.' Data elements displayed on-screen were

just plain wrong; legitimate transactions were being rejected. Thinking he had made an error, he went back over the morning's activity and saw that several transactions which had been entered and accepted by the software now looked 'strange.' He immediately set about cleaning up the odd transactions and continued where
he left off.

       By mid-day, it became clear that something ugly was still going on; he then contacted technical support who diagnosed the problem in short order as a corrupted data base and shut down that server application. The technician ran a program to clean up the data base errors and again enabled the application in time to catch a late lunch.

       By mid-afternoon, similar reports of 'flaky' applications and data were building from several departments; tensions were high. To make a long and painful story mercifully short, by some time after midnight Friday tech support had identified a virus which had somehow infested a number of computers and corrupted data files over a period of several days. The damage was compounded by continuing operation during the period before detection: the last 'clean' backups were almost a week old! Restoring and recovering lost data took place through Monday; bottom-line business losses plus direct expenses were in excess of $100,000; botched or lost customer transactions and delays damaged the company's reputation for quality service and impacted productivity for several weeks.

       Despite what was viewed as a reasonable and appropriate anti-virus program, what should have been a nuisance event rapidly escalated into an unfortunate (and entirely preventable) business disaster, despite what was perceived as an effective anti-virus policy. What went wrong?

  • Staff were lulled into believing that the anti-virus programs on their workstations took care of everything - so when a virus struck, the symptoms were not considered serious enough to even contact tech support.
  • Although a rudimentary 'virus response' procedure had been documented, nobody bothered to ever conduct a test. The good news was that this incident constituted the first stage of what is rapidly becoming an intensive testing program for incident response.
  • The escalation process was never addressed. Management was not notified until the problem had already reached crisis proportions.
  • Although backups of servers and mission-critical workstations were in place (and even faithfully stored off-site), no thought had been given to why those backups were there - that is, how they could be used to recover from specific threats such as a virus attack. As this organization learned, their backups were all but worthless in the face of damage which builds over several days.
  • Accountability had not been established for maintaining and updating the anti-virus software since it had been installed several months earlier. The particular virus was in fact addressed by a maintenance update which had not been installed.
  • Effectiveness of the anti-virus program had never been audited. An audit might have uncovered the fact that employees were routinely swapping data on diskettes to and from laptop computers which they were using for uncontrolled, non-business purposes outside the office; and, were downloading shareware from public networks without controls or monitoring.
  • It did not occur to anybody to consult, let alone activate, the MIS disaster recovery plan or the company's business continuity plan. After all, this was not really a disaster - or was it? Both of these contingency plans were vague regarding activation criteria.
  • The business continuity plan, even if it had been activated, did not effectively address crisis escalation, crisis management or crisis communication - prompt and carefully worded notification to customers and others might have averted much of the direct business impact.

May all your disasters be little ones

Copyright (c)1997-2003, Rothstein Associates Inc. All Rights Reserved.

Back to Top

Site Map | The Rothstein Catalog on Disaster Recovery | The Rothstein Catalog on Service Level Books

Contact Us | Management Consulting Services | Business Survival Newsletter | Original Feature Articles

Disaster Recovery Forum | Today's Industry News | Links to Industry Web Sites | ‘Keep Me Posted’ | Privacy Policy

 

E-mail Rothstein Associates Inc.