"24", This Computer Project Was Not

A fascinating case study of IT failure in another life-critical domain has come to my attention.

I think if the words "FBI" were replaced with "hospitals" and "healthcare IT", this could be a study of IT failure in the latter organizations. Many of the familiar issues are there:

The Failure of Virtual Case File and the FBI

Background

The FBI first sought to embrace technology in the 1980s during the onset of computer availability and hoped to have a paperless office where agents could quickly pull up case files, information, and photographs at the comfort of their desks without having to sift and sort through numerous paper files. The infrastructure in that time was limited to text based search engines and there were no provisions for photo storage or the ability to scan written reports. As a result, the FBI found its agents decided not to rely on the existing technology and were reverting back to paper.

After the attacks on September 11, 2001, the FBI was placed under scrutiny for being ineffective and inefficient in its operations due to the time it would take to share information with other law enforcement agencies, locate reports, and transmit them from one location to another (usually done via fax or by mailed CDs). To combat this, the FBI developed a plan known as Trilogy, which aimed for three primary goals: A new computer network, personal computers for most agents, and an online criminal database that would be titled Virtual Case File (VCF). An external contractor by the name of Science Applications International Corp. (SAIC) was contracted in June 2001 to begin the project with an estimated schedule of three years for completion and a first year budget of $14 million. The project continued until early 2005 (7 months over schedule), at which time the project scope had expanded by 80% with costs of $170 million and was riddled with issues. Ultimately, in early 2005, the project was cancelled but not after escalation and persistence on the part of the FBI.

The Problems and Failure of VCF

The FBI wanted SAIC to create the database from scratch instead of using off-the shelf Oracle programs that could have been customized. A study by the National Research Council (NRC) after the planned 3-year period in late 2004 was conducted to gauge the success of the program, and while the first two goals had been achieved (personal computers for most agents and the creation of a new network), VCF was found to be problematic and incomplete. The project plan was incomplete and there were no monitoring controls regarding finances or the schedule. The FBI threw quality out the window by wanting to bypass testing and release the product upon its ready date.

However, VCF failed the most basic functionality tests under the NRC and had not included network management, security, and storage systems, or basic sorting capabilities. The study also found that most of the FBI's skilled managers had left for the private sector and there were little to no individuals who had the IT experience or knowledge to interact effectively with contractors to achieve what was needed. The definition and scope of operations and processes were ultimately entrusted to SAIC who were outsiders.

In an attempt to salvage the project, the FBI immediately hired a federally funded R&D firm (Aerospace Corp.) costing $2 million to conduct an assessment of the project who concluded that the project needed to be scrapped and shut down due to the severity of the software issues. Upon investigation by Congress, there was a lack of financial controls and safeguards on the part of the FBI, enabling SAIC to continue to develop a program which was lacklustre and failed to meet objectives.

200 programmers from SAIC were used on the project when only a dozen were required and SAIC was not being properly monitored by the FBI. They felt as long as money was being funnelled to them by the government on the project, they did not need to be responsible for the effectiveness or viability of the program [was there a "hold harmless" clause as in health IT? - ed.] they were building and fired staff who expressed concerns over the direction of the program. [They must have been Luddites and IT skeptics who just refused to change the way they do things - ed.]

Further, the FBI took a trial by error approach to the project without truly understanding their end goal and without setting benchmarks for evaluating the progress of the project and took a nearly hands off approach by entrusting SAIC entirely. SAIC claimed the FBI were indecisive in what they wanted and there had been 19 government personnel changes over the project tenure which brought on scope creep and the focus of the project in a state of flux, in addition to a clear lack of leadership. A further $17 million was then spent by the FBI to perform more rigorous testing to try to salvage the project once more, which was another missed opportunity to cancel the project. It was only in early 2005 that the decision was made by Congress to terminate the project.

Source: Eggen, Dan; Witte, Griff. 'The FBI Upgrade that wasn't'. August 8, 2006. Website: http://www.washingtonpost.com/wp-dyn/content/article/2006/08/17/AR2006081701485.html (accessed on November 7, 2009).

"24" this was not.

Chloe O'Brian, where were you?

In the FBI's case, the failure exposes us to potential crime. In a hospital's case, the stakes are even more personal.

Even worse, at least here Congress intervened; health IT is a virtually unregulated industry and nobody is minding the store.

The UK's National Programme for IT (NPfIT) in the NHS paid the price of failure through problems such as above.

Will the "NPfIT in the HHS" meet a similar fate?

-- SS

Oct. 13, 2011 Addendum: where have we seen SAIC before?

How about here?

Case 3: Bedlam

Meanwhile, Science Applications International Corporation disclosed that computer backup tapes containing medical data for 4.9 million military patients [that number also amounts to almost 2% of the total U.S. population - ed.] had been stolen from an employee’s car in San Antonio. The data included Social Security numbers, clinical notes, laboratory test results and prescriptions.

-- SS

BUY IT NOW!

Still More Electronic Medical Data Chaos, Pandemonium, Bedlam, Tumult and Maelstrom: But Don't Worry, Your Data is Secure

Case 1. Tumult

October 5, 2011
New York Times
Patient Data Landed Online After a Series of Missteps

By KEVIN SACK

Private medical data for nearly 20,000 emergency room patients at California’s prestigious Stanford Hospital were exposed to public view for nearly a year because a billing contractor’s marketing agent sent the electronic spreadsheet to a job prospect as part of a skills test, the hospital and contractors confirmed this week. The applicant then sought help by unwittingly posting the confidential data on a tutoring Web site. [Got all that? - ed.]

In an e-mail sent to a victim of the breach, the billing contractor, Joe Anthony Reyna, president of Multi-Specialty Collection Services in Los Angeles, explained that his marketing vendor, Frank Corcino, had received the data directly from Stanford Hospital, converted it to a new spreadsheet and then forwarded it to a woman he was considering for a short-term job.

The position was with Mr. Corcino’s one-man shop, Corcino & Associates, Mr. Reyna wrote in the e-mail, which was authenticated by his lawyer, Ellyn L. Sternfield. The job applicant apparently was challenged to convert the spreadsheet — which included names, admission dates, diagnosis codes and billing charges — into a bar graph and charts, Stanford Hospital officials said.

Not knowing that she had been given real patient data, the applicant posted it as an attachment to a request for help on studentoffortune.com [I wrote about that earlier here - ed.], which allows students to solicit paid assistance with their work. First posted on Sept. 9, 2010, the spreadsheet remained on the site until a patient discovered it on Aug. 22 and notified Stanford.

My, how electronic data can travel when mishandled. Try that trick with 20,000 paper charts ...

The hospital, located on the campus of Stanford University in Palo Alto, demanded that the spreadsheet be removed, and the Web site quickly complied. Pressed for time, the job prospect wound up completing the assignment herself and, in the end, did not get hired, Ms. Sternfield said.

Ironically, this was all for naught.

Mr. Corcino, in his first public statement, attributed the breach to “a chain of mistakes which are far too easy to make when handling electronic data.”

Far too easy to make - especially by the dyscompetent.

... Breaches of private medical data have become distressingly commonplace, with two substantial ones disclosed in the last week alone. [We don't know the details of those yet; that's for next week - ed.]

Case 2: Pandemonium (from same NYT article)

In Orlando, officials with Florida Hospital reported that three employees had improperly combed through emergency department records of 2,252 patients, apparently to forward information about accident victims to lawyers. The employees were fired, and law enforcement officials are investigating.

Trolling for Torts - is this a new EMR TV game contestant show? Perhaps it could be followed by "Trolling for Tarts?"


Case 3: Bedlam (from the same NYT article)

Meanwhile, Science Applications International Corporation disclosed that computer backup tapes containing medical data for 4.9 million military patients [that number also amounts to almost 2% of the total U.S. population - ed.] had been stolen from an employee’s car in San Antonio. The data included Social Security numbers, clinical notes, laboratory test results and prescriptions. The company said the risk of harm was low because retrieving data from the tapes would require specialized knowledge, software and hardware. [Who's to say the theft was not by someone with that specialization, or someone paid by same to steal the tapes? - ed.]

The Texas breach is by far the largest since September 2009, when a new federal law began requiring disclosures of medical privacy violations involving at least 500 people. Some 330 such episodes have been tallied, including four others that affected more than one million people each.

We'd all be buried in stray clinical paper by now if it weren't for computers. Thank god for them!

Officials at the Department of Health and Human Services said the new reporting requirements had exposed deep vulnerabilities and encouraged renewed vigilance.

Exposed to whom? The blind, deaf and dumb?

“We’re moving in the right direction in terms of a culture of compliance,” said Leon Rodriguez, director of the department’s Office for Civil Rights, which investigates medical privacy cases. “Are there still a lot of problems out there? Yeah, my sense is there are still a lot of problems.”

The Titanic was moving in the right direction - towards New York Harbor, in fact, when it met a little unexpected obstacle. Perhaps a culture of brains would be better than a culture of compliance...

The Stanford breach was notable for the duration of public exposure, and for spotlighting the vulnerability created by a medical provider’s business relationships with outside parties.

Last week, lawyers filed suit in state court in Los Angeles, seeking certification as a class action and $20 million in damages from Stanford Hospital & Clinics and Multi-Specialty Collection Services, which is known as MSCS.

$20 million might hurt a bit, and might help motivate the organization to hire better and/or more appropriate clinical information management expertise - in house where it belongs (see below).

The threat of liability set off a predictable round of finger-pointing.

In written responses to questions, Lisa Lapin, Stanford University’s assistant vice president for university communications, said, “MSCS bears the complete and sole responsibility for the breach.”

It's their fault, not ours.

Ms. Lapin said the hospital had sent the data in encrypted form to Mr. Corcino, who requested it on behalf of MSCS to analyze a strategy for improving billing collections. She said Mr. Corcino had regularly represented himself as MSCS’s executive vice president and had been Stanford’s “primary contact” during a seven-year relationship. MSCS, a five-person firm that audits hospital accounts to maximize reimbursement, possessed the passwords to unencrypt the data, she said.

It was all about money and outsourcing.

“This mishandling of private patient information was in complete contravention of the law and of the requirements of MSCS’s contract and is shockingly irresponsible,” the hospital said in a statement.

It is foolish to believe that someone else can run critical aspects of your business, and it is even more foolish to believe that it is OK for someone else to run critical aspects of your business.

Ms. Sternfield, Mr. Reyna’s lawyer, said Mr. Corcino had never been an MSCS employee, but rather was paid a monthly fee to drum up business, typically in face-to-face meetings with health care executives. Mr. Reyna, she said, had no knowledge that the Stanford data had been sent to Mr. Corcino, or that he had passed it on.

Mr. Corcino was not authorized to use an MSCS title, Ms. Sternfield said, but she declined to say whether Mr. Reyna was aware of the practice. She acknowledged that Mr. Corcino sometimes used an MSCS e-mail account.

In his e-mail to the breach victim, who shared it with The Times, Mr. Reyna wrote that Stanford had sent the file to Mr. Corcino “for a potential MSCS project that would audit paid accounts to verify that the reimbursement was correct.”

For his part, Mr. Corcino said in a statement that he was an independent contractor but was “the marketing face of the company,” and that MSCS “allowed me to use the title of executive vice president.” He wrote: “Stanford sent the file to me at MSCS, and I imported the data into a spreadsheet that was forwarded to the job applicant as part of a skills test. I did not intend to provide any personal health information in the file. This was a marketing project.”

Without explaining how or why he sent the data to the applicant, Mr. Corcino said MSCS had not trained him properly and faulted Stanford for sending him private information that he did not need. That, he said, was the “first link in a chain of mistakes.”

“I regret that Stanford released a file containing unnecessary information,” Mr. Corcino said, “that MSCS did not have an appropriate training and audit system for the handling of electronic data and that I was not more careful with the file. While Stanford and MSCS left the information in the file I received, it was my mistake to not catch its inclusion and remove the data.” ... The hospital has terminated its relationship with MSCS, and Mr. Reyna has done the same with Mr. Corcino.

Even I can't follow all that. This will be one convoluted court case...

Stanford Hospital has reassured affected patients that the posted spreadsheet did not contain Social Security numbers, birthdates or credit card numbers, and has offered free identity theft protection services. The hospital said it had not uncovered any misuse of the exposed data.

Yet, that is. (Is it no wonder that sedatives are among the most highly-prescribed medications?)

Moving from the NYT article:

Case 4: Tumult (I'm running out of descriptors)

A large class action lawsuit again Health Net and IBM:

California Legal
Westlaw Journal Insurance Coverage

Health Net’s, IBM’s negligence compromised medical data, suit says

June 7 (Westlaw Journals) - Health Net Inc. and IBM face a class-action lawsuit seeking $5 million in damages over the loss of computer storage devices that held the medical histories, financial data and Social Security numbers of 2 million people.

Health Net Policyholder Alana Bournas’ class-action complaint in the U.S. District Court for the Eastern District of California alleges that the insurer and IBM breached their duty of confidentiality and negligently allowed the release of highly personal and confidential information of millions of Health Net employees and policyholders.

The complaint alleges violation of California’s Confidentiality of Medical Information Act, Cal. Civ. Code § 56; Cal. Civ. Code § 1798.2, which concerns the unauthorized disclosure of customer records; Cal. Bus. & Prof. Code § 17200, the state’s unfair-competition law; and public disclosure of private facts.

Companies will either pay the going price for competent employees, or pay for the mistakes of incompetent ones. It would probably be better for society, however, to do the former habitually.

The suit says IBM agreed to manage Health Net’s information technology database for five years beginning in 2008.

IBM informed Health Net Jan. 21 that it had lost nine disk drives containing the confidential information of 2 million people, including Health Net policyholders and employees.

Health Net failed to alert the victims of the breach until March 14, the complaint says.

IBM allegedly also failed to encrypt the data, thereby enabling anyone who possesses the hard drives to easily access the confidential information. This puts the victims at an increased risk of identity theft and “other unauthorized uses of plaintiff and class members’ personal information” the suit says.

Encryption, a feature now built into mainstream OS's by Microsoft and Apple? (Oh wait...IBM...)

Health Net’s attempt to compensate the victims by providing two years of free credit monitoring services through TransUnion is an inadequate remedy for the defendant’s conduct, Bournas says. This “remedy” fails to address unauthorized disclosures of medical information, and the monitoring services only protect against new account fraud but do not address fraudulent activity with existing accounts, the suit says.

These executives apparently can't even get the fix straight.

Moreover, the complaint says, Health Net has previously been accused of a similar breach of confidential information. In 2009 it lost the same types of records of nearly 1.5 million people and waited six months before notifying the victims. In settling the state of Connecticut’s lawsuit stemming from that security breach, the company promised “to enhance security procedures and training,” the suit says.

What can I say?

The current breach could have been avoided had Health Net and IBM taken proper precautions and implemented security policies to maintain consumers’ confidential data, according to Bournas. Therefore, the protections granted under California law require that Health Net be penalized for its negligence, she says.

The plaintiff notes that millions of people entrusted Health Net with their private data.

“At best, defendants’ actions allowed this private information to go astray. At worst, the private information is being viewed, sold, resold, and used for illegitimate and illegal purposes,” the complaint says.

The suit is seeking injunctive relief, compensatory damages, declaratory relief, and attorney fees and costs.

Bournas v. Health Net Inc., No. 2_11-CV-01262, complaint filed (E.D. Cal. May 11, 2011).

I would revise that to say "The current breach could have been avoided had Health Net and IBM hired personnel in adequate numbers with the qualifications and true gravitas (and not laid them off, of course) to maintain consumers’ confidential data."

Case 5: Maelstrom (I am reaching to the bottom of the barrel for such descriptors).

Wellpoint recently settled class-action suit in CA.

AMA news
By Pamela Lewis Dolan, amednews staff.
Posted Aug. 1, 2011.

WellPoint reaches tentative accord in data breach suit

It is the second settlement to come from lawsuits claiming that the company failed to protect the privacy of individual insurance applicants online.

WellPoint has reached a preliminary settlement that will, if approved, bring an end to a class-action lawsuit filed more than a year ago.

The lawsuit, filed in the Superior Court of the State of California, involves the potential exposure of data belonging to more than 600,000 individual health insurance applicants on a company-run website that allowed insurance applicants to track their applications.

The situation came to light when an applicant to WellPoint-owned Anthem Blue Cross of California sued the company in March 2010. The applicant was able to manipulate the web address within the site to gain access to other applicants' information, including names, addresses, dates of birth, Social Security numbers and health and financial information.

In other words, probably changing a simple number in the URL brought up someone else's records. Good going there, Wellpoint. What were the programmers thinking? (Were they thinking?)

When the suit was filed, the company said an upgrade to the system caused the information to become exposed. The company said a third-party vendor validated that all security measures were in place when, in fact, they were not. Changes were made to the system soon after the situation was discovered.

Blame someone else, yet again.

In addition to the class-action suit, the company was sued by Indiana Attorney General Greg Zoeller in July 2010. The suit, filed in Marion County Civil Superior Court, alleged that the company violated the Indiana Disclosure of Security Breach Act by failing to notify Zoeller, and the 32,051 Indiana residents affected by the incident, in a timely manner. That suit was settled in early July, when WellPoint agreed to pay a $100,000 fine. As part of the settlement, WellPoint admitted it had a security breach and failed to properly notify the attorney general's office as required by law.

Gevalt.

Under the preliminary settlement in the California class-action matter, WellPoint agreed to offer credit monitoring for two years to all affected individuals. Class members are eligible to receive reimbursement for identity theft losses of up to $50,000 per incident, as well as additional time to file identity theft claims until May 31, 2016. Those making identity theft claims are eligible for an additional five years of credit monitoring. The company also will donate a total of $250,000 to two nonprofit organizations whose efforts are directed at protecting consumers' privacy on the Internet.

It might have been cheaper and better for goodwill not to outsource a vital function...those third-party vendors can really hurt you. (I'd really like to know - was this "third party vendor" domestic, or overseas?)

WellPoint did not admit wrongdoing in the case, nor was it found guilty. A fairness hearing is scheduled for November, and the courts then will decide whether to approve the settlement.

Large corporations are immune from such formalities as admitting wrongdoing or being found guilty.

-----------------------

But don't worry. Your medical data's safe.

Sort of. See also:

• "Networked EMR's and Healthcare Information Security: Practical When Massive IT Security Breaches Continue?",

• "Networked, Interoperable, Secure National Medical Records a Castle in the Sky?",

• "Operation Aurora And a Widespread Reluctance to Discuss IT Flaws: Is Universal Healthcare IT Really a Good Idea in 2010?",

• "Medical data breach of the week - but your EMR data is secure, trust us, we're IT experts",

• "Insurers Test Data Profiles to Identify Risky Clients",

• and others

-- SS

BUY IT NOW!