Two Opposing Views of EHR: InformaticMD vs. NextGen's Holder of "American Medical Informatics Certification for Health Information Technology"

The AMA's publication American Medical News recently quoted me following comments from IOM EHR Safety committee member Richard Cook in the Nov. 21, 2011 article "IOM calls for monitoring and probe of health IT hazards" by Kevin O'Reilly:

... Not everyone on the panel agreed with delaying FDA regulation. [Per the IOM report on health IT safety released Nov. 10, 2011, see here - ed.]

Committee member Richard I. Cook, MD, filed a dissent in the report in which he recommended that health IT systems be regulated as class III medical devices.

"It is quite remarkable that we're in this situation," said Dr. Cook, associate professor of anesthesia and critical care at the University of Chicago Pritzker School of Medicine. [Also, an expert in Medical Informatics - ed.] "It's not surprising that such adverse events are being found related to health IT, and it's not surprising that those promoting these systems have neither looked for them nor anticipated them. To make large-scale investments in these systems and only now be looking at the impact on patient safety borders on recklessness."

Scot M. Silverstein, MD, agreed.

"The bone I have to pick with the IOM report is that the action agenda is weak," said Dr. Silverstein, a consultant in medical informatics at the Drexel University College of Information Science and Technology in Pennsylvania.

It is unethical to expand health IT so dramatically without understanding the precise nature of the risks it poses to patients, Dr. Silverstein said.

Ironically, right below my statement was the following from HIT industry figure Charles Jarvis, blaming the user:

Users faulted

Leaders in the health IT industry also had their share of objections to some of the IOM panel's conclusions.

"We don't think there's a great deal of data to substantiate that there are major safety problems with the majority of electronic health records systems in use today," said Charlie Jarvis, executive committee vice chair of the EHR Assn., a trade group that represents 46 organizations that supply most of the EMR systems implemented in medical practices. "These products are safe, dependable, time-tested and display a lot of the safety features we think are necessary to prevent problems going forward."

Jarvis, also a vice president at the health IT firm NextGen, said vendors and the government should work to help physicians and other health professional users understand systems, take advantage of their safety features and avoid errors.

[Charitable translation: computers are infallible, so medical errors due to HIT are the user's fault, the Sept. 2011 National Institute of Standards and Technology (NIST) report on usability be damned. Clinicians should spend their valuable time learning to compensate for and then actually wading through mission hostile user experiences. If only those stupid doctors and nurses would use our cybernetic miracle tools the way we want, the members of the EHR Association could be making even more money. Oh, and by the way, the NIST's concept of "use error" [1] is nonsense. - ed.]

I presume the "EHR Assn." is the HIMSS EHR Association, with HIMSS itself being a gargantuan "cause-based, not-for-profit organization exclusively focused on providing global leadership for the optimal use of information technology and management systems for the betterment of healthcare":

The HIMSS Electronic Health Record (EHR) Association is a trade association of Electronic Health Record (EHR) companies, addressing national efforts to create interoperable EHRs in hospital and ambulatory care settings. The EHR Association operates on the premise that the rapid, widespread adoption of EHRs will help improve the quality of patient care as well as the productivity and sustainability of the healthcare system.

I observe that there are no conflicts of interest here that could cause Mr. Jarvis' stated opinions to be skewed towards the rights of computers and away from the rights of patients ... right?

First, Mr. Jarvis makes a logical error related to the error illustrated in my earlier post today "Magical Thinking on Health IT from ModernMedicine.com." His error is that of "proof by lack of evidence" [2]. No need to actually study the issue rigorously, despite repeated risk management-relevant incident reports (as opposed to the industry's preferred and highly erroneous term "anecdotes").

Just one recent, highly alarming example of an "anecdote" affecting probably tens of thousands of patients due to programming malpractice and grossly negligent quality assurance, at both vendor and end user hospitals, is illustrated here. Since it's an "anecdote", perhaps Mr. Jarvis would agree there's nothing to see there, so we should all move along.

(See the Aug. 2011 post "From a Senior Clinician Down Under: Anecdotes and Medicine, We are Actually Talking About Two Different Things" which puts the misuse of the "anecdotes" label in its proper place - the garbage can.)

Not only is "proof by lack of evidence" in the face of hair-raising incident and defects reports (e.g., as in FDA's MAUDE database) a prima facie logical fallacy unfitting in medicine, and in fact alien to medical ethics, but the IOM report specifically stated in no uncertain terms that nobody really knows the magnitude of the risks. This is due in part to numerous inhibitory factors in evidence diffusion. From the IOM report:

... Several reasons health IT–related safety data are lacking include the absence of measures and a central repository (or linkages among decentralized repositories) to collect, analyze, and act on information related to safety of this technology. Another impediment to gathering safety data is contractual barriers (e.g., nondisclosure, confidentiality clauses) that can prevent users from sharing information about health IT–related adverse events. These barriers limit users’ abilities to share knowledge of risk-prone user interfaces, for instance through screenshots and descriptions of potentially unsafe processes. In addition, some vendors include language in their sales contracts and escape responsibility for errors or defects in their software (i.e., “hold harmless clauses”). The committee believes these types of contractual restrictions limit transparency, which significantly contributes to the gaps in knowledge of health IT–related patient safety risks. These barriers to generating evidence pose unacceptable risks to safety.

Imagine such a situation in, say, the pharmaceutical, automotive, aviation, or nuclear power industries. The responsible individuals would likely be hauled off to jail.

However, this all may be irrelevant. After all, who can argue with the expert personal opinion of someone who holds "the American Medical Informatics Certification for Health Information Technology?" That astonishing credential could conceivably elevate Mr. Jarvis' opinion over all others - even mine, with my meager background in the domain.

I don't know if he still claims that credential, but he did as I described in my post about prior interactions with Mr. Jarvis and NextGen (dating back to 2004) in my Feb. 2009 post "NextGen and Vendor/Doctor Dialog: Yet Another Patronizing EHR Company of Certified HIT Experts?"

I guess the fact I'd never heard of such a qualification represents my dearth of familiarity with the field of Medical Informatics and healthcare information technology.

-- SS

Notes:

[1] “Use error” is a term used very specifically by NIST to refer to user interface designs that will engender users to make errors of commission or omission. It is true that users do make errors, but many errors are due not to user error per se but due to designs that are flawed, e.g., poorly written messaging, misuse of color-coding conventions, omission of information, etc. From "NISTIR 7804: Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records." It is available at http://www.nist.gov/healthcare/usability/upload/Draft_EUP_09_28_11.pdf (PDF).

[2] Example of proof by lack of evidence, courtesy Scott Adams: "I've never seen you drunk, so you must be one of those Amish people. "

More on these issues is at the site "Contemporary Issues in Medical Informatics: Common Examples of Healthcare Information Technology Difficulties."

BUY IT NOW!

NIST on the EHR Mission Hostile User Experience: Blame the User? Nyet...

I have often had to respond to those who claim that EHR's don't cause medical errors, users do. That subset of the health IT irrationally exuberant seem common in the health IT industry and pundit channels.

NIST (The U.S. National Institute of Standards and Technology) has recently issued a draft report "NISTIR 7804: Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records." It is available at http://www.nist.gov/healthcare/usability/upload/Draft_EUP_09_28_11.pdf (PDF).

The report is quite negative concerning the low usability of today's commercial health IT, and recommends significant improvements that have been standard practice in other mission- and life-critical IT sectors for decades.

I will have more to write about this report, but I found a passage and footnote early in the report striking.

Of note, from the new NIST draft report:

This passage, from page 10:

The EUP (EHR usability protocol) emphasis should be on ensuring that necessary and sufficient usability validation and remediation has been conducted so that use error [3] is minimized.

[3] “Use error” is a term used very specifically to refer to user interface designs that will engender users to make errors of commission or omission. It is true that users do make errors, but many errors are due not to user error per se but due to designs that are flawed, e.g., poorly written messaging [or lack of messaging, e.g., no warnings of potentially dangerous actions - ed.], misuse of color-coding conventions, omission of information, etc.

This passage describes what I have termed a mission hostile user experience.

"Blame the user" as the default, reflex reply to clinical IT-related medical errors, and the "hold vendor harmless for defects" clauses that facilitate this excuse are now heading to the junkpile - in the clinic, hospital, and courtroom.

It is my hope that the the Wild West, free-for-all, cavalier, get out of jail free days of the health IT industry are coming to a close.

Hint to health IT industry: you may actually need to invest in people who know what they're doing, instead of hiring the cheapest labor possible. (See, for example, my Aug. 2010 post
"EPIC's outrageous recommendations on healthcare IT project staffing" for more on that issue.)

-- SS

Oct. 25, 2011 Addendum:

This comment (#5, October 25th, 2011 at 12:38 pm) by "a practicing front-end designer and application developer" over at HIStalk is quite interesting. The writer points out the differences between designers and developers, and opines that:

If you’re going to improve vendor design, it has to begin with an internal commitment to value designers and what they contribute to the product development process. They can not be an afterthought. There are very few companies with this mindset. Almost all HIT companies are developer-driven, so the first thought, and one that is promoted in this report, is to turn developers into designers. This will not work! A developer and a designer require two fundamentally different skill sets that are not easily transposed. Developers are trained to think rationally and analytically; design requires empathy for the user.

In health IT, empathy for the patient as well, I might add.

-- SS

BUY IT NOW!

NIST Provides Healthcare IT Industry with Remedial Undergraduate Computer Science Education

The National Institute of Standards & Technology (NIST) has published a guide entitled:

NIST Guide to the Processes Approach for Improving the Usability of Electronic Health Records

It is available free at this link in PDF: http://www.nist.gov/itl/hit/upload/Guide_Final_Publication_Version.pdf (hat tip to an AMIA colleague for posting the URL on an AMIA mailing list.)

The NIST was commissioned by HHS/ONC to study Health IT issues such as usability and report on them.

I find the publication both welcome, and pitiable.

As I started to read ch. 6, for example, I observed material that is suitable for undergraduate computer science instruction:

6. User-Centered Design Process in EHRs

User-centered design is a bedrock principle for creating usable systems and devices. [You don't say? - ed.] One of the most common reasons why systems are poorly designed is that designers and developers fail to engage users in appropriate ways at appropriate times. [Hear that, my young Paduan learners? - ed.] At its core UCD is a process that relies on systematic understanding of users and their environments, and iterative design and testing based on user performance objectives. (Details on usability testing are provided in Section 9.)

UCD has been shown to be effective in many fields. In aviation, for example, this method has been used to develop cockpit navigation displays for low-visibility surface operations. [22] By taking the limitations and capabilities of the flight crew into account, navigation errors have decreased by almost 100%. The adoption of UCD has also been shown to be effective in the design of personal computers. When working on a redesign of the laptop computer, a UCD process was employed. Users were asked to offer feedback about the current model and to offer input about ways to improve the current design. User-centered design was successful in increasing market share, brand equity, and customer satisfaction. [23] In fact, user-centered design has been elevated to an ISO standard. [24] UCD serves to engineer improved human performance into a system or device, and has been crystallizing for several decades as a design philosophy. [25]

While there is no singular model of UCD, the instantiations embody the following principles:

• Understand user needs, workflows and work environments
• Engage users early and often
  
• Set user performance objectives
• Design the user interface from known human behavior principles and familiar user interface models
• Conduct usability tests to measure how well the interface meets user needs
• Adapt the design and iteratively test with users until performance objectives are met

[HHS needs ONC to commission NIST to provide schooling for the HIT industry on these bons mots? - ed.]

As an iterative process, UCD is a cycle that serves to continually improve the application. For each iteration, critical points and issues are uncovered which can be improved upon and implemented in subsequent releases. An illustration of the UCD process is included in Figure 1.

"User centered design process in EHR's." Undergraduate-level computer science 101 instruction from NIST for the healthcare IT industry? (click to enlarge)

Here is Figure 1:

User centered design 101 (click to enlarge)

I might even have used this in teaching high school students about computer programming.

Readers can download the entire report at the above URL.

I find this publication, or, rather, the need for it to exist at all in 2010, remarkable. Absurd and an embarrassment, in fact. Master of the Obvious [1] material that apparently was not so obvious to this industry.

This is after all a multi-billion dollar industry making claims its products will "revolutionize medicine" and other exceptional claims (but without exceptional evidence). These claims have been pushed so hard that many tens of billions of dollars (with penalties) have been earmarked to either entice -- or coerce -- physicians and hospitals to use these products.

What has this industry, including vendors and highly paid management consultants and contractors, been doing, exactly, for the past thirty+ years?

What have been their product design and development practices, such that leaders of their own trade group HIMSS (as I pointed out in other posts) opine we should be "patient" for them to figure it all out about how to do health IT better and they need more time, and that the technology does not support its users properly due to lack of efficiency and usability of EMRs currently available? (As at my July 2010 post "The National Program for Healthcare IT in the U.S., and the Elephant in the Living Room".)

That HHS needs NIST to provide undergraduate level remedial teaching to the multibillion dollar health IT industry is a very poignant commentary indeed on the priorities of that industry regarding engineering rigor, talent management, attention to safety, and other factors affecting human lives.

These observations speak strongly to the need for regulation for this industry, for a talent management and trade association shakeup of major proportions, and most especially for an awakening of our government servants to exactly what the real situation is with respect to HIT on the ground.

12/14/10 addendum: it struck me that the Guide might need another chapter. I suggest a chapter entitled:

"Listening to informatics experts when they say your product will kill people, instead of firing them."

-- SS

[1] This was another pithy line from my early medical mentor, pioneering cardiothoracic surgeon Victor Satinsky, MD at Hahnemann Medical College.

BUY IT NOW!

The National Program for Healthcare IT in the U.S., and the Elephant in the Living Room

The National Institute of Standards and Technology (NIST) has begun to address deficient clinical IT usability. A PDF with presentations on the topic from the recent NIST conference on HIT usability is here (warning: very large, 26 MB).

There is a critical "meta-issue" that's being ignored regarding usability, though, yet it is the elephant in the living room.

First, I will detail the elephant, then ask the simple, logical question that arises (the "inconvenient" question that nobody seems to be able to give a straight, non-marketing-spin answer to).

Here are the details of the elephant.

First, poor usability ---> increased risk to patients.

This is a first principle; it is not open to debate.

Now:

If NIST is just now getting involved in "improving HIT usability" (the improvement of which should have occurred at least two decades ago);

While HIMSS's former Chairman of the Board admits the technology remains experimental:

... We’re still learning, in healthcare, about that user interface. We’re still learning about how to put the applications together in a clinical workflow that’s going to be valuable to the patients and to the people who are providing care. Let’s be patient. Let’s give them a chance to figure out the right way to do this. Let’s give the application providers an opportunity to make this better;

While HIMSS itself admits in this 2009 PDF that

"Electronic medical record (EMR)!adoption rates have been slower than expected in the United States, especially in comparison to other industry sectors and other developed countries. A key reason, aside from initial costs and lost productivity during EMR implementation, is lack of efficiency and usability of EMRs currently available";

While the National Research Council (the highest scientific authority in the U.S.) last year reported that :

"Current Approaches to U.S. Health Care Information Technology are Insufficient" and that the technology "does not support clinicians' cognitive needs." The study was chaired by Medical Informatics pioneers Octo Barnett (Harvard/MGH) and William Stead (Vanderbilt);

While it's not just the user experience that's the problem, either...

Insurers are starting to recognize this, e.g., "NORCAL Mutual Insurance Company: "Electronic Health Records: Recognizing and Managing the Risks" ;

While hospitals and vendors cannot yet manage the technology reliably - how many medical mistakes have/will occur as a result of screw ups like this one, now confirmed to have occurred at a religious-denomination hospital chain headquartered in the Great Lakes region of the U.S.?

This patient won't get a second chance, either.

The above issues are the elephant in the living room. Or, shall I say, in the Boardrooms and meeting rooms where health IT is planned and discussed?

Health IT is great stuff, guys; it might actually work well one day!
Let's roll it out nationally and penalize those Luddite doctors
who refuse to "use it meaningfully" because it's not very usable.
Oh, just ignore that strange creature over there in the corner ...

Considering the size and weight of the elephant, here is my question:

Why are we rolling out this technology nationally under penalty of Medicare garnishment?

I cannot get a straight, unspun answer to that question.

Perhaps we need Bill O'Reilly to ask these questions of health IT officials on his FOX News program, The O'Reilly Factor, where spin is attacked relentlessly (the "No Spin Zone.")

-- SS

BUY IT NOW!