"This massive monster of incomprehensibility"
« previous post | next post »
Atul Gawande, "Why doctors hate their computers", 11/5/2018, underlines the often-noted difficulty of working with badly-designed software:
I’ve come to feel that a system that promised to increase my mastery over my work has, instead, increased my work’s mastery over me. I’m not the only one. A 2016 study found that physicians spent about two hours doing computer work for every hour spent face to face with a patient—whatever the brand of medical software. In the examination room, physicians devoted half of their patient time facing the screen to do electronic tasks. And these tasks were spilling over after hours.
But the most interesting part of the article, at least for me, was the discussion of reading the records rather than writing them.
The article gives an example from a primary-care physician, Susan Sadoughi:
The problem lists have become a hoarder’s stash.
“They’re long, they’re deficient, they’re redundant,” she said. “Now I come to look at a patient, I pull up the problem list, and it means nothing. I have to go read through their past notes, especially if I’m doing urgent care,” where she’s usually meeting someone for the first time. And piecing together what’s important about the patient’s history is at times actually harder than when she had to leaf through a sheaf of paper records. Doctors’ handwritten notes were brief and to the point. With computers, however, the shortcut is to paste in whole blocks of information—an entire two-page imaging report, say—rather than selecting the relevant details. The next doctor must hunt through several pages to find what really matters. Multiply that by twenty-some patients a day, and you can see Sadoughi’s problem.
The software “has created this massive monster of incomprehensibility,” she said, her voice rising. Before she even sets eyes upon a patient, she is already squeezed for time.
A physician friend once told me that he used to be able to tell a lot about a new patient with a glance at their folder, even before reading it — how thick it was, how many sheets of which color were in it (because different departments used different colors of paper). It took him a long time to get nearly to the same level of efficiency in dealing with the digital version. Electronic health records have been promoted as a boon to epidemiology, but this remains problematic, partly because of the "massive monster" problem, but also because of the many different data models (and different cultures for using them) in different practices, clinics, hospitals and programs. As a result, "understanding" a patient's digital records is actually a difficult AI problem — and of course the massive data sets required for modern machine learning solutions are not easily available.
Dr. Sadoughi also complains about redundancies in the electronic input process, which has apparently gotten worse rather than better with successive iterations:
“Ordering a mammogram used to be one click,” she said. “Now I spend three extra clicks to put in a diagnosis. When I do a Pap smear, I have eleven clicks. It’s ‘Oh, who did it?’ Why not, by default, think that I did it?” She was almost shouting now. “I’m the one putting the order in. Why is it asking me what date, if the patient is in the office today? When do you think this actually happened? It is incredible!” The Revenge of the Ancillaries, I thought.
Similar problems certainly exist in many non-medical domains. The fault seems to be partly what Geoff Pullum has called "Nerdview": "people with any kind of technical knowledge of a domain tend to get hopelessly (and unwittingly) stuck in a frame of reference that relates to their view of the issue, and their trade's technical parlance, not that of the ordinary humans with whom they so signally fail to engage." In addition, the people who write the programs in question get no effective training in HCI ("Human-Computer Interaction"), and are not disposed by nature to think about their task in those terms. And finally, their bosses (and their bosses' committees) are very sensitive to problems of missing data, development costs, etc. — but not to the problem of making the system work easily and efficiently for users. (Or even for other computer programs.)
So as soon as I get used to the oddities of one of the University computer systems I need to use regularly, there's a new version with new design disasters to learn to cope with. I recognize that the old-fashioned paper-based systems had their own problems — but…
For a couple of examples from my own history, see "When bad interaction happens to good people", 8/15/2007 (featuring The Legend of Facility Focus), or "Annals of human-computer interface improvement", 2/17/2010. I've resisted the temptation to review the last dozen or so HCI disasters I've encountered, but it's not because things have been getting better.
ErikF said,
June 8, 2021 @ 8:38 am
HCI is as much a language as English in a lot of ways: if you don't design a program so that the impedance mismatch between the programmer/designer and user is minimized, the program will be unusable. Well, technically it is usable, but only by the people who wrote and tested the program (assuming testing happened!)
I got more training in HCI in my library science program than I did in my computer science one. Hopefully this is changing for current CS students.
Paul Topping said,
June 8, 2021 @ 9:44 am
Based on what's here, the problem seems at least partly to the need of physicians to learn to manage information rather than badly designed software. The software allows the saving of entire reports but the doctor needs to learn that it is important to also add a summary in their own words. If they forget to do that, they end up having to reassess the entire report and in a different context than when it was new. Working with computers requires a new approach to the problem and that takes time and dedication.
That's not to say that the software can't be improved. When software is introduced to a process, you often find users miss physical cues that they implicitly relied on (eg, size of paper stack, colors of sheets). These may need to be added to the software design. Often the software can do a lot better than the paper system. Obviously a thick bundle of paper sheets can be unwieldy. Software can also be user customized to tell the physician at a glance whether any test in the large stack has a gold star on it (whatever that may mean).
Phillip Helbig said,
June 8, 2021 @ 10:42 am
A train stops at a train station. A bus stops at a bus station. On my desk is a workstation.
Kristian said,
June 8, 2021 @ 11:02 am
The district around Helsinki has recently adopted a electronic health record system from Epic (the company discussed in the New Yorker article) and spent hundreds of millions of euros on it, but nearly everyone who uses it and has cared to express an opinion has criticized it (and medical records here were already electronic, so we're not talking about a transition from paper to electronic records). Denmark also had major problems with Epic.
Nothing doctors do with this software should be more complicated than making purchases online, so there's really no excuse for it's being so clumsy and counterintuitive. If an online merchant had such bad software, no one would use their website, and no one would use a smartphone app that was so badly designed, but apparently Epic has no incentive to produce a user friendly product.
Richard Hershberger said,
June 8, 2021 @ 11:52 am
I work in a personal injury law firm. I read medical records as part of my job, distilling them down to the bits that are relevant for us. The signal to noise ratio is terrible. Partly this is because we are only interested in certain things, but the copying and pasting is out of hand, often at the expense of relevant content. If the patient has been getting therapy for several weeks. surely the daily notes should reflect the patient's condition on that day, not merely at the initial examination. Sadly, this often is not the case. Then there are emergency room records. Twenty years ago a typical ER visit would result in a few pages. Nowadays even the most banal visit produces many pages, though generally not more content.
Anna in Pdx said,
June 8, 2021 @ 12:22 pm
As the caregiver for a patient with multiple health issues, who has a very good HMO, I can attest to this. His electronic chart is so overwhelming that the doctors just seemingly rely on us to explain issues.
BillR said,
June 8, 2021 @ 1:11 pm
I spent way too much of my life designing and coding end user applications in the banking, manufacturing, government, and university industries, starting in the 70s (I switched out of that stuff in 1998 because I did not want to get sucked into fixing undocumented COBOL programs in 1999 to deal with Y2K). At first it was just writing code that printed everything on paper for people who did the actual work of the enterprise, and then CRTs showed up and things started to get “interactive”. Just translating from paper to screens wasn’t gonna work, we had to start learning how people did the work they did, and this is where nerdview reared its ugly head. It wasn’t enough to be a good coder any more. We actually worked pretty hard at getting things to work the way users wanted, but it turns out (surprise!) it can be difficult for users to explain what they want. It was bad enough with text-only displays, but then the graphical UIs came along and t only got worse. Remember when the graphics started out looking like paper? That didn’t last long, did it?
The only graphical UI app I ever did was a HyperCard version of Mr. PotatoHead, and all I got for that was a threatening letter from a copyright attorney for Hasbro or Mattel (I forget which) warning me to cease and desist. The good old days…
Peter CS said,
June 8, 2021 @ 4:44 pm
This is very sad, but not surprising. I spent 50 years in the trade of specifying, designing and writing bespoke software. I always tried to start by watching and interviewing the users. I always suggested mocking up the user interface and having the users try it and suggest improvements – an iterative process, often trying several designs. I would time users doing tasks using different screen designs to see which was quicker. Even when the system was built I would ask to test it in real life and had budget held back to make any final tweaks which the users wanted. And if they were used to using color to distinguish between reports, we gave them the same colors, and so on. Of course, procurement departments hated it because they didn't know what they were getting at the outset, and many insisted on our building the system they had designed or specified, usually without much reference to the users, and then complained to us when the sort of problems outlined above arose. Doing it 'my way' – and of course it isn't just me that uses that sort of approach – generally costs more, maybe even 50% more, but those systems are much more likely to be around 20 years later when the poorly-designed ones have long since been scrapped.
Matt Sayler said,
June 8, 2021 @ 6:47 pm
"HCI is as much a language as English in a lot of ways" It's a set of languages and a set of cultures, and those concepts interact with the languages, cultures, etc. of the programmers and the users.
Making good interfaces is hard. As mentioned by several commenters, identifying and watching your users is important, but of course that takes time, effort, and creativity. You can't simply present the first thing that comes to mind, take some notes on how people use them, and send some change requests to the programmers to complete.
SteveB said,
June 8, 2021 @ 8:03 pm
When you find software with a good user interface, it is nearly certain that there is a team of dedicated designers behind it, all trained in design. Use of such teams are not profitable in medical software. This is sad, because it almost certainly has a real human cost, as bad software design can lead to bad decisions on the part of those using it.
Barbara Phillips Long said,
June 9, 2021 @ 10:43 am
On several occasions in the last couple of years I have dealt with physicians who arrive trailed by people carrying very large laptops. The doctors dictate notes during the encounter, and the staffers deal with the computer interface. Whether having someone else enter information helps with reading the charts, I don’t know.
In general, when I have problems with the usability of a computer interface, it is because the designers have built in certain inflexible assumptions. To give one example, after my husband died a decade ago, his paycheck stopped, so I wanted to halt automatic bill payments from our joint bank account. Even with a death certificate in hand, I was not allowed to discontinue any of the automatic payments he had authorized because only the person who had set them up could stop them, whether or not the account had more than one owner.
David L said,
June 9, 2021 @ 1:33 pm
@Paul Topping: As I see it, the issue seems to be that the software either doesn't allow a physician to manage data in a reasonable way, or makes it difficult and counterintuitive. You could have a system, for example, that asks a physician to summarize a patient visit in a free-form way, similar to writing notes by hand, and then allows them to attach any number of lab reports, scans, images and whatnot so that they would be accessible but not visible without a further click or two.
You say "Working with computers requires a new approach to the problem and that takes time and dedication." But that's back to front. The time and dedication should come from the software designers, whose job should be to build something that suits how physicians work and makes their lives easier.
Kristian said,
June 9, 2021 @ 2:17 pm
"You could have a system, for example, that asks a physician to summarize a patient visit in a free-form way, similar to writing notes by hand, and then allows them to attach any number of lab reports, scans, images and whatnot so that they would be accessible but not visible without a further click or two."
Yes, you could, very easily, but that is (apparently) not what the administrators/bureaucrats want. They want a system that forces physicians to record everything in a predetermined way, everything in the right field, presumably so that certain things can be drawn out from the database afterwards. So a lot of time is spent looking for that particular field, otherwise the software doesn't allow one to proceed.
A lot of this seems to be for billing purposes (esp. in America where each patient might have a different insurance policy that covers different procedures in a different way). The "sexier" explanation is that it's some kind of big data / AI thing. But even if we agree this is a reasonable goal, a lot of this software still isn't very good.
Keith said,
June 9, 2021 @ 5:33 pm
I worked for a decade in the documentation department of the software division of a French geophysics company.
The software was destined for users worldwide, there was little to no localisation; users were expected to have a working knowledge of either French or English, but much of the interactive software was provided with the interface in English only.
This wasn't a problem for drop-down menus and options, but one day, working on some software that was still in development, I found an amusing little icon depicting a stereotypical little bed-sheet ghost. I hovered the pointer over the icon and waited for the tool-tip to appear; it said "display the signal spectrum".
A little light bulb lit up in my head: the French word for "spectrum" is "spectre", which also happens to be a synonym for the more common word "fantôme", but the pun falls flat in English. I suggested replacing the little ghost with a little quarter circle rainbow.
Xmun said,
June 10, 2021 @ 1:54 am
@ Philip Hellbig
A train stops at a train station, yes, but a bus stops at a bus stop.
Xmun said,
June 10, 2021 @ 1:56 am
Sorry. Should be @ Phillip Helbig
Andrew (not the same one) said,
June 10, 2021 @ 6:55 am
A bus does also stop at a bus station (especially if ;'stop' means 'terminate').