0:00

So let's talk about some research that has been done on,

0:03

on the topic of radiologist errors.

0:05

Um, one of the, the nicest papers, uh,

0:09

was published in a JR in 2014 by Kim and Mansfield.

0:13

And, uh, the title was Fool Me Twice.

0:15

And the reason they gave it that title was

0:17

because they found that, uh, errors were often repeated,

0:20

so the patient would come back with the same finding, uh,

0:23

that had been missed the first

0:24

time, and it was missed again.

0:25

Um, so they studied the problem

0:27

of diagnostic error in a very systematic way,

0:30

and they gave us a nice epidemiology and taxonomy of errors.

0:33

Um, and, uh, they found that, uh, there's multiple types

0:37

of error that they could, that they could point to,

0:40

although some were more important than others.

0:42

And of course, sometimes they occurred in combination too.

0:45

Um, so, uh,

0:47

656 radiological examinations of all types.

0:50

So these were plain films, ultrasound, ct, whatever, uh,

0:54

and that, that had delayed diagnosis where,

0:57

where initially the finding was missed,

0:58

but then it was picked up later.

1:00

Uh, and, uh, they were collected from July 1st, 2002

1:03

to January 31st, 2010.

1:06

So a big sample over a long period of time.

1:08

And, uh, they didn't really have a very systematic way

1:11

of collecting these, you know, whenever they were aware

1:13

of a case, they put it in the file.

1:15

Uh, and, uh, so,

1:16

and each, each of these cases was then reviewed

1:18

by two radiologists,

1:20

and they were classified according to a 12 step schema

1:23

that they had developed.

1:26

So this was another example of an effective 12 step program.

1:29

And, uh, they found that errors were perpetuated.

1:32

They said that that is repeated on subsequent exams.

1:36

Um, and here's their classification scheme,

1:39

and, uh, I won't go through this in, in,

1:41

in excruciating detail,

1:42

but I wanted to point out a couple of categories.

1:44

Uh, one, they called under reading number

1:49

category number four, and you know, the finding is mist,

1:52

and I would call that a perceptual error,

1:54

simply not seeing something.

1:57

Um, and, uh, they also had a category of number,

2:01

which they called number three, lack of knowledge.

2:03

And that is the finding was seen,

2:06

but you didn't know what it was.

2:07

You didn't recognize it because you had some,

2:09

some knowledge gap.

2:11

And, um, you know, they had things like number nine,

2:15

you know, the finding was not in

2:17

a location that you expected.

2:19

And, um, uh, number 11, uh,

2:23

complication from a procedure.

2:25

Um, number seven, the failure

2:28

to consult an old study, which might have helped.

2:30

And number 12, they called satisfaction of report.

2:34

And number 10, satisfaction of search.

2:36

We've all heard of satisfaction of search,

2:39

and that's when there's more than

2:40

one abnormality on an image.

2:42

You find one, but you don't find the other.

2:44

They, uh, um, thought it might be due to failure to continue

2:48

to search for abnormalities after the first one was found.

2:51

But this has actually been exhaustively studied

2:52

by other researchers, most notably Elizabeth Kapinsky

2:55

of Emory University, using eye tracking. And what she found

2:58

Is that, uh,

3:00

after the first finding is made,

3:01

the person doesn't stop searching, their eyes continue

3:04

to search, but there's like a 40% chance they

3:07

won't find the second finding.

3:08

And interestingly, it's not all,

3:10

the mis finding is not always the one that's more subtle.

3:12

Sometimes you see the one that's more subtle, uh,

3:15

and then you fail to see the one that's more,

3:17

uh, conspicuous.

3:19

So that's an interesting thing.

3:21

I don't believe that the mechanism is, uh, getting satisfied

3:25

and saying, I'm done here and,

3:26

and prematurely closing your evaluation of the exam.

3:30

So this type of error, which is very common, I believe,

3:33

is actually misnamed satisfaction of search.

3:35

But that's the number 10.

3:36

And number 12 was a satisfaction of report.

3:39

Are they called complacency of report?

3:41

And that's when you, you look at the thing, uh,

3:43

and it was read previously by someone you really respect,

3:47

uh, and you say, well, David Yosem thought there was nothing

3:50

wrong with this, so I'm gonna, I'm not gonna say

3:52

that there is, I'm not gonna contradict him.

3:55

Uh, and, um, so, uh, that's, uh, error number 12.

3:59

And here's an example from their paper.

4:01

Um, it was a, a, a stone study,

4:04

a stone CT non-contrast CT of the abdomen pelvis.

4:07

And it was a patient with flank pain.

4:09

Rule out kidney stone was the history

4:11

and the impression with no evidence of kidney stones.

4:14

Um, and there's a very subtle finding here

4:16

in the paraspinous muscle.

4:17

And then later, uh, a month later, you can see it's,

4:20

it's gotten to be much larger.

4:21

Here it is an MRI,

4:22

and it, it turned out to be a, a, a mix of fibro sarcoma.

4:27

Very rare, a nasty tumor in the chest wall, uh,

4:30

that was, that was missed.

4:33

And, uh, you could argue the delay in the diagnosis made no

4:36

difference in this patient's outcome.

4:38

Um, so there were two types of errors made in this case.

4:43

Type four, you know, the finding was missed,

4:45

a perceptual error, it was subtle, uh,

4:48

and then type nine location, that wasn't

4:50

where you were looking in this type of study.

4:53

So if you, you look at their 1,279 errors

4:57

in their 656 cases, the, uh, the time

5:00

between the initial exam where the finding was missed

5:03

and the date of the correct diagnosis was, you know,

5:08

from right away to eight years.

5:11

And then, uh, 67% of the cases had more than one type

5:14

of error, and they found

5:16

that there are more errors in plain films, uh,

5:18

and fewer errors, you know, as you come down in ultrasound.

5:22

And they also found

5:23

that errors were not detected on subsequent studies

5:25

about 30% of the time.

5:27

So it's kind of, uh, striking that, you know, um, you,

5:30

you'd have, um, so many errors, you know, 54% in plain film,

5:35

11% in mr 3% in nuclear medicine,

5:39

but only 1% in ultrasound.

5:41

And you know, why, why is that?

5:42

Well, I, I have a theory,

5:43

which I'll tell you in, in a, in a minute.

5:45

So if you look at the number of errors as a percent

5:48

of total errors, you'll see there's two bars

5:50

that really stand out.

5:52

They're type four,

5:53

simply not seeing something, the perceptual error

5:56

And type attend the so-called satisfaction of search,

6:00

which makes up about half as many, uh, errors.

6:03

And we'll talk about, I have a, I have a, I have a theory as

6:05

to why that would be, uh,

6:06

but look at type three, um, 3% of errors.

6:09

So that's the knowledge gaps.

6:11

So as you see, it's very few of our errors are due

6:14

to gaps in our knowledge.

6:16

And part of that is because we all do lots of CME

6:18

and we're all well-trained and we're really smart.

6:20

Uh, but, uh, you know, it does imply that doing more CME

6:24

and learning more is not gonna make a huge difference in,

6:27

in our error rate when most

6:29

of them are simply not seeing something

6:31

or this mysterious thing called satisfaction of search,

6:33

which is misnamed because we don't stop searching,

6:36

but we still don't see in cases with only one type of error.

6:39

Look at this, almost all of them were simply failing

6:42

to see something, the perceptual error.

6:44

So what does the perceptual error look like?

6:46

Well, here's an exa a nice example.

6:49

The history on this was swallowed coin,

6:53

and there it is.

6:55

Um, and, um, it was missed twice

7:00

by a very skilled pediatric radiologist

7:03

and, um, who later couldn't understand

7:05

how I, I could have possibly missed that.

7:06

I mean, the history with swallowed coin,

7:08

there's a swallowed coin.

7:10

It's not particularly inconspicuous.

7:13

Uh, so, uh, that is a, a, a perceptual error.

7:17

So conclusion, uh, most errors are simply failing

7:19

to perceive abnormals, which are, which are, you know,

7:22

deemed sufficiently conspicuous,

7:24

sometimes even obvious, like the swallowed coin.

7:26

In, in retrospect, relatively few errors are due

7:29

to misinterpretation of findings that are detected.

7:32

And very few errors are due to lack of knowledge

7:35

in their, in their series.

7:36

Most errors went undetected for a significant period

7:39

of time up to 12 years.

7:41

Uh, and, uh, nearly one third

7:43

of delayed diagnosis were not

7:45

recognized on the subsequent study.

7:46

Which brings me to one of my favorite New Yorker, um, uh,

7:51

comics where the psychic is telling the woman

7:53

that she could tell all of us, in radiology,

7:55

you will make the same foolish mistakes you have made.

8:01

So this makes me laugh every time, uh, not only once,

8:05

but many, many times again.