0:00

So Michael, one of the things that you commented on was

0:03

that the error rate has been relatively consistent

0:07

for many, many, many years.

0:09

I'm curious because the number of images that we have

0:13

to read per study has dramatically increased, maybe 20 fold

0:18

with fin section ct, for example, or MRI high resolution.

0:23

How do you, how do you, um, balance the paradox

0:26

that we're making the same number of errors

0:30

and yet we're reading so many more images per study?

0:35

That's a great question,

0:36

and it does imply that we are getting better, uh,

0:39

in some ways, uh, that, uh, um,

0:42

but, uh, we are hitting a wall sort

0:44

of bi I think it's a biological neurobiological limit, um,

0:47

that, uh, beyond which we just,

0:49

we can't seem to cross by ourselves.

0:51

And perhaps if we had systems

0:53

that could help us detect when we're in an error prone

0:56

mental state, or if we had that second reader, uh, you know,

1:00

academic medicine, we've had, um, the benefit

1:02

of having double reads on, on most cases, you know, the,

1:05

the resident, the trainee reads it

1:07

and the attending reads it,

1:08

and that's been error correcting, uh, tremendously.

1:10

I mentioned earlier that we had the decreased error

1:12

measurement in the Kim in Mansfield study in ultrasound.

1:15

And it's interesting

1:16

because ultrasound is, is one modality

1:19

that's typically triple red.

1:20

It's double red in private practice,

1:22

and it's triple red in the academic setting

1:24

because the sonographer has an impression

1:26

and then the resident has an impression

1:29

if you have a fellow, they have an impression

1:30

and then it gets to the attending.

1:32

So, you know, it's been looked at an extra time

1:35

or two along the way compared to other modalities,

1:38

and it, it drove down the error rate

1:40

because perceptual error is driving this.

1:43

So, um, on a personal note,

1:46

when I look at my errors in our peer review,

1:49

I make many more errors when I'm reading

1:51

with a trainee than when I'm reading

1:54

by myself despite what you just said.

1:56

And I think that that is a little bit of, you know,

1:59

placing confidence in your colleagues' report.

2:02

Mm-hmm. Um, rather than

2:05

looking at it perfectly afresh, et cetera.

2:08

Um, any comment about that? Yeah,

2:10

Uh, that is definitely an area of cognitive bias

2:13

and, uh, where I honestly, I think

2:15

that what's happening there is probably more anchoring, uh,

2:17

where, uh, um, you know, they've thought about it

2:20

and they came up with an idea

2:21

and they tend to mentally anchor you.

2:23

You have the anchoring bias, the first idea that you hear,

2:26

you say, well, that's pretty reasonable, you know,

2:28

that seems to fit the data.

2:29

You know, and then, and then you, uh,

2:31

because you're anchored, you, you, you don't pick up on,

2:34

on the bits of information that contradict that.

2:38

Um, and, uh, so that, I think,

2:39

yeah, that's probably what it's,

2:42

Uh, I also wanna make a,

2:43

a personal comment about satisfaction of search.

2:45

So one of the cases that was a pretty blatant miss

2:47

by me was in a patient who had fibromuscular dysplasia,

2:51

had dissections as well as intracranial aneurysms,

2:55

and came in with neck pain and, and, um, dizziness.

2:59

And it was a CTA study, and I was busy with all the vessels

3:05

and commenting and everything turned out that

3:07

what I missed was if vestibular schwan on a CTA

3:11

that you could see in retrospect, um,

3:14

and I have to remind myself and tell me where this is true.

3:17

I say it to myself, but I don't know if it's true.

3:19

And that is that the likelihood of someone

3:23

having an additional abnormality

3:25

that I've found an abnormality on is more likely

3:29

than someone de novo having an abnormality.

3:33

In other words, is it true that for someone in medicine,

3:37

a patient, it's more likely that they have two abnormalities

3:41

than all comers having a single abnormality?

3:45

Is that a true fact, or am I making it up?

3:48

No, I, I, I think there's something to that.

3:49

It hasn't been systematically studied,

3:51

but we do have the sense that some patients, if it weren't

3:53

for bad luck, they'd have no luck at all.

3:55

And so they, uh, they'll have,

3:56

they'll have multiple diseases,

3:58

and we, we, we talk a lot really too much about Occam's

4:01

razor trying to find one explanation for something.

4:03

Uh, but there, there was ham's dictum,

4:05

which is the contradiction of that is patients can have

4:07

as many diseases as they please.

4:09

And, and, uh, so, uh, it is possible that, you know,

4:13

that patients who have, uh, one abnormality

4:16

or are, are more prone to have another.

4:18

We do know that, uh, for example,

4:20

if you have one malignancy,

4:21

you're more likely to have a second primary.

4:23

Uh, and, um, so there may be something,

4:25

there may be something to that.

4:28

My final question is regarding what you mentioned about

4:31

with artificial intelligence.

4:32

So a lot of people have been reluctant

4:35

to embrace artificial intelligence

4:38

because of the propensity for false positive reports

4:42

by the AI model, be it for cervical spine fractures

4:46

or for even in mammography.

4:49

In your opinion, overall, is the benefit

4:53

of a second read by an ai, uh, reader mm-hmm, um,

4:59

outweigh the negatives of having a search

5:03

for these false positive areas

5:05

that the AI model has pointed you to?

5:08

Well, yes. Uh, yes.

5:10

I, I think so, but I think we have to be careful.

5:12

Um, first of all, AI and,

5:14

and radiology is still really in its infancy.

5:16

Um, and, uh, you know, we've all been wowed by chat GPT,

5:19

but, um, uh, these new algorithms that find stuff,

5:23

even the most accurate ones,

5:24

are gonna have a high false positive rate.

5:26

Uh, Dr. Grayson Barrett

5:27

of Brown has done some very nice work on this.

5:29

He, I heard a lecture, uh, by him just last week, um,

5:33

where he explained, you know, he's, he,

5:35

he is a statistician, uh, a human factors, uh, person.

5:38

And, and he really explained

5:39

and went through, uh, the likelihood

5:41

of having false positives is, is high

5:43

because of the prevalence

5:44

and the pretest probability disease, even in when your,

5:48

when your artificial intelligence module is extremely

5:51

accurate, and there's always a trade off

5:53

between sensitivity and specificity.

5:55

You're gonna have to have, um, you're gonna have

5:58

to have some, uh, false positives.

6:00

Now, there are real dangers in, with radiologists working

6:04

with ais that we can, you know, over, um, value them

6:09

and, um, you know, blindly believe them,

6:12

or we can doubt them when we shouldn't doubt them

6:15

and trying to find that happy medium.

6:17

I think that we still are, uh, needing a better way for us

6:21

to connect with the AI

6:22

and work with it synergistically to get

6:25

that human AI dyad right.

6:28

I think that we're not there yet.

6:29

The various things that, you know, were scores or heat maps

6:32

or various outputs of AI are still not quite right.

6:36

Uh, I mentioned Dr. Elizabeth Kapinsky a few times.

6:38

Uh, we were at a conference last week

6:40

and she commented that, um,

6:42

it would be much nicer rather than the, you know, for the AI

6:45

to put an arrow on something

6:47

or to give you a score that the likelihood

6:49

of something being abnormal is, is actually to, to be able

6:52

to show you things that you can't easily see yourself.

6:57

Do you foresee a future where

7:01

a medical malpractice plaintiff lawyer will say

7:04

the AI read it as positive

7:07

and you read it as negative?

7:10

We know that the AI is highly accurate, doctor,

7:16

why would you overturn the AI interpretation?

7:20

And isn't the AI the standard of care?

7:24

Absolutely. Uh, we are going

7:25

to be seeing those exact words coming out of the mouths

7:28

of plaintiff's attorneys very, very soon.

7:31

And of course, you know,

7:32

you can bash the pinata the other way too.

7:34

You can say, well, doctor, you know,

7:36

these things aren't always accurate.

7:39

Why did you agree with it?

7:40

Anytime there's a bad outcome, the AI can be used to, um,

7:44

to turn the radiologist into a pinata

7:47

and bash them with this tick until the candy comes out.

7:49

Uh, but on the bright side, um,

7:55

if the use of AI results in fewer errors,

7:58

fewer patients will be harmed

8:00

and there'll be fewer lawsuits.

8:02

But when, um, harm occurs, the AI having used AI

8:06

or not, it'll probably not be helpful.

8:08

In fact, it'll probably make things worse. Uh, Dr.

8:11

Um, Baird has done some work

8:13

with mock juries on this very question,

8:16

and he did find that it did make things worse

8:18

for the radiologists, uh,

8:20

that's been published in a pre in archive Preprint.

8:23

Um, but when the jury was told, you know,

8:25

what the false positive and the false

8:27

negative detection rates of the AI were,

8:30

it made their verdicts a lot more reasonable.

8:32

Uh, so, uh, people lay people can understand these concepts

8:36

and they understand, um, you can, you can help them to,

8:39

to get past magical thinking about AI

8:42

and undermine, you know, that twisted argument

8:44

that really does work both ways.

8:46

You know, whether you agree or disagree with the AI

8:49

or you, you add a little element of magic to the ai

8:51

or you take away a little bit of his abilities

8:53

and you, you know, can cast aspersions either way.

8:56

Um, I'm generally optimistic for the long term

8:59

that if we can re that we can reduce errors

9:02

and reduce malpractice, uh, lawsuits.

9:05

Uh, but, um, certainly it'll be, it'll, it'll be used.

9:08

And the title of, uh, Dr.

9:09

Uh, Baird's paper is, just

9:11

because you're paranoid doesn't mean they

9:13

won't side for the plaintiff.

9:18

Dr. John Banja and I actually published a paper

9:20

about this very dilemma.

9:21

Uh, it was in J-A-C-R-A few years ago.

9:25

Thank you very much, Dr.

9:27

Bruno, the world's expert on making errors

9:30

through personal experience, through personal experience.

9:32

Yeah. We appreciate your time and effort.

9:35

Thank you very much. Thank you. Thanks for having me.