0:02

Hello and welcome to noon

0:04

conferences hosted by MRI Online.

0:06

In response to the changes happening around the

0:08

world right now and the shutting down of in-person

0:10

events, we have decided to provide free daily

0:12

noon conferences to all radiologists worldwide.

0:15

Today we are joined by Dr. Edwin F. Donnelly.

0:17

He is a thoracic radiologist and associate

0:19

professor at Vanderbilt University Medical Center.

0:23

where he has worked since the late 90s

0:25

with past experience as an NIH-funded

0:27

principal investigator, medical school course

0:29

director, and residency program director.

0:31

He currently chairs the ACRAC Thoracic 2 panel and

0:34

vice co-chairs the RSNA AAPM Physics task force.

0:39

Reminder that there will be time at the

0:40

end of this hour for a Q&A session.

0:43

Please use the Q&A feature to ask all questions and

0:45

we will get to as many as we can before our time is up.

0:48

That being said, thank you so

0:49

much for joining us today, Dr. Donnelly.

0:50

I will let you take it from here.

0:53

Okay.

0:53

Well, thank you for having me and, um, I'm

0:56

really happy to be here and be back again.

0:58

Thanks, everybody, for tuning in and listening

1:01

to what we're going to talk about today.

1:03

So my hope today is to spend about 45 minutes

1:06

just talking about artificial intelligence

1:08

and machine learning, a sort of the basics

1:11

of how it's done, a non-mathematical approach.

1:14

Um, sort of give you the sort of walking-around

1:16

knowledge you need to be able to understand when

1:19

people have studies that are using these techniques.

1:23

And then we'll look at some of the major,

1:25

um, areas where research is being done,

1:27

uh, because it's a very exciting field.

1:29

There's a lot of really neat stuff being done.

1:33

I just want to start out with some terminology,

1:35

um, get everybody on the same page, and I know that

1:38

not everybody defines terms exactly the same way.

1:42

So, we'll talk about some generally accepted

1:44

terms that, you know, we use, and you'll be in

1:47

good shape if you understand these definitions.

1:50

And the first word is artificial intelligence,

1:52

or the first phrase, and it's really, um,

1:55

important that we know what this is if we're

1:57

going to sort of use it in everyday conversation.

2:00

So, of all the terms I'm going to

2:02

discuss today, this is the broadest one.

2:05

This includes the largest number of techniques,

2:09

Um, it's sort of the outer, outer set

2:11

that's going to contain all of the others.

2:14

And not everybody will exactly agree on how we

2:17

define artificial intelligence, but basically

2:20

it's the ability of a machine to perform a task

2:24

that generally requires human intelligence.

2:26

So, not a ro, uh, little thing that a

2:28

robot can do, um, but something that generally

2:32

would require human-level intelligence to do.

2:37

And nowadays, we would say that also

2:38

that it needs to be a little bit more

2:40

complicated than a simple calculation.

2:41

So there might have been a time where, um,

2:44

maybe a pocket calculator might have sort of

2:46

seemed like artificial intelligence to people.

2:48

Um, but generally needs to be something higher level.

2:52

So, I'm going to give you an example.

2:54

If you've seen the movie War Games, which I highly

2:57

recommend, if you can find it from the 80s, we're

3:00

going to do a little simple tic tac toe program.

3:02

And that movie had a, um, AI, um,

3:07

device that made a tic tac toe program.

3:10

Or the programmer made one.

3:11

And so anyway, here's how a

3:12

tic tac toe program might work.

3:13

You could program one like this

3:15

if you were so inclined.

3:17

Um, you might have the tic tac toe board

3:19

here, and you let the opponent go first.

3:21

The computer lets the opponent go, goes

3:23

first, and the opponent picks a spot.

3:26

And they happen to pick this top left corner, um, and

3:29

then the computer can go through and calculate all

3:33

the different possibilities of what could happen next.

3:35

So it can look through and say, okay,

3:37

I'm the computer, I'm, I'm zero.

3:39

What happens if I go here, here, here, and you

3:42

know, and then it can pick each one of those

3:44

individuals and say, okay, well, if I go here,

3:46

then what are the possibilities of a compound?

3:48

And it can look at all the different possibilities

3:49

and calculate what's the best next move.

3:52

And so, in general, playing tic tac toe

3:54

would require human-level intelligence.

3:57

But a computer could be designed, and it would just

3:59

be a pure little algorithm that could go through and

4:02

play this game. Um, but the thing about that tic tac

4:05

toe program is it never gets any better. Now maybe

4:09

a clever programmer might say, "Oh, well, I know that

4:11

if I, you know, when somebody does this corner, if I

4:13

go in this corner, it might trick them." But that's

4:15

the programmer being clever, not the machine getting

4:17

any better at the game. So, after one or a million

4:21

games of tic tac toe, the computer algorithm, you

4:24

will still work exactly the same way. When we talk

4:27

about machine learning, we're talking about a

4:29

specific subtype of artificial intelligence, where

4:33

the machine has the ability to iteratively get better

4:37

at its task as it has more experience doing it.

4:43

And so that's why it's called learning.

4:44

So, these terms of intelligence

4:46

and learning, these are analogies.

4:48

There is no intelligence involved

4:51

in what the computer is doing.

4:52

The computer is, in fact, not learning anything, but

4:55

it is iteratively getting better, uh, at what it does.

4:58

Does is by adjusting what we'll call weighting factors.

5:02

And I'm going to show you that here in a very simple,

5:05

and a very simple, I don't know if you guys can see

5:08

that my internet connection is unstable, example.

5:11

So what I've done is I've, I've kind of, um, just

5:13

simplistically come up with, let's make a machine

5:16

learning algorithm that can classify animals.

5:20

And what I'm going to do is I'm going

5:22

to have some animals where I'm going to

5:25

look at their weight and their height.

5:27

And those are the only two things

5:28

I'm going to know about the animals.

5:29

Okay.

5:31

And I'm going to try to classify whether they

5:32

are giraffes or whether they are elephants.

5:35

And I have some initial data to

5:37

start with, some known points.

5:38

So I know some data points for giraffes,

5:41

and I know their heights and their weights.

5:43

And I know some data points for some elephants,

5:45

and I know their heights and their weights.

5:47

But I want my computer program to be able to

5:49

then take the height and weight of an elephant,

5:51

a different animal when I don't know what it

5:53

is, whether it's a giraffe or an elephant and

5:54

then tell me it's a giraffe or it's an elephant.

5:58

So conceptually, I'm going to do this.

5:59

I'm going to take either the height or I'm going

6:01

to take both the height and the weight of the

6:02

animal and then to put them into my algorithm.

6:05

And then my algorithm is going to give

6:06

me some sort of mathematical output.

6:08

So I've chosen for simplicity here just to

6:10

say, okay, if it's, if it's less than zero,

6:13

it's going to be a giraffe; greater than zero,

6:14

it's going to be an elephant.

6:15

I haven't written the program yet, so I'm just

6:16

going to write it in such a way that that's

6:18

the result. I can have it be a number between

6:20

zero and one and say what's less than 0.

6:22

5 it's a giraffe or whatever.

6:24

It doesn't matter.

6:25

Um, just so we understand what we're doing here and

6:28

maybe looking at that, we might realize, okay, well

6:30

the giraffes tend to be kind of tall and the

6:34

elephants tend to be kind of heavy. And so if I this

6:39

wf here, I'm going to call my weighting factor. If I

6:41

give a positive weighting factor to the weight, it's

6:45

more likely to make the answer come up positive.

6:47

If I give a negative weighting factor to the

6:49

height, it's more likely for it to come up positive.

6:53

Less than zero, which is what

6:54

I would want for a giraffe.

6:55

So a very simple mathematical program I could

6:58

have is to take my weighting factor of one

7:01

times the weight of the animal, and then add

7:04

negative one times the height of the animal.

7:07

Or basically, my program, if you see, it's

7:09

really just the weight of the animal at

7:11

this point minus the height of the animal.

7:13

And if the weight is more than the height, it's going

7:15

to be positive, and my computer is going to call it

7:16

an elephant; and if the weight is less than the height,

7:18

it's going to be negative, and my computer program

7:20

is going to call it a giraffe. And so that's all good

7:23

and well. But what happens if, as I get more data and

7:26

I get some more known giraffes and some more known

7:28

elephants, I find out that doesn't quite work? Well,

7:30

it's a little bit off, and the computer can iteratively

7:33

adjust itself. And so maybe over time, as it gets more

7:36

known values, it can improve these weighting factors.

7:40

And it turns out that the computer maybe does a

7:41

better job if it applies a weighting factor of 1.

7:45

4, multiplies that by the weight, and negative 0.

7:49

8 multiplied times the height.

7:50

I mean, I haven't given you any units

7:51

because it doesn't really matter.

7:54

So that's what we mean by learning.

7:57

The computer program got better.

7:59

The algorithm is exactly the same.

8:02

It's a weighting factor times the weight

8:03

minus a weighting factor times the height.

8:06

And it's going to give us our value of

8:08

either a positive or a negative number.

8:10

But as the computer can iteratively improve these

8:13

weighting factors, it can get better at doing its task.

8:16

And that's what we mean by learning.

8:17

That's what we mean by machine learning.

8:19

And that's how these programs are going to work.

8:23

By the way, you may hear the term perceptron.

8:24

We're going to use it just a

8:25

couple of times in this talk.

8:26

And this is basically what it is.

8:28

A perceptron, um, is an analogy for a neuron.

8:33

You can think of these inputs

8:35

as affecting the dendrites, and this is sort of

8:38

the cell nucleus and the axon coming out here.

8:41

This is how we get into things,

8:43

why it's sometimes called neural.

8:44

This is sort of how a nerve works, right?

8:45

Nerves take in various inputs,

8:47

process them, and then give an output.

8:49

And that's what we're doing in our program.

8:53

Now, if we can subdivide machine learning

8:55

further, um, and I've just broken it into two.

9:00

We're going to spend our time today

9:01

talking about neural networks.

9:03

Um, there are a lot of other, I call them traditional.

9:08

They're just non-neural network

9:10

forms of machine learning.

9:11

They're very important.

9:12

A lot of them are used very commonly in radiology.

9:16

We just don't have enough time to talk about all

9:18

the different things that are going on.

9:20

But these use various statistical

9:21

and mathematical ways of

9:26

doing the machine learning.

9:27

But we're going to focus on neural networks.

9:29

There's a lot of activity right now in neural networks.

9:33

And so let's look at what we mean by that.

9:35

So a neural network

9:37

is, if we think about that perceptron that I just

9:39

showed you, that's an analogy for a neuron, like

9:42

that's sort of like what one neuron is like.

9:45

We can take multiple neurons or perceptrons,

9:50

we're just going to call them nodes, we can take

9:52

multiples of these and connect them together so

9:54

that the output, instead of being, uh, it's a giraffe

9:58

or it's an elephant, can be a number that's an

10:00

input to another one of these nodes, and that

10:04

node might get input from multiple other nodes.

10:07

And then it can give output to one or more nodes.

10:10

And so these all interconnect with each other,

10:13

um, and that's what we mean by a neural network.

10:15

It's these different nodes connecting to each other

10:19

in such a way that they process, one will process the

10:22

output of another, and then come up with a new output.

10:26

So I'm going to show you what a neural

10:27

network looks like conceptually.

10:29

We tend to organize them in terms of layers.

10:32

I'll show you some layers, and then

10:33

I'll show you how we connect them.

10:37

Here's my first layer of nodes.

10:38

So this is a row; these are just going to be

10:40

nodes that take input from the outside world.

10:44

So these might be taking things like the

10:45

height and weight of the animal; they might

10:46

be taking values of a pixel intensity.

10:49

But things are going to be coming

10:51

into our network, into these nodes.

10:54

We're going to have several nodes that are

10:55

going to be interconnected with each other.

10:57

And then eventually, we're going to have

10:59

to have some sort of output because we

11:00

want to get an answer out of this machine.

11:02

Now, how many nodes we have at any one of these layers,

11:05

and even how many layers we have, is all variable,

11:08

depending on the task and what you're working on.

11:12

But how it works is, this is just like our perceptron.

11:16

So a value coming out of this node is

11:19

being used as the input for another node.

11:22

And that value might also go to this node, and

11:24

this node, and, and I've sort of drawn it, I

11:26

only drew the first two here, but as if this

11:28

was a fully connected, every node on the input

11:30

is connected to every node on this layer.

11:34

Um, I didn't draw them all; it just got too messy.

11:36

And then every one of these nodes connects to every

11:39

one of, uh, another layer of nodes and that can go

11:42

on and on for however many you need for the task at

11:45

hand that eventually you'll get to the output, which

11:50

is sort of the final step, and then you'll get your

11:53

answer. So we call these the hidden nodes because

11:56

you don't really see them in the outside world.

11:57

So you see what you put in, so you see the input

12:00

nodes, and you see what comes out, but we don't see

12:02

all the processing that goes on in the insides.

12:05

That's all we mean by hidden.

12:07

Um, we don't have to have every

12:08

node connected to every other node.

12:09

In fact, we typically don't in imaging.

12:12

But this is what we're talking about

12:13

when we talk about a neural network.

12:15

We're putting in values.

12:17

We're weighting them differently and then putting

12:20

out new values to other nodes, and they're all

12:22

interconnected, and eventually we get an answer out.

12:24

And the reason it's machine learning is because

12:26

the weighting factors that we're going to be

12:28

using, uh, to process these numbers can be

12:32

iteratively improved as we get more and more data.

12:37

And then finally, it was sort of making

12:39

concentric, um, uh, subsets here.

12:43

We have what we call deep learning.

12:45

Uh, deep learning is basically a neural network

12:49

where we have many of those hidden layers.

12:52

So on the one I drew, we had two layers.

12:54

Um, but not everybody agrees on what,

12:57

how many layers do you need to have

12:58

before you can start using the term deep?

13:01

Um, it’s probably you need at least

13:02

two deep layers, maybe as many as 10.

13:04

It doesn’t really matter for us.

13:07

Um, what’s happening though is in the world of

13:12

machine learning and artificial intelligence outside

13:15

of radiology, there are huge advances going on.

13:19

Uh, and people are making great, um, strides in

13:22

all different aspects of analyzing images or other

13:25

aspects of, uh, learning that can impact radiology.

13:30

And as that technology advances,

13:32

we in radiology advance, uh,

13:34

benefit from those advances.

13:35

So deep learning is a booming, uh, issue everywhere

13:40

and we in radiology are seeing the effects because

13:42

we can then benefit as they do advances in say

13:46

facial recognition that translates into things

13:48

that will also benefit us, uh, in radiology.

13:52

So that’s why we chose to focus

13:53

on deep learning, uh, today.

13:58

Now when it comes to, um, using these, um, models

14:02

or systems that you can use, there’s really three

14:04

different, uh, types of learning that we need to talk

14:08

about today, um, and how they’re related to radiology.

14:13

And I’m going to show you the three of them.

14:14

We’re going to talk about two of them in depth.

14:15

So supervised learning is basically what

14:18

I did with the giraffes and the elephants.

14:20

So I had known values.

14:22

I had giraffes with known heights and weights,

14:24

and I had elephants with known heights and

14:26

weights, and I trained the network on those.

14:30

And that’s what we call supervised learning.

14:32

There’s also unsupervised learning where we

14:35

don’t tell the computer the answers, or we

14:37

don’t label the images, and we let it kind

14:39

of work out what’s going on on its own.

14:42

And it may find unanticipated

14:44

patterns through that technique.

14:45

I’ll show you how that works.

14:48

And the third one, which we’re not really going

14:49

to talk about today, um, is pretty interesting.

14:52

It’s reinforcement learning.

14:53

And that’s where the computer is given a goal,

14:55

but not really told how to get to that goal.

14:58

Um, this is very common in robotics.

15:01

If you're into gaming and you've

15:02

seen some of the advances.

15:04

of, um, what AI has done and

15:07

sort of video games and stuff.

15:09

Basically, the computer gets a little bit

15:11

of a reward when it does the right thing,

15:13

and then it just goes off on its own and

15:15

sort of explores once I encode the project.

15:18

But as it gets more and more rewards, it

15:20

learns what works and what doesn't work.

15:23

And there are some interesting ways that that

15:24

could happen in radiology too, but I

15:27

really want to focus on these first two, and

15:29

especially the first one, and just show you.

15:34

Sort of how it works conceptually.

15:35

So for supervised learning, we have some sort of input.

15:39

And in this case, I'm going to put in these images.

15:42

You can see I've got some different animals here.

15:45

And then we have an expert, and an expert

15:47

comes along; in this case, it's going to

15:49

label what each of these animals is.

15:52

And so, as you can see, I've labeled

15:54

some of them dog and some of them cat.

15:57

And then the network.

15:59

The neural network gets trained based on these images.

16:03

So these images, along with these labels of the truth,

16:05

so, um, the picture plus the label that

16:10

this is a cat; these go in, and the model

16:12

gets trained; those weights get adjusted.

16:14

Um, and then eventually we have what we

16:18

call a trained model that we can use.

16:20

And how do we use it?

16:20

Well, then we have some additional

16:22

images that we don't know what they are.

16:23

So we have these two animals, and we have no idea

16:26

what they are because we're not experts in animals,

16:28

and we can feed them to our computer, to our

16:31

model, and then it's going to give us the output,

16:33

and it's going to say, okay, this here, one on the

16:35

top is a cat, and this one on the bottom is a dog.

16:38

And you can see the analogy, of course, in radiology

16:41

is that we would have imaging, say a chest x-ray.

16:44

We don't know what the diagnosis is.

16:46

You put it into the model, which has been trained on

16:48

known diagnoses, and then it's going to spit out that

16:51

the answer is that it's, um, pneumonia or whatever.

16:55

Right?

16:55

So that's kind of where a lot of what's

16:58

going on in radiology, uh, is right now.

17:01

This is, I would say, the furthest along of any of

17:03

the learning types of techniques in radiology,

17:06

and that is training it on known entities

17:08

to be able to then reproduce that on unknown entities.

17:12

Now, unsupervised learning is very interesting

17:14

because of the future possibilities it

17:16

could have for us in medical imaging.

17:18

So, the way an unsupervised learning algorithm would

17:21

work, we might have our input, I just chose the

17:23

same images, um, but they're not labeled in any way.

17:26

I'm just feeding the computer these different

17:29

images, and it takes them and it processes

17:32

them, and it on its own finds correlations and

17:36

things that it associates with each other, and

17:39

so it's maybe associated these images together.

17:43

I mean, we don't know why; it's a mystery.

17:44

Maybe it's because the paws come out in the front the

17:46

same, um, maybe these two are together because they both

17:50

have something around their neck and these two around

17:52

bedding. I mean, I don't know what, but it's sort of a

17:54

complete, uh, mystery. But the interesting

17:58

thing is that the computer is now not limited by our

18:04

thought process. So, we think that this is pneumonia.

18:08

We think that this is a dog. The computer is not

18:10

limited by our thought process, and it may find

18:13

connections that we hadn't anticipated when it

18:16

starts, for example, looking at medical data.

18:18

Maybe it's going to find certain correlations

18:21

that we never even thought to look for, so

18:23

there's a lot of interesting things, uh,

18:25

potential for this kind of unsupervised learning.

18:27

It's not anywhere near as advanced,

18:29

um, progressed far along as the

18:32

unsupervised learning is in medical imaging.

18:35

Uh, but it's really an exciting,

18:36

uh, area of the future.

18:40

Now, if we think about the sort of tasks

18:42

that we can do with AI, what sorts of

18:44

things can these machine learning models do?

18:47

Um, there is so much research being done in so many

18:51

different fields that almost anything that we

18:55

do with computers or machines, or even without them.

18:58

Could be adapted by, um, AI systems and improve. But I

19:02

picked some of the major sort of categories of things

19:04

that these systems tend to do, and then I'll give

19:08

you a little example. So one would be what's called

19:10

regression, and regression is taking a bunch of past

19:13

values and predicting a future value. So imagine if

19:17

you were really good at coming up with a model that

19:21

could take data from past stock market performance

19:24

and maybe add other data and anything else you could

19:27

find. Maybe, um, you know, maybe flight patterns—who

19:30

knows what? But your computer program can then predict

19:33

where the stock market is going to be, um, say next

19:36

week. I mean, that'd be very powerful; a lot of people

19:37

are obviously working on that. So, and the better your

19:40

model is, of course, the better you will predict the

19:43

future stock market and then make a lot of money.

19:46

So that's regression. Classification is like

19:49

we just looked at with the dog or a cat.

19:51

Here's an image.

19:52

What is it?

19:52

It's a dog.

19:53

It's a cat. It's a fur coat.

19:55

Um, it's a basketball.

19:56

So being able to just say what something is

20:00

or what something isn't, um, is very powerful.

20:03

Object detection is a little bit different.

20:05

Let's say you are, um, writing

20:08

code for a self-driving car.

20:10

And that vehicle is driving along and it

20:12

needs to identify different things it sees.

20:15

So it needs to see that there's a stop sign coming up.

20:16

It needs to see that that thing that's right in

20:18

front of it is a pedestrian, uh, in the roadway.

20:22

Uh, when there might be multiple pedestrians, there

20:24

might be five different pedestrians at different

20:26

points in its field of view, and there might be a stop

20:28

sign coming up, but it needs to be able to identify

20:29

each object, um, and know what's coming at it.

20:34

So that's a little different than classification.

20:36

And then generation, which is creating new, um,

20:42

data or information or images, or in this case,

20:44

there's been a lot of stuff in the news and the

20:46

real news about fake news. Fake news is, um, can

20:50

be used; we can have AI programs generate data,

20:54

entire stories, articles that sound entirely real.

20:57

Um, but they're totally generated by a computer.

20:59

Um, so all of these things are going on.

21:03

There's a ton of research in all

21:05

these different fields, and they all

21:06

have applications to us in radiology.

21:09

Um, so regression, instead of predicting the stock

21:12

market, you might be able to take other imaging

21:14

and predict what a CT pixel value would be.

21:17

So maybe you're trying to create

21:18

a CT scan out of something.

21:20

Um, and you can use regressive techniques to

21:23

make that kind of prediction. Classification we

21:26

talked about a little bit, sort of pneumonia or

21:28

non-pneumonia on the chest X-ray, fracture or

21:30

not fracture, you know, subdural hematoma or not.

21:33

And so a lot of things on the images looking

21:37

at the image classifying the images based

21:39

on disease or not disease. But then also object

21:42

detection could also be very important in imaging.

21:46

So in chest radiology, I think about, okay,

21:48

can it detect the different central lines

21:49

in the chest and the other hardware devices?

21:51

Or maybe you're a mammographer and you want it to

21:53

detect, um, the malignant calcifications or masses, or

21:57

you want it to find things in the liver on a CT scan.

21:59

So picking out abnormalities within the image,

22:02

obviously incredibly important to us in radiology.

22:06

And then just generating, uh, images or other

22:09

things like reports, but you know, there's

22:11

a lot of work generating CT scans from MRIs.

22:14

You know, why would you want to do that?

22:15

We'll talk about that, um, shortly.

22:17

So these are the kinds of tasks that we're

22:19

going to ask, uh, AI to do in radiology.

22:24

Before we get too deep into that, there are a couple

22:26

more, um, concepts we need to talk about, and these

22:29

are related to what I've called computer vision.

22:31

Um, not everybody would classify everything

22:35

that I'm talking about exactly the same way, but

22:37

these are image, um, manipulation and processing

22:41

techniques that are very important to radiology.

22:44

Um, and now that they overlap with machine

22:47

learning, um, are becoming even more important.

22:51

Traditional computer vision is

22:52

a process of taking, um, images

22:56

and processing them in various different ways.

22:59

Often what you want to do is extract

23:01

a particular feature, maybe extract

23:03

corners or edges or round things.

23:07

It involves a lot of programming and

23:10

various image processing algorithms.

23:13

I would include in this segmentation, which is the

23:16

automatic outlining of structures that can be very

23:18

important, um, for various things in radiology.

23:22

Radiation oncology is very important, uh, for them

23:25

to be able to outline tumors, uh, and anatomy,

23:28

um, and generally to do most of these traditional

23:33

computer vision techniques, it requires a lot of

23:35

very specific coding for each individual feature.

23:38

So if you want to extract edges, you

23:40

need to process the image a certain way.

23:42

Um, but it can provide some really powerful

23:44

results, um, as the output.

23:48

Um, this is a friend of mine from the

23:50

Nashville Zoo I took a picture of.

23:52

Um, and...

23:55

Let's say I wanted to take this

23:57

image and I wanted to find the edges.

24:00

So there are image processing techniques I can do

24:02

to this image that will then pull out the edges and

24:08

de-emphasize the non-edge portions of the photo.

24:12

So you can see what's happened

24:13

now that it's been processed.

24:14

The lines of the face of this

24:16

bird are much easier to see.

24:19

We can see where the eyeballs, the

24:22

pupils, so this would be, um, perhaps the first

24:27

step in doing some additional processing.

24:30

Maybe I'm classifying birds based on the distance

24:32

between their eyes, or maybe there are features in

24:34

here that are important, or maybe I'm going to

24:36

do some measurements and, um, but, um, a lot of

24:41

different things could then happen, and we can do

24:42

a lot of different sorts of processing to bring

24:45

out whichever features we need for the next step.

24:47

In our, um, in our whatever we're trying to do.

24:53

So what we really like about these

24:55

traditional computer vision techniques is

24:56

we know exactly what the code is doing.

24:58

So when I did that processing to the

25:00

image, I knew exactly what it was doing.

25:02

It was taking a blurred copy of the image,

25:04

subtracting it, and bringing out the edges

25:05

in a certain way that is very predictable.

25:09

What we're seeing now with neural networks is, it's

25:12

a little more of what I would call a black box.

25:13

So.

25:14

The computer is able to extract features, but

25:16

it's not using techniques that we understand.

25:18

It's using these interconnections between

25:21

the nodes and the different layers.

25:25

And we can encourage this by wiring

25:29

together the nodes in a particular way.

25:31

So by wiring them in sort of clusters

25:34

that we call convolutional layers, we can

25:37

encourage it to extract features in the

25:39

images, but we don't exactly know always.

25:44

What features it's looking at, and we

25:45

can't predict how that's gonna work.

25:49

So as we do that, you'll hear the

25:50

term convolutional neural network.

25:52

Um, and so that is a specific type of neural network.

25:54

It's a sort of a subset of deep

25:56

neural networks or deep learning.

26:00

Um, and it just has to do with the way we interconnect

26:02

them to bring out features within the image.

26:07

Alright, the next computer vision sort of

26:08

technique I want to talk about is computer

26:10

aided detection, um, and diagnosis.

26:15

So, the whole term CAD itself has always been a little

26:17

bit confusing for people because it can stand for both

26:19

computer aided detection and computer aided diagnosis.

26:23

Some people will use DE for detection, DX for

26:25

diagnosis, but we just want to talk about the concept.

26:29

It's been very interesting.

26:31

deeply studied in mammography.

26:33

Um, they've been using it since before

26:35

we had digital mammography systems.

26:36

People would scan film mammography into computers

26:39

that would then try to detect abnormal lesions.

26:42

In general, a CAD system, a detection

26:45

system, we want it to be very sensitive to

26:47

abnormalities, but not necessarily specific.

26:50

That is, we wanted to find a lot of potential

26:52

cancers, even though we know there's

26:55

going to be a lot of false positives.

26:58

These systems have really shown, um, promise as

27:02

being either a second reader or maybe as a way to

27:05

preview a study before a radiologist looks at it.

27:07

They're nowhere near ready to, you know, interpret

27:10

or just even read screening mammography on their

27:12

own, but they have been shown to help a radiologist

27:15

and maybe find a lesion that might've been missed.

27:19

Um, and so they could be beneficial that way.

27:22

And there's been a lot of studies.

27:23

So it's moved on from mammography.

27:25

There's a lot of studies on chest x-rays, various CTs, MRIs,

27:28

other things looking for cancer or, um, other things.

27:34

So you've probably seen these

27:35

because they are, um, in wide use.

27:37

So you have a, uh, say a mammogram and then

27:39

a computer will superimpose upon it spots

27:42

that it thinks are potential abnormalities.

27:44

And then it's up to the radiologist to then

27:46

go and look at each one of those and say, you

27:49

know, okay, that is an abnormality, that's not a

27:51

potentially significant abnormality, and move on.

27:54

A lot of times these create a lot of frustration.

27:56

for the radiologists because they find so many

27:58

abnormalities, which turn out to be nothing.

28:02

Okay, the other part of that D is those computer

28:04

aided diagnosis, and so that is trying to

28:07

draw a conclusion from the image, not just

28:09

find an abnormality, but then decide something

28:12

significant about what that abnormality is.

28:16

Now the, one of the most common things is,

28:19

is that lesion a benign lesion or a malignant lesion?

28:21

And you can see obviously why that

28:23

would be very beneficial to the patient.

28:24

If we knew this is malignant and this is benign,

28:27

that would really help guide our management.

28:30

It might also quantify a disease.

28:32

So maybe a computer program could quantify how much

28:35

calcification is in the coronary arteries, so that

28:38

would be a sort of a sort of computer-aided diagnosis.

28:41

Maybe it can do something like calculate the bone

28:43

age from the hand film that might otherwise

28:47

take us manually looking through an atlas.

28:51

And most of these diagnostic kinds of

28:54

things are based upon looking at various patterns

28:56

in the image and trying to draw conclusions.

29:00

This is an image from Dr.

29:01

Hagstrom has created a pretty interesting

29:04

bone age detection system.

29:09

And here you can see where his

29:11

program is analyzing the image.

29:13

It's doing a lot of processing of this image,

29:16

extracting a lot of features, a lot of data, and

29:20

then calculating what it thinks the bone age

29:24

is, so there are a lot of really interesting

29:27

tasks that we can have the computer do for us.

29:31

And as we these get combined with machine learning,

29:34

his program uses machine learning as well.

29:35

But as these computer vision

29:37

tasks get mixed with machine learning,

29:39

there's even more that can be done.

29:42

And the third sort of computer vision, if you

29:44

will, technique to talk about is radiomics.

29:48

The radiomics is very interesting

29:49

because it's a specific technique that

29:51

was really developed just for radiology.

29:53

You might gather that from the name.

29:55

The name is a play on words.

29:57

So you've heard of genomics and proteomics, and

30:01

so radiomics was supposed to be sort of, um, in

30:05

that same group of, um, sort of scientific study.

30:10

And that is large volumes of data that you can go

30:13

through and try to extract and gain information about.

30:16

The patient or the disease or

30:17

whatever it is you are studying.

30:20

So the way radiomics works is it extracts

30:23

a really large number, hundreds sometimes,

30:25

of imaging features, uh, from a study.

30:29

Uh, it quantifies them mathematically.

30:31

So these might be simple things

30:33

like the size and shape of a lesion.

30:35

It might be the texture of the lesion, which is a

30:36

little bit more complicated, but then there are even

30:39

more complicated higher-order features that it can

30:41

look at and extract. And then it basically searches

30:45

for statistical correlations between these radiomic

30:49

features in the image and say the patient's disease

30:53

or prognosis. So radiomics has really taken off in

30:56

oncology because it's really showing a lot of promise

31:00

in this benign versus malignant question, but there

31:03

are applications to it elsewhere in imaging too.

31:06

That's really going to do sort of the same thing.

31:09

You might find this interesting.

31:10

I just did a search of PubMed articles using

31:14

radiomics, and for 2020, I just did the first quarter

31:17

and multiplied it by four so we could compare.

31:20

You know, we're at the point.

31:21

Remember, this is a radiology-specific thing.

31:24

So, in 2020, basically, there are three

31:27

or four articles about radiomics

31:30

every single day being published in our literature.

31:34

Um, so there was just a ton of research

31:37

being done in this and a lot of really

31:39

interesting things, um, happening.

31:42

All right.

31:42

So when it comes to computer vision, we have these

31:44

different techniques, and there's a lot of overlap,

31:46

but it doesn't really matter if you want to try to

31:48

say radiomics is actually a computer-aided diagnosis.

31:52

And, um, we don't really care how they overlap

31:57

or what you want to call any one particular technique.

32:03

But all of them become a lot more powerful

32:05

when we combine them with machine learning.

32:06

So as these, um, techniques, which are designed to

32:13

extract features from images can then take those

32:16

features and use them in a deep learning system.

32:19

And then that deep learning system can

32:20

then extract really important information

32:23

that we can then use in radiology.

32:26

So how do we use these applications?

32:28

So, I just want to talk about some of the major

32:30

applications for machine learning in our field, um,

32:35

and give you an idea of what we can do with this.

32:38

So one thing that you can do is create novel images.

32:41

So what do I mean by a novel image?

32:44

A novel image is a false medical

32:46

image that never existed before.

32:48

And if you hear me say that, that

32:51

sounds like a sort of a nefarious thing, but

32:53

it's actually a lot of really potential good

32:54

uses, um, for these, um, types of images.

33:00

So, one group has done a really interesting

33:02

uh, study where you can take, you can have the

33:05

computer generate what a disease would look like

33:07

in a, say, a hypothetical patient, but over time.

33:09

So, you can take one chest X-ray and evolve

33:13

it to show the various stages of pulmonary

33:15

edema, um, as a person, patient goes into, say,

33:18

heart failure and then out of heart failure.

33:20

And by having the computer generate this, everything

33:23

could be the same except the important features.

33:25

Uh, that are changing as the patient

33:27

goes in and out of pulmonary edema.

33:28

So it could be a really powerful educational tool.

33:33

I think about it trying to teach residents or

33:36

computers about various hardware, uh, combinations.

33:40

So it's one thing for me to teach you where, say, a

33:43

right IJ central line is and then a left IJ central

33:47

line, and the two together, but if you want to

33:50

train a computer, you need to have all the different

33:53

possible combinations come together in and out of

33:56

different images, and it's hard to do if you just

33:58

want to go through and look at, um, where they

34:01

happen to coincidentally happen in existing patients.

34:04

So there are a lot of, um, sort of hypotheticals

34:07

that you, boy, I wish I had an example

34:08

of this, this, and this all in the same

34:10

film where you could put them together.

34:11

And again, not just for teaching

34:12

people, but also for teaching

34:15

the machines.

34:19

Here's an interesting, um, example.

34:21

So this is a chest radiograph.

34:22

It's not particularly interesting,

34:24

and then it's been modified.

34:25

So if you look right here, I'll actually go back.

34:27

Um, and what's happened is we've mathematically

34:30

added a pulmonary nodule right there.

34:34

There's a couple reasons why you might want to do this.

34:37

So there are people doing interesting studies

34:40

on image perception, and by being able to

34:43

mathematically add pulmonary nodules to chest

34:47

X-rays, you could study, for example, what is

34:50

the smallest size nodule a person could see?

34:52

Or you can study the effect of density

34:54

on how well a nodule can be seen.

34:56

Or you can study which nodules are

34:57

missed based upon where they are located.

35:00

Um, so there were a lot of sort of

35:01

controlled studies you could do if you

35:03

were able to manipulate the nodules.

35:05

And that's much, much easier to do than if you had

35:08

to go and find all these novel nodules on your own.

35:13

Also, if you wanted to train a neural network to

35:17

detect pulmonary nodules, if you could add the

35:20

pulmonary nodules yourself into all the different

35:23

spots, you could create a better training set

35:26

potentially than if you had to go out and find all

35:28

these nodules on their own. So a lot of really powerful

35:32

things you could do, um, with such systems. One of the

35:38

reasons it's really good to find alternate ways to

35:41

get novel images to train artificial intelligence

35:44

systems is because it's really difficult to get a

35:46

good training set, uh, out there to train a system.

35:50

So it's very easy if you're studying. Say you're a

35:53

neural network programmer, and you have an idea for

35:57

improving categorization. You, well, you can go search

36:01

Google Images, and you can get a thousand cats, and you

36:03

can get a thousand dogs, and you can test your system

36:06

on that, and you know exactly which animals are dogs,

36:09

which animals are cats, and there's no issue for that.

36:11

But try to do the same thing with

36:13

pneumonia versus not pneumonia.

36:15

We can't even get the radiologist to agree.

36:17

We can't even get the chest radiologist

36:18

to agree on who's got pneumonia, who's

36:20

not, and it's not always even clear.

36:21

So there's a lot more unknown when it comes to an

36:23

actual image, and you need very large numbers of

36:27

cases, which is, um, difficult to do if you want

36:31

to, um, get studies from multiple institutions

36:34

because suddenly you have privacy issues.

36:36

You need to make sure they're labeled correctly

36:37

without giving away private health information.

36:40

By labeling, I mean this is pneumonia.

36:42

This is pulmonary edema. This is a left subclavian

36:44

central line that goes up the right IJ vein, and

36:47

you know exactly how you do that, and consistently

36:49

do that. Um, is very difficult, and I might try to

36:53

come up with a really good data set just based on

36:56

images that I get at Vanderbilt University Medical

36:58

Center. But maybe my patient population is not

37:01

representative of the nation as a whole,

37:04

and when somebody tries to, um, use the same

37:07

technique, say, in the Northeast, where maybe

37:10

they don't have the same degree of histoplasmosis that

37:13

we talked about three weeks ago that we have here.

37:15

And then suddenly the program doesn't work.

37:17

So there are a lot of logistical issues when it

37:21

comes to creating, uh, training sets for AI systems.

37:24

And so maybe if you could generate

37:26

your own, it would be much easier.

37:28

You would have a known truth of which there'd

37:30

be no doubt because you created that image.

37:34

A really fascinating time.

37:35

My favorite topic at all of, um, artificial intelligence

37:38

are GANs or Generative Adversarial Networks.

37:42

We can't really talk too much about them today, but you

37:44

will see more and more about them, um, in the future.

37:48

So, what these are is a competitive computer

37:52

system to generate, in our case, novel images.

37:56

And by competitive, I mean they

37:57

really employ game theory

38:02

to the process.

38:03

So you have two different computer systems.

38:06

One of them generates images, and it can generate

38:09

them from scratch, or it can generate them

38:10

based on some pre-existing image, whatever.

38:12

It's going to generate some sort of new image.

38:15

And the other system is operating in parallel

38:18

and is going to try to determine whether the

38:21

image that's being shown was a real image from

38:25

reality or if it was an image that the other system

38:27

created, and both systems employ machine learning.

38:31

So both systems iteratively get better over time.

38:35

And so what that means is the system that's

38:37

creating the images gets better and better over

38:39

time, and they sort of compete with each other.

38:42

And so there are some fascinating things with,

38:45

there's a, um, high-resolution celebrity photos.

38:49

They've got thousands of computer-generated images.

38:51

They look like celebrity photo shots of

38:54

people that have never existed before.

38:55

And these things are coming into medical

38:57

imaging, uh, and they're going to have a lot

38:59

of really interesting, um, applications for us.

39:03

Okay.

39:04

Another really important area, uh, for artificial

39:06

intelligence is improved image reconstruction.

39:09

Um, so using these techniques, one thing

39:12

you can do is create a CT scan that has

39:14

less noise and/or uses less radiation dose.

39:17

So those two entities are tied to each other.

39:20

So if you can reduce the noise, you could

39:22

reduce the radiation dose and vice versa.

39:25

Um, but by using these techniques,

39:26

we can then improve, um, CT scanning.

39:30

There are.

39:31

AI techniques that can allow MRI scans

39:34

to be acquired much more quickly.

39:37

Because the computer can figure out which

39:38

parts of the image might change over

39:39

time, it could maybe only acquire certain

39:42

things that it needs to fill in the gaps.

39:45

And so you could have less motion, you could

39:47

do more sequences over the same period of time,

39:49

you could just get an all-around better study.

39:52

I mentioned earlier converting an MRI scan,

39:55

there's a lot of work being done on that.

39:58

Might seem a little simplistic to us

40:00

at first, but there's actually a lot

40:01

of reasons why you might do that.

40:03

First of all, um, CT scan has radiation

40:06

dose, obviously, but an MRI does not.

40:08

So if we could image somebody with an MRI

40:10

scan who might have otherwise have needed a

40:12

CT scan, we can eliminate that radiation dose.

40:15

A patient who might need both an

40:17

MRI scan then gets fewer studies.

40:21

Logistically, it's easier because

40:22

they only needed to get the MRI scan

40:23

instead of the MRI scan and the CT scan.

40:26

Um, this is being used a lot, uh,

40:29

for PET attenuation correction now.

40:31

Or at least being studied, that is, a patient has an

40:33

MRI scan which they needed for diagnostic purposes

40:36

and they get the PET scan, but they don't need to get

40:39

the CT scan because they can then do the attenuation

40:42

correction off of a virtual CT scan that they created

40:45

from the MRI. Same thing in radiation therapy. Again,

40:50

you might need the MRI to see for diagnostic purposes

40:53

and again, it has no radiation dose, but then you

40:56

can do your planning which requires things like

40:58

attenuation correction off of a CT scan, a virtual

41:03

CT scan basically created from the MRI scanner.

41:08

I mentioned image segmentation previously as a

41:11

computer vision technique, but as an AI technique,

41:14

it's really becoming more and more important,

41:16

um, in and of itself, and also as a precursor

41:19

to other things that we can do, uh, with images.

41:23

So basically, image segmentation is where

41:24

the computer outlines the anatomy or the

41:27

pathology, or both, that we are looking at.

41:30

interested in.

41:33

So if you've been involved at all in radiation

41:35

therapy planning, this is a really important step

41:38

that they have to outline the tumor very carefully.

41:39

They have to outline the normal anatomic structures

41:42

very carefully, um, because they're going to have to

41:44

determine which dose, what dose they're going to get.

41:47

Um, it's very labor-intensive and it's

41:49

not, it's not always very easy because

41:51

the tumor can distort the anatomy.

41:53

So to be able to have a computer do this

41:55

automatically can save a lot of time and effort.

42:01

Having the computer be able to do image segmentation

42:03

can also help in doing, say, image registration

42:06

or image fusion. So if you're trying to combine

42:08

two different modalities and the computer can

42:12

on its own outline major anatomical structures,

42:14

that's an important way, um, to improve how the

42:18

two images can then be registered and fused

42:20

together, particularly if you're not talking

42:22

about a rigid registration, so something where

42:24

there might be movement, say you're trying to

42:26

an abdominal, uh, CT and an MRI, where maybe

42:29

there was some movement and the computer could

42:32

do that segmentation and match up the anatomy.

42:36

And it's also an important precursor

42:37

to future image modification.

42:40

So for example, you might have an AI system

42:43

that can remove the bones from a chest X-ray.

42:46

Well, the first thing it can do is outline the bones,

42:50

segment out the ribs, and then go on to

42:52

proceed to however it's going to then

42:54

do the rib or bone removal.

42:56

So it's an important precursor to doing

42:58

future things and, as you'll see, an important

43:01

precursor to, if we're going to have the machine

43:04

learning program do image interpretation, being

43:07

able to outline anatomy and pathology as a

43:10

precursor is going to be important as well.

43:13

And just to take you back to Dr.

43:14

Hagstrom's, uh, bone age, we didn't talk too much about

43:18

the segmentation, but look at how detailed the

43:22

program, uh, has outlined some of the bone anatomy.

43:25

It's even looking at the cortex, it's looking

43:27

at, you know, growth plate versus epiphysis.

43:30

Um, so being able to exquisitely outline structures

43:34

on images is very important, and the machine learning

43:36

can really help us do a much better job of that.

43:42

And then finally, of course, sort of the, um, uh,

43:45

holy grail, I guess, if you will, of, um, AI and

43:49

radiology is being able to interpret, uh, images.

43:53

I, I sort of think of this in sort of

43:55

three sort of subsets or three sort of

43:57

tiers of things, um, that can happen.

44:01

And we'll look at them sort of each individually.

44:03

So what is having the computer do a very specific task?

44:07

Um, and then some of this is available right now.

44:10

So there are some programs that offer this,

44:13

uh, right now in various stages of research or

44:16

marketability. But say I might, for example, be able

44:19

to do some organ size measurements or some tumor

44:22

measurements automatically. You click on the lung

44:24

nodule and it can, it can, you know, determine its

44:26

size or do it without clicking on it, even better

44:29

AI. Certainly the CAD programs that detect lesions.

44:34

Um, that is a sort of a very specific task

44:36

finding potential abnormalities in the mammogram

44:38

in the mammogram, um, evaluating lesions.

44:42

So a lot of these computer-aided diagnosis and

44:46

radiomics types of techniques that have now

44:47

been combined with artificial intelligence can

44:50

look at a lesion and make some predictions.

44:54

Obviously, no, no system has reached, you know,

44:58

a point where you, you don't need to do a biopsy kind

45:00

of thing, but there are systems that are coming

45:03

close to being able to help us stratify who does

45:05

need a biopsy, who doesn't need a biopsy, who,

45:07

who can maybe go with serial evaluation versus

45:10

who should be more prompted to get a biopsy.

45:11

And I think as that improves, it's

45:13

going to be incorporated more and

45:15

more into the decision-making process.

45:18

If you're an IR or IO, uh, kind of person here,

45:22

there's a lot of work being done in that field.

45:24

Uh, so one of the areas they're looking at

45:25

is predicting tumor response to treatment.

45:27

So deciding, you know, which hepatic tumor is

45:30

going to benefit if you do a chemoembolization

45:33

and which ones, which ones won't.

45:34

Um, so that sort of predictive kind of,

45:37

um, being studied, or maybe just looking

45:40

for a specific diagnosis on images.

45:43

So maybe it just takes a CT scan and tries

45:44

to look at the liver contour and to say

45:47

there is cirrhosis or there's not cirrhosis.

45:49

So something like that, um, being studied.

45:52

And again, here's our example of our CAD system

45:55

where, you know, we still have the whole image there

45:57

as the radiologist to look at it, but the computer

45:59

is at least helping us with a little something.

46:01

It's pointing at a specific task of finding some

46:04

things that are potentially wrong, uh, with the study.

46:08

By the way, this is a normal mammogram,

46:09

so there's nothing wrong with it.

46:12

So another important and interesting area of research,

46:15

and some of the stuff's in the early stages,

46:17

but maybe coming out soon, is finding emergent findings.

46:21

So the idea is not even to be as good as

46:24

the radiologist or even to really help the

46:26

radiologist, but to be able to find things

46:28

that are time-sensitive, maybe before

46:31

anybody else would have looked at the image.

46:32

So it's sort of a triage system.

46:34

So this is being looked at on, on head CTs, say,

46:39

to look for, you know, large stroke, large vessel

46:41

occlusion, cerebral hemorrhage, those kinds of

46:43

things, maybe inpatient, down in the middle of the

46:45

night, nobody's going to look at it right away,

46:47

but if a triage system could alert and say, hey,

46:50

somebody needs to look at this, a test x-ray, is

46:52

there a pneumothorax, is there a malpositioned

46:53

line, something that you might, you know, otherwise

46:55

have a critical, um, send a critical alert if you

46:58

were to see it and alert somebody to it if the

47:00

computer could find it more quickly than the patient

47:03

could benefit, say, free air on an abdominal film.

47:05

So, you know, you have a study like this, um,

47:09

where the patient has an epidural hematoma

47:11

and they've got shifted their midline.

47:13

The radiologist is not going to have any trouble making

47:16

this diagnosis, but if the computer can alert somebody,

47:19

say, within minutes of the study being done as opposed

47:23

to until somebody gets around and looks at it on their

47:25

worklist, there might be some benefits to the patient

47:28

or how long is this patient going to sit around?

47:30

With these large bilateral pneumothoraces

47:32

before somebody calls up the film to look at

47:34

it. And so that could be a real benefit and

47:37

probably an early win hopefully for AI. But

47:42

what about the general interpretation?

47:44

Well, this is what I think is pretty far away, uh,

47:47

you know, if we're ever, ever really going to see that.

47:49

And that is sort of where the computer

47:52

just totally interprets the image.

47:54

Now I do think there is a future for

47:55

the, um, sort of hybrid, if you will,

47:58

between machine learning and radiology.

48:01

And we're seeing this with,

48:02

the CAD and the radiomics.

48:03

Techniques, I think that I say they're sort

48:05

of leading the way and they're sort of showing

48:06

us what that pathway is, and that pathway is

48:10

basically that the computer is a tool that can

48:12

help the radiologist either be more efficient

48:15

or be more accurate, or maybe be both.

48:17

But by working together with the tool,

48:20

they can produce a better product, a better, you

48:23

know, interpretation or a more timely interpretation

48:25

than the radiologists could without it.

48:29

And so I really think that the future

48:30

we're looking at is sort of a hybrid.

48:33

This is not unique to radiology by any means.

48:34

I said, path and derm, um, similar stuff is being done.

48:38

Uh, the fundal exam, you know, we looked inside

48:40

the pupil and the eye, the images.

48:43

Um, so anything where they use any kind of

48:45

imaging, all of this stuff is being done,

48:49

but it's really not even unique to imaging.

48:51

So this field overlaps heavily

48:53

with, I'll just call it robotics.

48:55

And so, you know, the surgeons are dealing with robotic

48:57

surgeries and more and more stuff becoming autonomous.

49:00

If you're going into IR, you're going to have

49:02

the same sorts of things where more and more,

49:05

um, computer-controlled devices are doing things

49:08

that used to previously require humans to do.

49:11

But again, it's going to be a hybrid future.

49:16

This is just like straight off of the, um, off

49:19

of the regular list from, from a week or so ago.

49:22

And, um, yeah, I just look at this, this

49:24

kind of thing pops up every single day.

49:25

And I mean, how is anybody ever going to train

49:27

a computer to interpret something like this?

49:29

But weird things like this are always going on.

49:32

You're always going to need a radiologist, uh,

49:34

to figure it out because nobody's ever going to

49:36

train a system with an image like this.

49:40

Um, and why I threw this on

49:42

here just to remind you that.

49:44

Remind me to tell you a little story.

49:46

Um, so I went to med school in the early nineties.

49:50

And sort of a novel thing back then, although it

49:53

was out there in the commercial, people had it were

49:55

these computers that would interpret the 12-lead EKG.

50:00

And this is sort of the future, and you know

50:02

what's going to happen to the cardiologist

50:04

that the computers could interpret the EKG.

50:07

But the reality is, even today, all

50:12

of these are still being looked at by cardiologists.

50:13

People appreciate the early interpretation

50:16

that the computer might give them, but,

50:18

but nobody sees it as a substitute.

50:19

And I think that's sort of a model.

50:20

I mean, we've had these now for 25, 30 years, um,

50:24

and I think we coexist fine with them, and

50:27

there are plenty of cardiologists in the world,

50:28

and that, um, has not been a problem for them.

50:32

So I guess in summary, I think it's extremely doubtful

50:35

that any physician of any subspecialty is going to be

50:38

operating without tools like these, whether they be,

50:41

you know, image interpretation tools or robotic tools

50:43

or algorithms that are going to tell a primary care

50:46

person what's the proper pneumonia, uh, antibiotic

50:50

in this particular situation. This is going to

50:52

happen to everybody in the not too distant future.

50:57

But I think it's, you know, so it's doubtful that

50:59

we're going to see anybody not having these tools,

51:00

but it's equally doubtful that we're going to see

51:02

any of these tools operating without physicians.

51:04

I don't think they're going to operate on their own.

51:06

Um, we're going to see a hybrid

51:07

future with both of us there.

51:11

Um, so, we're doing great on time.

51:15

I'm going to look at some of

51:15

your, some of your questions.

51:16

If you have any questions you'd like to

51:18

ask me, you can type them into the Q&A.

51:20

Um,

51:23

so somebody asked about, uh, how many images do you

51:26

need, um, to get a program going such as breast cancer?

51:31

You know, how many images to take?

51:32

Well, that's, that's actually, um, sort of an area that

51:35

my research focuses on: image data sets

51:39

and you know how many images do you need, and

51:42

sort of the question is sort of how low can you go?

51:44

How few images can you get away with? And surprisingly,

51:47

um, low in some areas, and it sort of depends on the task.

51:50

A lot of times you don't know until you know. But

51:53

I will say for basic things, ballpark 100 normals and

51:59

100 abnormals is an interesting ballpark to start with.

52:03

You could get upwards, depending on the disease, of

52:08

two to ten times that many. You could probably get

52:10

away with a half to a fourth that many. Um, a lot of

52:13

times it's going to, um, depend on, um, the disease, the

52:18

model, um, and sort of what's available to you, uh.

52:23

Difference between machine learning and deep learning.

52:25

Um, so, um, you anyway, the difference between machine

52:29

learning and deep learning, and basically deep learning

52:32

is a subset of machine learning, um, where deep

52:36

learning involves multiple, like somewhere between

52:39

two and ten, maybe at least, or more, um, hidden

52:43

layers in that neural network that we talked about.

52:45

So deep learning is a subset

52:47

of, of, uh, machine learning.

52:49

It's a subset of neural networks,

52:51

which is a subset of machine learning.

52:56

If the computer makes an error and

52:57

mislabeled a lesion, who goes to court?

53:00

Um, so, um, you know, I don't know the answer to that,

53:04

except that it, it, what I see is the future of the

53:07

sort of the hybrid of the physician and the computer.

53:12

Ultimately, everything rests on the physician.

53:14

I don't care so much about the legal

53:16

ramifications, but that is sort of we are

53:17

the ones who take care of the physician.

53:19

And so I see these computers sort of like the CAD.

53:22

The CAD program finds a lot of abnormalities that

53:24

are potentially cancer or the early detection system.

53:30

It might flag a lot more head CTs than really

53:32

have epidural hematomas, but it's really

53:34

the radiologist who then has to look at

53:35

the image and say, "Okay, that is a cancer.

53:37

That is an epidural hematoma," or "No.

53:39

No, that's a meningioma. I don't know why this crazy

53:41

computer thinks that that's an epidural hematoma.

53:43

It's totally fine." So ultimately, I think from a

53:46

patient care standpoint, I don't know too much

53:48

about the law or really worry about it too much

53:50

in my day-to-day practice. But I really think

53:53

we are ultimately responsible for the patient and

53:54

the computer is our tool. So just like if the PAC

53:57

system's not doing something well, we need to fix

54:00

it and get that image out there so we can see it.

54:02

It's going to be the same thing with AI.

54:03

AI doing something weird or flagging something unusual.

54:06

We need to, you know, ultimately make

54:08

the decision and we need to be the

54:09

ones who take care of, um, the patient.

54:14

Okay, are you worried?

54:15

About the future of the human radiologist.

54:17

Are we going to be replacing the future?

54:19

Let's say in the next 20 years or even closer.

54:21

I don't think it's going to happen.

54:22

I don't think it's possible. Um, I think medical

54:26

imaging is too complex. You know, a big part of

54:30

my job is talking to clinicians about their

54:33

patients and what they see in the findings.

54:34

Oh, it's very common that there's already

54:37

you know, I read a CT scan on the patient. The

54:39

report's in there. They've read the report and

54:41

now they want to call me and talk about it.

54:42

They're like, oh, what do you think about this?

54:43

What if we did this?

54:44

This is so I think we are an integral part of

54:47

taking care of the patients, conversing with

54:49

the clinicians about what's on the imaging.

54:51

It's not just a checkbox. This is pneumonia or this

54:54

is not pneumonia thing that we do, and I don't see

54:57

computers replacing that really probably ever. Um

55:05

Next question is radiomics or other AI

55:07

able to look at the symmetry of the study?

55:10

A large part of what I've seen in the lecture

55:11

so far is comparing one side to the other,

55:14

perhaps looking at what is absent from one side.

55:15

Oh, no, no.

55:16

So, um, they can. They don't, I mean,

55:20

symmetry is something it could look at,

55:22

but it's certainly not the only feature.

55:24

So, radiomics itself

55:26

doesn't even look at the, generally

55:28

doesn't even look at the whole image.

55:29

It segments out the lesion.

55:31

So, say there's a tumor or a lung nodule and it

55:35

looks at just that nodule outline, the nodule,

55:37

or maybe the surrounding borders or whatever,

55:39

um, and then extracts features from that.

55:41

From that.

55:42

So it might be looking at the size or the shape

55:44

or, you know, how bumpy, how, how much variation is

55:47

there in the pixels from one pixel to the next pixel?

55:50

Um, does it form patterns, things like that.

55:52

Um, so it's looking at really sophisticated

55:55

high-level mathematical correlations and

55:58

not just the symmetry versus, um, asymmetry.

56:04

Uh, some of these are asking the same question.

56:05

Um, I'm trying to, I don't find CAD useful, and

56:11

MAMO is AI different than CAD segmentation.

56:13

So I think really what's happened is CAD,

56:18

whether it be the computer-aided detection or

56:19

diagnosis, has really sort of taken on a new

56:23

life as it's been merged with machine learning.

56:25

So we can call it machine learning, or we can call

56:27

it CAD, but it's basically a computer program either

56:31

finding abnormalities on a study or making predictions

56:34

about whether that abnormality is cancer or not.

56:38

I think, um, it is always going to be, um, sort of a

56:46

questionable benefit because there aren't a lot of,

56:49

um, mammographic lesions that radiologists miss.

56:54

There aren't a lot of pulmonary

56:55

nodules that radiologists miss.

56:56

The ones we do tend to be very small

56:58

and have little clinical consequence.

57:00

So I think the reason a lot of people don't

57:01

find the detection tools all that useful is

57:05

because they're really not finding a lot of

57:07

things because we're not missing a lot of things.

57:09

So, well, the future may be that it can detect things

57:13

that we previously didn't know were important or, or

57:16

maybe they're pulmonary nodules on chest X-rays where

57:18

we do miss a lot. You know, if they could get better

57:20

at that, they're not very good at that right now.

57:22

Um, there might be a, a bigger upside

57:26

benefit, but I think the real potential

57:28

benefit is in evaluating the lesions.

57:30

So looking at a lesion and making predictions.

57:33

Is it not to replace

57:35

pathology? Not to say, okay, well, this is obviously,

57:37

you know, a squamous cell carcinoma, but to better

57:41

stratify the patient's risk.

57:45

And so if you can know that based on all

57:47

these features, this is a less risky lesion.

57:49

This is a lesion

57:50

we can serially follow at 6,

57:53

months versus this is a lesion.

57:55

This has got a lot of suspicious features.

57:57

And we do that already by what we see.

58:00

So if I see a speculated lung lesion, you know,

58:04

that's going to make me a little more worried that

58:05

it's cancerous, whereas if it's a nice smooth round

58:07

lesion, you know, less worried. So we already have some

58:09

features like that. But if the computer can extract

58:12

more features that can even do a better job and help

58:16

us categorize them even more, I think we will find these

58:18

things to be increasingly useful. Um, but we'll just

58:23

have to, you know, a lot of that's still future. One

58:25

of the things I really think is interesting about, um,

58:29

the field of AI and its application to radiology

58:32

right now is we are on the early cusp of everything.

58:35

So there's about, I'm just going to say out loud, maybe

58:40

20 years' worth of people training—people say train

58:45

probably right now, plus or minus 10 years. That group

58:47

of people are going to just have a phenomenal career

58:52

um, watching as these tools get developed and tested

58:55

and incorporated into our practice. Really at the

58:57

infancy, and we don't really know what's going to happen.

59:00

I do think there's going to be more and more things

59:02

being um, merged into our workflow that we're

59:06

going to start working with. Exactly which tools

59:08

are going to be the most beneficial, um, and which

59:11

tools are not really going to be all that useful.

59:13

It's going to be, um, hard to say. Um,

59:19

uh, so I'll just do one more question here

59:20

because we are pretty much out of time.

59:23

Uh, here's a good one.

59:25

What are some ways that the current radiology residents

59:29

can do to help you to AI-powered radiology future?

59:33

So it's really, um, a lot of things that

59:37

are available to you if you're interested

59:40

in, um, AI and the future of radiology.

59:44

If you are interested in being on the, uh,

59:49

sort of programming side and doing sort of

59:51

making AI models, it has become incredibly

59:54

easy to, um, to make your own AI model.

59:59

Um, so, someone even asked a question about this, but,

60:03

um, their programs, uh, TensorFlow, Keras, they work

60:07

together, they use the Python programming language,

60:10

all of which are open source and available, um, and

60:13

there are tutorials online, there are datasets online,

60:16

there's so much stuff you could get up and doing that.

60:19

That's for a very small subset of you that would

60:20

maybe be interested in doing that sort of thing.

60:25

But there are a lot of other

60:26

things that need to be done.

60:27

First of all, the world of AI research has

60:31

very little understanding of what we as

60:34

radiologists do or what we look for in images.

60:36

They are used to, in my opinion, a lot of

60:39

cat versus dog type categorizations, where

60:42

there's a very clear cut answer and they have

60:45

really low noise images with vibrant details.

60:48

We are used to working with a lot of uncertainty

60:53

in images at the edge of noise because they're trying

60:55

to keep radiation dose down, and there's a lot of

60:57

uncertainty about what the patient's diagnosis is.

60:59

They have an opacity, but I don't know if it's

61:00

pneumonia or if it's a contusion or whatever.

61:02

So there's an important role for being able

61:06

to help these AI researchers correctly label

61:10

images, create large datasets, you know,

61:13

get these images together so that they can

61:15

accurately train models that are working.

61:20

And then there's gonna be a whole world of, and there

61:22

is right now, a whole world of testing out these tools.

61:24

So companies will loan you, potentially, their

61:28

systems, or of course, they'll try to sell them to you.

61:30

So, but, you know, incorporating these, so maybe

61:32

somebody's got a new tool for evaluating, um, pulmonary

61:36

nodule growth, and it can look at two different

61:38

CT scans and it can do a volumetric evaluation of

61:41

the nodule and then determine whether it's growing.

61:44

Those are things like that that could potentially

61:45

be incorporated into your workflow. So they're either

61:47

available now to you, or they will be soon, so you

61:49

could be a person working with these models

61:52

that are sort of a little more further down the

61:54

pike, you know, sort of on the verge of being released,

61:57

or maybe being studied whether they're

61:59

useful for being released, and you could study that.

62:01

So there's just so much that you could do.

62:03

It's really an exciting field, but it is

62:05

definitely going to overlap all of our lives and all

62:07

of our careers over

62:08

the next several decades.

62:12

So we're out of time.

62:13

I want to thank everybody so much for

62:15

listening. I hope you got something out of this.

62:17

I know it's sort of a broad overview.

62:18

It's hard to cover anything too deep, uh,

62:20

when we talk about it at this level, but I

62:23

hope you find it useful, and thank you very

62:25

much for tuning in. Perfect.

62:27

We bring this to a close.

62:27

I want to thank you,

62:28

Dr. Donnelly, for your time today.

62:29

We really appreciate it.

62:30

And thank all of you for participating in this

62:32

noon conference, and remind you that it will be

62:33

made available on demand on MRIonline.com.

62:36

In addition to all previous noon

62:38

conferences, please join us tomorrow.

62:40

Dr. David Uson will be with us for

62:41

the noon conference on salivary glands.

62:44

You can notice that on MRIonline.com.

62:46

Uh, and follow us on social media for updates

62:48

and reminders on upcoming noon conferences.

62:50

Thanks again and have a wonderful day.