Sign In

Vinay Prasad on Cancer Screening


Intro. [Recording date: August 2, 2023.]

Russ Roberts:Today is August 2nd, 2023, and my guest is oncologist and professor of epidemiology Vinay Prasad of the University of California San Francisco. This has Vinay’s fourth appearance on EconTalk. He was last here in January of 2023 talking about the FDA [Food and Drug Administration] and the death of duty. Vinay, welcome back.

Vinay Prasad: Russ, such a pleasure to be here.


Russ Roberts: Our topic for today is screening for cancer. Screening seems like an unambiguously wonderful idea. The idea is to catch the cancer before it’s manifested, when it’s too late, when you happen to notice it in a very unpleasant way. And, it seems like a great idea to catch it early. But it’s complicated. Why? Why is it complicated?

Vinay Prasad: It’s something that everyone is interested in, and we’ve got a number of blood-based companies in this space, but it’s also one of the most tricky things we do in medicine. And the short answer, Russ, is that when we talk about cancer, what we’re typically talking about is what the pathologist tells us they found on a biopsy.

So, if somebody comes in with a lump in the breast and you biopsy it, or a polyp in the colon and you cut it out, and somewhere in that specimen they see that the cells, that they’re invading the basement membrane, they look cancerous. So, it’s sort of a histopathologic–means how it looks like on the slide. The challenge is, of course, that just because something looks like cancer doesn’t mean we know what its behavior will be, how it will act in the future.

And, some of these lesions that we find are definitely the sort of lesions that are going to kill you. Some of the lesions are the sort of lesions that are going to kill you were it not for cutting it out in that moment. So, if you catch it early and cut it out, now it’s not going to kill you.

Some of them, they’re going to kill you regardless of whether or not you cut it out. It’s already spread, the damage is already done.

And then some of them are lesions that might not cause you harm in the rest of your natural life. And, that’s a very counterintuitive idea and something people called over-diagnosis.

And, the problem with screening is that it has to have the right balance of these things. You have to catch a lot of the cancers that, if you didn’t find it would’ve done something bad, but now that you found it, we have a good outcome; and not so much of the ones that they’re going to do something bad anyway. That’s just adding extra time, anxiety, to your life and not so much of the ones that aren’t going to do anything.


Russ Roberts: Now, related to this, of course, we’d like to know which of those kinds that the cancers are, but we have this thing called Stages: Stage One, Stage Two, Stage Three, Stage Four. Aren’t they a attempt to measure and quantify–not quantify–but rank or qualitatively assess the odds that it’s going to be bad for you?

Vinay Prasad: Yeah, you’re right. Stages are put forth by the American Cancer Society and they are broadly used for a few purposes. One, to track cancer over time. Are we seeing an increase in a certain stage of cancer, an increase in a certain cancer?

In the 20th century, we had a huge decline in gastric cancer, which we attribute to improvements in food transportation/refrigeration. We had a huge rise and fall in lung cancer, which we attribute to the rise and fall of smoking. So, these kinds of staging and cancer tracking systems are good for that.

Staging is also a way to delineate how many people are presenting with just the lump in the breast–sort of a Stage One cancer–and how many women are presenting each year with metastatic cancer–so, breast cancer that’s spread beyond the breast is Stage Four cancer. You can track both of those over time.

Now, you’re absolutely right that stage and prognosis are tied together. And in fact, a Stage Four generally has a lower five-year survival. Fewer people are alive at five years than Stage Three and Stage Two, etc.

But, staging is not a perfect system. I mean, Stage Four disease is not 0% alive and Stage One is not 100%. There’s exceptions. I mean, there are bad outcomes that happen in every stage. Let me just say it’s a crude risk stratification. There’s so many other risk stratification schemes beyond that, but this is a crude one.

Russ Roberts: And it’s–actually, what I said is not quite right. It’s not so much an assessment of what the nature of these cells and how they’re going to metastasize as more crude measures of where is it. And, it’s not just the size. The screening will often identify the size from a mammogram or other test. Right?

Vinay Prasad: Yeah. Staging often includes things like size, depth of invasion, the number of places it’s gone to, the specific places it’s gone to. And, if it’s spread in distant sites–that’s typically Stage Four.

The staging systems varies a lot by cancer. For instance, in testicle cancer, there’s only three stages, because the outcomes are so good. So, there is no Stage Four.

In anaplastic thyroid cancer, it’s so bad, there’s only Stage Four. It’s always Stage Four. There’s no other stage.

So, you’re absolutely right that it’s often related to the places where the tumor has gone. Although some staging systems use more complicated things like how it looks like on a fancy new scan called PET [Positron Emission Tomography] scan, or laboratory markers, or depth of invasion–how deep the tumor has invaded. These things all go into staging. And it’s constantly being revised every few years.


Russ Roberts: So, you have a powerful metaphor for helping us think about the complexity of detecting cancer. Cancer is obviously a very scary thing. It’s so scary we call it the C-word, sometimes. You don’t want to actually say the word out loud. I don’t know if that’s a healthy cultural reaction. My first thought is probably not. I’d rather go the other direction. We’re blessed to live in a time where we have better techniques than we had in the past for both screening and treatment. We’ll talk about that.

But, you have a metaphor for how we think about this mix of kinds of cancer and their likely outcome on us.

Vinay Prasad: Yeah, and I can’t even take credit for the metaphor. The metaphor goes back quite some time, and I’m not sure people exactly know who came up with it.

But, the metaphor is a barnyard metaphor. And, the metaphor is basically, like, imagine you’re a farmer and you have a barnyard and you have lots of different animals in your barnyard, and you want to find a way to keep the animals in your barnyard. And, that’s, I think–the idea of catching the animal before it leaves the barnyard is the metaphor for catching the cancer before it causes a problem.

And, one can imagine there’s three types of animals in your barnyard. There are rabbits, turtles, and birds. The thing is the fence, it’s going to be really good at catching those rabbits. They are hopping, they’re jumping, and when they get to the fence, they’re going to be stopped and they’re going to come right back to your yard.

The turtles–actually turns out you probably didn’t even need the fence. They’re moving so slowly that even in the next year or two, they’re not going to get outside your yard. This is how the metaphor goes.

And, the birds, meanwhile, are moving so quickly that no fence can stop them. They’ve already flown right out of your yard. And, those are also cancers.

So, the idea is that the turtles, the birds, and the rabbits are all cancers. Some cancers are so aggressive that even when you screen people, they have already spread.

And, in fact, Russ, I would just say that when you look at all of the screening tests we’re going to talk about today, and maybe the ones we’re not going to talk about, one thing to point out to the listeners is that no screening test reduces death from that cancer to 0%. So, we debate how well they work. We debate the benefits and harms.

But nobody debates the fact that you can get all the colonoscopies you want and there’s still a risk of dying of colon cancer. You can get all the breast cancer screening you want. There’s still a risk of dying of breast cancer. Typically, that risk is 80% of the risk. I mean, even the proponents think it only lowers cancer death by 20%. What that means is there’s a lot of birds. There’s a lot of birds.

And then the other thing, Russ, is we should have some humility in medicine. We don’t know how many turtles there are. And turtles matter a lot. Because, every time you find a turtle, you’re going to treat that person as if they had a rabbit or as if they had a bird. They’re going to get the full court press of treatment. But they may not have needed it, much of that treatment or even any of that treatment. And so, that’s just harm being inflicted on someone. So, this is the delicate balance of screening.


Russ Roberts: Coming back to your opening statement about–we see cancer, but we don’t always know the nature of those cells and how they’re going to spread–I assume there’s a lot of people looking at how we might distinguish turtles, rabbits, and birds. Because that’s huge. We’ve talked a few times in the program about prostate cancer. My dad had it. I’m 68 years old. I might have it. I think many men–my understanding is that many men at the time of death have prostate cancer. It’s just a turtle. And, your heart attack or your stroke or your pancreatic cancer kills you before the prostate cancer does. But, you have prostate cancer. Of course, tragically, there are prostate cancers that are rabbits: that if you don’t detect them early, you’re done. Others are birds: It’s too late.

So, I assume we’re trying to figure out ways to anticipate. I mean, another way to think about it is: in other medical problems, sometimes taking a wait-and-see attitude is the right approach. The scary thing is if you wait too long, what you see is it’s too late. So, talk about that.

Vinay Prasad: No, that’s absolutely right. So, you made many astute points. One point you made is that most men die with prostate cancer, not from prostate cancer. It’s absolutely true. Autopsy studies have gone back for decades showing that–it’s almost like every decile, that’s the decile of prostate cancer. So, 60-year-old men, 60% will have some prostate cancer on autopsy that didn’t have anything to do with why they died. 80% of 80 year old men, etc. It almost is at that level. Most men are going to have some of it, and it’s not going to be a problem.

Of course, there are some men who die terribly from prostate cancer; and we don’t want that to happen. It’s 2% of all male deaths. If there’s anything we could do to try to lower that, we would want to do that. And, that’s where the screening idea comes in place.

Now, you make a really excellent point, which is: Aren’t you trying to sort out what are the rabbits from the birds, from the turtles?

And yes, many people are. They’re using things like, in prostate cancer, MRI [Magnetic Resonance Imaging]. So, can I conduct an MRI of the prostate to get a better sense of things? They’re doing things like genomic analysis, proteomic analysis.

But, the one thing I would say about this whole space, Russ, is that: in order to figure out a molecular test that distinguishes rabbits from birds, from turtles, you need to link it to some gold standard. What is a rabbit? What is a bird and a turtle? And, that takes–the only gold standard is time.

So, I guess the argument I want to make is that some of the research I’m critical of is that: Yes, they’re finding things that they think predict more aggressive behavior, but they really haven’t answered the fundamental question, which is: Is this the tumor that if I cut out, the person is going to be alive and well at 85; and if I don’t, they’ll be dead at 57?

A number of people are investing in this space. One notable example is the Google AI and they’re training the Artificial Intelligence-Google Image Detection on mammography specimens to see: Can we find more cancers? And, in fact, they have a Nature paper that shows maybe they can find more cancers.

But, are they really finding more rabbits?

And, the answer is that you would need a gold standard way to tell me what’s a rabbit from a turtle from a bird.

And the gold standard way is to know that if this is the tumor I cut out, the person is going to live to 85. And, if I didn’t cut it out, they’ll be dead by 57. Whereas, if this is the tumor I cut out, so that’s a rabbit. If I cut out this tumor, they’re going to die at the age of 75 from leukemia. Either way, that’s a turtle. Or if I cut out this tumor, they’re going to die at 62 of breast cancer either way. And that’s a bird.

And, to do that, you really need longitudinal data. You need data sets that have tracked these tumors over time.

I’m not sure we have a lot of those data sets that are capable of this question. I think what it will take is prospective randomized studies–that’s what I always like to say.

And that’s a challenge in the space, though. I think there’s some knowledge challenges. But, Russ, you’re absolutely right. The holy grail is a blood test that tells you: You are the person; I know your future in two worlds, the world where I don’t do something and the world where I do do something. And that’s the holy grail. Can we find that from your biopsy?


Russ Roberts: So, let’s turn to some of the specific things that we know about screenings of various kinds. And, before we do that, we should say a couple of things, I think. And I’ll say them, and then you can either assent or dissent.

One is: The bottom line of a lot of this, unfortunately, is going to be that screening is not as effective as we might hope. That certainly doesn’t mean you shouldn’t screen. A lot of these results that show little or no effect for certain types of screenings are for the average person, not for the person with the genetic proclivity, not with the cancer in their family of a certain kind. You can give us some insight into that.

The other thing I want to mention is that I think in the background of our whole conversation, you’re a doctor who cares about numbers a great deal. I’m an economist. And, we pretend that we can be rational and objective and somewhat thoughtful in terms of what is fundamentally a risky and uncertain part of life, which is whether you’re going to be killed by a cancer.

And, I think the overwhelming attitude of most human beings who are not economists or oncologists who care about data–the overwhelming attitude is: Well, I’d rather know than not know. Better safe than sorry. And, I’d rather treat than not treat because I get the thing out of me. The idea of saying, ‘Oh, it’s a turtle, don’t worry about it,’ I think it’s very hard for both the patient and the family and loved ones of the patient. So, talk about those two things. One, what kind of data are we looking at here for which kind of population? And, secondly, the psychological issues that are part of this, ‘it’s a reality.’

Vinay Prasad: Yeah, so I think you made many good points. One, this is not medical advice. You should talk to your doctor.

Two, you make a point that people often make, which is this is about average risk populations. And in fact, most of the data we’re going to talk about which comes from large randomized studies or population surveillance data is for average risk populations. I always make the point that in some ways we’ve failed the high-risk people because those are the people we should have been doing special trials in, but we haven’t. And so, we have a mantra in medicine: ‘Well, if you’re high risk and I don’t have data, the answer has got to be more screening.’ But, I’m a little skeptical of that narrative. I’m not sure that that’s the case. It could be that they have even more of the harms of screening; and unfortunately they have more birds maybe, and you’re not able to change the natural history.

I guess there’s two more things–I just want to say, the psychology. The psychology part is: Look, especially in the tech world where there’s a lot of enthusiasm for screening, the psychology is: Information cannot be bad. All information is good. The only answer is how you use that information.

Only a doctor will tell you that information can be terrible. Information can sometimes rot at you. It can change your behavior. It can be not useful information, but it can cloud your vision of yourself. You can go from thinking of yourself as a healthy person to a sick person, even though nothing has changed, and even though you’re going to die of the same day at 75 in a car accident, for instance. That’s something screening can do to you. You can end up getting chemotherapy. I mean, once you’re told you have cancer, it’s very difficult to say, ‘Okay, I’m just going to watch it.’

I think we’ve made progress in that space. We’re doing a better job than we did 20 years ago, but it’s still difficult. Imagine telling me, ‘You have prostate cancer and we’re not going to cut it out. We’re just going to let it sit there and watch it.’ I’m like, ‘Oh my God, it sounds terrifying. What’s it going to do? It can only do something bad.’ So, that’s a huge psychological barrier.

The last thing I want to say is: there are two more things we should introduce as concepts upfront. One is this idea of competing risk. A colleague of mine always says that we forget with screening, but each individual screening test is at best going after 1% to 4% of the things that kill you. In other words, most of what kills us is cardiovascular disease. I mean, that’s the reality. And then you screen for breast cancer or prostate cancer, which is 2% to 3% of all deaths or colon cancer, which is just a few percent.

And so, the first thing you should have is the humility to know that there’s so many other things that could kill you that you’re not even looking at in this moment.

The next thing is–competing risk is: if I get a colon polyp found and you cut it out, but two years later I have leukemia and I die of leukemia five years later, did you benefit me? Maybe, if that colon cancer would’ve caused a problem in those seven years. But, if not, you didn’t benefit me. I mean, you just made me worried about my colon when that was really not what was going to get me in the end. And, that’s sort of a competing-risk problem that really makes it difficult for cancer screening because they’re typically done in older people who have a lot of competing risks.

And then, the last thing I want to introduce is–you said this really well. Our screening is better: It’s able to find more things. Our treatment is better. One principle of screening has always been that screening tests work really well if there’s a differential treatment effect. In other words, if you find it early and you treat it–like a breast lump–you can get rid of it forever and the benefit is big. But, if you find it late and it’s already spread distantly, our drugs are very ineffective and there’s not much we can do. And that difference in the treatment effect from early to late is what we’re exploiting in a screening test.

As one example, in testicle cancer since the 1970s and 1980s, we can cure testicle cancer even when it’s spread everywhere. Like Lance Armstrong. Our cure rates are like 95%, 96%, 97% for metastatic testicle cancer. So, actually because we can cure it so well when it’s advanced, there’s no longer an impetus to find it early.

And the USPSTF–United States Preventive Services Task Force–says: Don’t examine your testicles every month in the shower. It’s USPSTF Grade D, because you’re only going to find incidental things and lead to losing a testicle, which is the way we actually–we don’t biopsy a testicle; we actually just remove it. And, even if it presented late, you still have an excellent result. So, there’s no differential to exploit.

And finally, the thing I’d say is our treatments are getting better for breast cancer, prostate cancer, etc. and the advanced disease, which many of us believe is eroding whatever benefit of screening was there in the first place.


Russ Roberts: I’m going to mention two other things that are in the background of this conversation. One is, of course, there’s no free lunch. A lot of people’s attitude toward screening is, ‘Well, if you find it,’ first of all people say, ‘Well, you don’t have to treat it.’ Which of course, emotionally is very difficult. That’s one of the examples you gave of the information not always being helpful.

But, most people forget that in many cases there is downside risk both from the test and the treatment, if it’s a turtle. [?It doesn’t?] matter what it is: actually just the treatment itself is often you’re going to be taking poison because you need to get rid of the cancer and it’s going to be poisoning other things as well.

So, this idea that tests are free because worst case scenario, you don’t find anything. No, that’s not the worst case scenario. The worst case scenario is you endured the test. The second part of it is you had a false positive. It said you had something in fact you don’t have. You start a treatment that has a destructive component. Or, worse–not worse but alongside with that–sometimes the test itself leads to damage. And, it’s small. A friend of mine has a test coming up and I asked him, I said, ‘What are the risks of the test itself?’ He said, ‘Well, my doctor reassured me that it’s only 1%.’ I can’t remember what the numbers. Let’s say 1% of the times that the scope pierces something it’s not supposed to pierce. I said, ‘Iell, wouldn’t be so interested in that national rate. I’d kind of want to know the rate of the doctor you’re seeing, because he knows that number. And if it might be 5% for him.’ And that could be because he looks at harder people, difficult cases–a lot of reasons you have to take those data, consume those data, thoughtfully.

But, I just want to put that on the table.

The second thing I want to put on the table is–and you and I are more aware of this, I think than most people–many doctors, almost all of them, are loving, caring people who got into the profession they’re in because they want to cure and make people healthier. They also make money from these tests, or different people in the profession–in the industry–make money from these tests. And so, there’s an enormous machine encouraging these tests that–and it’s a Bootlegger and Baptist problem, meaning you feel good about yourself if you’re pushing the test because screening is good; and of course you do benefit, personally, but these two things work together. But it’s actually a little more–it’s not so healthy that there’s an enormous personal and financial incentive in some of these situations. So, talk about those two things.

Vinay Prasad: Gosh, I mean–it’s really well put, Russ, and I agree with everything you said. One, some screening tests have harm in and of itself from the screening test. For example, you gave colonoscopy and the risk of perforation of the colon, which some people put at one in 10,000. In this one study we’ll talk about it was zero. But, that’s because everyone was awake during the procedure, which might lower the risk of that; but it causes another risk of you remember what they’re doing to you.

So, that’s a risk–perforating the colon. And, we’ve all seen the very, very rare case where that spirals downward. Yes, you perforate the colon, a lot of people get better. But every once in a while someone deteriorates from that. Every once in a while someone’s going to die from that. And it’s a death that wouldn’t have happened otherwise. It’s very rare, nothing to worry about, but it happens.

You talked about–you used the word false positive. I would say a different word, which is that you found something that looks like cancer. So, maybe people call it a true positive, but it’s not the kind of cancer you wanted to find. So, it is essentially a false positive. It was a turtle. And then, you’re subject to a battery of treatment that often includes chemotherapy, administered IV [intravenous/within the vein] for breast cancer, or radiation administered to the prostate.

And, I have definitely seen patients who–mammogram found the lump. They had surgery, radiation to the breast, and chemotherapy. And then three years later they get leukemia, which is a known side-effect of the chemotherapy they got. So, it could be treatment-induced leukemia. And that’s a very grave diagnosis. That person is dead. Would they have died without the mammogram? I don’t know. But, that chain of events was started by screening. And, if that was a turtle, maybe you have shortened their life.

I’ve also seen cases where you screen someone for prostate cancer and then you radiated the prostate, but now he has radiation-induced proctitis or inflammation of the bowel in the rectum. And he has painful, bloody stools, and it’s lasting for month, after month, after month. He’s suffering. Is his life extended or is his life made miserable by this?


Russ Roberts: And of course, what we care about–at least when we’re thinking about it somewhat rationally–is: how many of those are there versus how many of the outcomes where we save someone’s life?

I don’t think I’ve ever met anyone who–I’ll say it more in a positive way. People I’ve met who have screened discovered something and retreated will always say that the screening saved their life. I can think of three personal friends of mine who believe that. I’m an economist. I usually smile and say, ‘I’m so happy you’re with us.’ But, in the back of my mind, I’m thinking, ‘You don’t know that.’

And, what we’re going to talk about–really we will, listeners–we’re going to talk about the fact that when you have a large group of people, you can get a measure of how frequent these kind of events are and get a much better idea of whether the screening saves your life.

Say something, though, before we move to the actual data. Say something about the financial incentives.

Vinay Prasad: Yeah. No. And those two things go hand in hand. I think when you talk about the incentives for screening, it’s both of the things you mentioned: the financial and the psychological.

So, of course, everybody who has had a lump found from mammography, and most people who’ve had a prostate cancer found or a polyp clipped, they feel like they benefit. Even if they’ve suffered some complication. Even if they’ve had to go through an arduous treatment, they’d still feel like, ‘Wow, were it not for that screening test, I wouldn’t be here today.’

But, as you point out, Russ, they don’t know that to be true. They don’t know their individual counterfactual. Unfortunately, the only way to know that is randomized data with lots of people, so we can actually start to count and tally these things up. Which thankfully we do have some. But, so, that psychological drive is so powerful that if you think you benefit, of course you don’t want to hear anyone criticizing that test. And, I hear that a lot.

The second thing is the financial part. Cancer screening turns a lot of healthy people into patients. That makes a lot of money for the whole system. In fact, some of these juggernauts of screening campaigns, they massively enrich hospitals and providers and practitioners.

Everyone is sensitive to that because nobody sets out to be a gastroenterologist just because they are greedy. That’s not true at all. I mean, they are good people who want to do good. But they have to acknowledge that a huge chunk of that specialty is the revenue that comes from that screening colonoscopy. And so, when that revenue is threatened, as in the recent trial called NordICC–which failed to find a benefit on colorectal cancer mortality–a lot of people are going to be very defensive.

And, it feels a lot to me, Russ, like motivated reasoning. After the fact they say, ‘This study is wrong.’ Well, of course you do it every day. It paid for your beach house. And, you feel like it’s doing good. And so, of course it’s hard for you to consider that maybe it’s not.

And, Russ, the thing I always tell people is–for a doctor–the methamphetamine of being a doctor, the most addictive thing in our minds, is you do something that you really think benefits your patient and you get a little financial bonus for that at the end of the month. And, that combination of money, plus you’re doing the right thing, that’s the methamphetamine of being a doctor.

And so, those things are super-addictive. And we get addicted to them. And it’s very hard for us to think clearly about those substances and procedures.

Russ Roberts: Yeah, we’ve talked about this on the program before: that, in these kind of complicated situations where there’s uncertainty and a financial stake, it’s often helpful to ask the doctor–let’s say you’re helping your mother or dealing with some health crisis. You say to the doctor, ‘Well, if this was your mother, what would you do?’

Sort of forcing, obviously it’s not his mother, so he probably can evade that technique if he wants to, but I think it has a psychological effect on the doctor.

But of course, if you’re doing these procedures, you’ve probably convinced yourself.

In the case of economic regulation, it’s called cognitive capture: the idea that you advocate for a regulation that benefits you because, well, you could talk yourself into it. Exactly what you were referring to.

So, I would think–especially in the case of practitioners who benefit from these procedures–it must be extremely hard for them to step back from their own stake in the matter and try to give you a measured piece of advice.

Vinay Prasad: Absolutely. And, the last thing I’d say on this topic, Russ, is–because I think that’s really well put–is that, you can be a great gastroenterologist, you can be a great urologist, you can be a great cancer doctor, and not have spent a lot of time thinking about cancer screening, which is a program run at a population level that exploits different principles of epidemiology and is really something different than the individual doctor’s experience.

Sometimes people tell me, ‘Well, I’ve seen the person it cured.’ And it’s the same fallacy that you made. Right. You didn’t know what would’ve happened to them.

And, the only way to really know this is to look at large population studies and to put your economist hat on. To put your epidemiologist hat on. And, I think that’s something that, unfortunately, doctors are not trained in. So, that’s yet another bias. So, you have the–‘Everyone says it does good, the financial bias; and that I’m not really trained to read these studies’ bias.’

Russ Roberts: Yeah. I’m just going to add one of my favorite insights from Nassim Taleb, which is: you don’t ask the carpenter who built the roulette wheel how to play roulette because he might be the world’s best carpenter, might be the most beautiful and balanced roulette wheel and fair, but the carpenter might not know very much about statistics, and it’s a separate thing. And so, I think a lot of people trust their doctors because they assume they are the expert, but they are the expert in certain pieces of the experience, but not all of them, and many of them are not trained in risk analysis.

Vinay Prasad: Absolutely right. The one I use is that you don’t ask the guy who tears your ticket at the movie theater what project you should produce in the next movie cycle just because they–but I like your example better. I like Taleb’s.


Russ Roberts: Okay. So, let’s talk about the data, which is somewhat sobering. Not somewhat–it’s extremely sobering. Normally, there’s a bias in empirical work toward finding something. Finding nothing is usually not the road to getting a paper published. No one wants to find out about things that don’t work. But, in medicine, fortunately, a lot of people have taken serious looks to see if something actually works. And, when they find nothing, it’s very publishable, unlike[?] other interventions. So, in this case, it’s a pretty bleak story. So, you could try to summarize it–I mean, the main thing that I think to focus on for listeners who haven’t consumed these studies in any detail is to emphasize the point about all-death mortality, because that is not the first thing you’d think about unless you’re an economist, to be honest.

Vinay Prasad: Yeah. So, do you want to do it cancer by cancer or do you want to do it like a broader summary of the whole space? How should I get into it?

Russ Roberts: We can go cancer by cancer. I’d say we should do a few–a few of the more common ones where colonoscopy, mammogram. The PSA [Prostate-specific antigen] used to be one that I think people have moved away from for prostate cancer. But, we can start with those two: colonoscopy and mammograms.

Vinay Prasad: So, I guess we could start with mammography. Mammographic screening developed, now, about half-century ago. And the idea is simple: that if a woman has a yearly breast radiograph we’ll be able to find cancer and maybe cut it out earlier than before she even feels a lump, which is hopefully before it spreads; and all these–so, that’s the idea.

We’ve had at least seven large randomized control trials [RCTs] of mammographic screening, different age groups. We’ve gone down to, I think 39 years old all the way up into 69 years old. There are differing recommendations for women between the ages of 40 and 50, and 50 and above. There’s a lot of debate on what’s the upper-bound age. Should you stop at 75 or 70, or should you stop at 80, or something like that? People have different feelings there.

At some point where, sort of outside of the randomized evidence, what does the randomized evidence generally show? There’s two things they look at. One is you randomize tens of thousands of women to annual screening or biannual screening or something like that using the best screening machines of the time. And then, tens of thousands of women are randomized to the control arm of ‘No recommended annual screening.’ You follow them for years. And then, the two things they look at is how many women died of breast cancer and how many women died for any reason. Okay: that’s the death from all-cause.

I have been a big proponent, in my career, that we really need to be looking at that all-cause death for some of the reasons you described, Russ, which was that what if you got the mammogram, you found a turtle, you got treated for the turtle, you got chemotherapy; and you had a leukemia two years later and died from that. Well, you want to penalize the screening arm for that harm, if it was in fact related. You wouldn’t do that if the only endpoint you were looking at is dying from breast cancer. You might miss this leukemia diagnosis and think it’s unrelated. This is called the ‘slippery linkage’ bias–the link gets, slips away.

The other reason I like looking at all-cause death is that ultimately it’s what patients care about. People say–you want to say on something on that?

Russ Roberts: Well, it’s the only thing we care about, really.

But, I wanted to say one thing about the leukemia. Of course, we don’t fully understand, always, what the side effects of various diagnostic techniques and treatments are. So, in your talk on this–we’ll link to you have a wonderful YouTube summarizing the mammography data–you mention about: Well, you wouldn’t expect it to do X. But, of course, we don’t really know that.

So, living under a diagnosis, say, of cancer might stress your heart in ways that are not–we don’t fully understand the stress of that, the emotional pain. So, I just think it’s really important. Important, it’s not the right word. You have to look at all-cause mortality if you have any confidence in the fact that you have a randomized trial.

Vinay Prasad: Absolutely. I completely agree with that. And, I’ll give you a piece of data to bolster your argument, which is that there are studies that show that in the immediate aftermath of a prostate cancer diagnosis for men, there’s a slight increase in suicide.

Imagine that suicide, if it’s attributable to being told you have prostate cancer, even if it’s a very, very small increase, that should be a penalty that the screening test–I mean, those are deaths that wouldn’t have occurred if you didn’t tell the man that. Okay, so it should look at all-cause mortality in my opinion.

If you look at all-cause mortality in all of the mammographic screening trials put together, you will find there’s just no signal there. It’s just not budging all-cause mortality. It looks pretty null. Confidence intervals crosses[?] one; the actual effect size is like 0.99. It’s as close to just totally null as it gets.

Now, proponents of mammography say, ‘Well, that’s unfair. You don’t have the power to find a difference.’ I mean, there could be a difference that exists. The studies just aren’t designed and sized for that. And, in fact, we’ve done some power calculations. I had a paper on a few years ago. You’d probably need, like, 3 million women randomized and there are usually in the tens of thousands. Put together, maybe it’s like 300,000, 400,000 women. You need 3 million.

But, the point I want to make is: if you need 3 million women randomized to see the effect, maybe it’s a small effect. I mean, maybe it’s something that might not be worth a $100-billion medical campaign. That’s something that we could think about.

The next thing I’d say is: if you look at just the reduction in death from breast cancer, I always like to separate the trials into this thing you’ve alluded to, which is what we call adequately randomized or suboptimally randomized studies. This is not my nomenclature. This comes from the Cochrane Group–that is the independent research group looking at the studies.

And, it’s exactly as you say: they look for some endpoints that they think are implausible, implausibly related to screening. So, they look at dying for something other than breast cancer. And, if there’s severe imbalances between the two groups, they think there’s something a little bit odd about randomization. And, in fact, that’s true for some of the very older studies.

And I guess listeners may not know this, but in the history of medicine, the first randomized controlled trials came out in the 1940s; and now in 2023, it’s a juggernaut of randomized studies. I mean, we are putting out maybe tens of thousands of random, maybe a hundred thousand randomized studies a year. It’s just a machine. We’ve gotten a lot better at randomization. We used to have envelopes that you would randomize people and open the envelope. Sometimes people would hold them up to the light to try to subvert randomization.

Now we have computer-generated automatic telephone randomization. Pretty much everything about the design and conduct of studies is better today than it was when many of these trials were run–when these mammographic screening trials were run. Many of them use things like Mailer. So, people are invited by mailed invitation to participate in the program. There can be some biases because the group of people who didn’t show up, but the group of people who was assigned to the control arm may include some people who are already deceased, for instance. That’s been cited as a problem with some of these older studies.

This is a little bit long-winded, and so I’ll just cut to the point. The point is that even the biggest optimist about mammographic screening would probably cite a 20% reduction in dying from breast cancer. They can’t claim a reduction in dying for any reason.

And, that means 80% of breast cancer deaths are not avoided. And, this is largely using studies that happened decades ago. And, probably a lot of the changes in breast cancer treatment have eroded that benefit. So, I think that’s what proponents would say.

A critic, like me, would say, is that I truly have no idea if I advise a woman to undergo this screening test if she’s going to live longer. I just don’t know. And I don’t know if she’s going to live better. And, I have to ask myself, what are we doing as a profession that we cannot answer that most basic question?


Russ Roberts: So, you say you just don’t know. Isn’t it a little stronger? Isn’t it: As far as we know, there’s no effect? It’s not we don’t know if it works or not. We’re saying with the evidence that we have, it doesn’t work. It does not extend the lives of the average person. Again, there may be many categories of people who should screen and it will have a positive impact on longevity. But, for the average person, there’s no evidence that it works.

Vinay Prasad: That’s how I would put it. I’d say there’s no–I mean, this is where the statisticians quibble about ‘the absence of evidence is not evidence of absence’ sort of thing. But, I would say that: Look, there’s a conventional burden in medicine, which is if you do something for 50 years, you got to prove it works. And you never proved it.

You take all the trials, you put them together, there’s no all-cause mortality benefit. Yes, maybe they’re underpowered. Maybe you need to run 3-million-person randomized studies. But, until you do so, maybe you are the one who should cool your rhetoric. You don’t have evidence that you live longer or live better from doing this.

And, I also think more to point, that if you told someone doing this what I’m saying right now–that actually if you pooled[?] all the studies, there’s no evidence, women live longer–would they really want to do it?

And the answer is. We don’t know because that’s not how we counsel people. Too many women have mammograms because the doctor says, ‘All right, now how are you doing, Susie? Okay, last thing before you go: we’re going to schedule your mammogram. You turned 40. So, we’re just going to go ahead and schedule that, so you can schedule that at the front desk on your way out. Have a great day.’ That’s the consent. That’s so inadequate.

Moreover, a lot of hospitals, they incentivize the doctor. If they have 85% of women in the target age group getting mammograms, they get their yearly bonus. And, if it’s 82%, they don’t. And so, they’re incentivizing people.

Russ Roberts: How big is that bonus?

Vinay Prasad: I would say that different–I mean, it varies by institution or program–but I’ve heard that it can be as much as 10% to 15% your salary.

Russ Roberts: That’s a lot of money.

Vinay Prasad: And, I don’t want to say it’s just for mammograms. It’s usually for some composite of a number of your patients who have low blood pressure, adequately treated and the sugar is treated, and then they get the mammogram and the colonoscopy.

But, to me, this is one of the most gray areas of medicine, screening. It’s one of the most preference-sensitive areas, which we’ll talk about with this gentleman. Like, different people can have different preferences. Someone’s preference could be to, ‘I don’t want to do the colonoscopy and I’m willing to take a little bit of risk. That’s my life.’

And, when you start to incentivize ambiguous and preference-sensitive decisions, you’ve got a problem. These should best be tackled with shared decision making. Be totally honest with the patient, what we know, what we don’t know.

And, I say ‘patient’–that’s the wrong word–person, because this is somebody who’s healthy. They’re not a patient, they’re a person until you make them a patient.

I think, and the final thing I’ll say, Russ, is in my experience, whenever I talk to people one-on-one and I tell them the way everything I’ve been saying so far–what we know, what we don’t know–most people say, ‘I had no idea that that was the case. Nobody ever told me. And, had I known that, that at best it’s 20% reduction in breast cancer death and there’s no signal in all-cause mortality, I would never have done it. What the hell? I’m not coming in here for that triviality.’

Russ Roberts: And, it’s a 20% reduction in a very small number, just to be–

Vinay Prasad: Very small number. Maybe can I put it in perspective?

Russ Roberts: Yeah.

Vinay Prasad: So, I think there’s a graph from–this is coming from memory, so a listener can check me, but I think it’s called ‘The Ppinion of the Swiss Medical Board,’ New England Journal, maybe in the last decade, and it analyzes a hypothetical cohort of 1,000 or 10,000 women undergoing screening. And, the gist is that a woman undergoing screening–versus not–there’ll be five deaths from breast cancer that go down to four deaths from breast cancer, but there’s 39 deaths from other reasons, or 39 or 40 deaths from other reasons in both groups. I say 39 or 40 because we don’t know we improve all-cause mortality.

And so, to put that in perspective, this graphic also asks women their perception of the risk of breast cancer death and then the reality. And, it shows massive differences in that perception and reality: that, women think breast cancer death is a huge probability, but the reality is it’s only a small fraction of death. So, 5 out of 45, something in that ballpark.

Russ Roberts: That’s because–

Vinay Prasad: Going down to 4 out of 45, yeah–

Russ Roberts: And that’s because it’s relatively rare and we have better ways of treating it, whether it’s found early or late. Correct?

Vinay Prasad: Yes. And so, what I would say is that every single one of these mammographic randomized studies, in my opinion, likely overstates the benefit because, since those studies were conducted, the treatments have gotten so, so much better.

And, just like the testicle cancer example, as your treatment for advanced disease gets better, that differential effect that you’re exploiting for screening is smaller.

And, I suspect that–and in fact I’ve written this, we’re going to submit the paper–I think we need a new study. I mean, we need to study that’s up-to-date with what we’re doing now.

And, let’s just talk about breast cancer. I mean, breast cancer has probably had 30 new drugs approved. Breast cancer has improvements in surgical technique. The radiation we give for breast cancer, it’s a different machine; it’s given better than it ever was given. Our ability to do scans and detect small breast cancer–stage people more accurately–has gotten better. Supportive care has gotten better. Anti-nausea medicines have gotten better. Our drugs have gotten better. It’s a different disease than it was in 1985, thankfully. And that, in my opinion, will erode whatever benefit of screening was there in a study that was finished in 1985.

Russ Roberts: That’s a fantastic point. Would you tell a woman in your life who had no genetic predisposition to cancer, not to get a mammogram? A woman, a loved one, a spouse, a close friend, and they come to you and they say, this is your specialty, but what should I do?

Vinay Prasad: My answer is–of course, I like to talk to people, because I do think, Russ, there are different types of people. There are always those people who even if the chance that something will help them is very slight, they want to do it; they’re willing to give it a shot. And, there’s other people who are like, ‘Look, even if skipping that morning cup of coffee would help me live an extra month, there’s no way in hell I’m skipping my morning cup of coffee,’ kind of people.

But, my honest answer is: Yeah, there’s lots of screening tests that I personally wouldn’t do and I would advise somebody I care about, ‘You don’t have to do it if you really don’t want to.’ And, mammography is one of them. We can talk about colon cancer screening, but I probably personally wouldn’t do a colonoscopy. I’m not yet at the age group they want to sink their tentacles into me, but they’re coming for me. They’re coming for me and I’m probably going to decline.

And, I’m not going to do a PSA screening, personally. I’m not a smoker, so I’m not eligible for lung cancer screening. But, if I were a smoker, I definitely wouldn’t do lung cancer screening, which I think is extremely unfavorable. And, if a loved one came to me and said, ‘Do I have to have my mammogram? I’m 42.’ I’d say, ‘Absolutely not, if you don’t want to; and you can read the data and make your own decision.’ And, I’d be happy to advise somebody–like, I’m not persuaded it should have ever been a program.

Now, the reason I think people are reluctant to say that is that many doctors will say that privately, but of course people worry that if you say that publicly that you’re crossing some line. But, I don’t know. What is the line we’re crossing? We don’t believe, we’ve never been persuaded that it makes you live longer, live better. So, of course you wouldn’t recommend it to someone you cared about because I’m not persuaded of that.

Russ Roberts: Say something about the paternalism that you’ve talked about.

Vinay Prasad: There was an old ad by the American Cancer Society that was literally taken out in newspapers that said, ‘If a woman hasn’t had a mammogram, she needs more than her breasts examined,’ meaning that she’s crazy. She needs her brain examined. This is the American Cancer Society.

The point I want to make is that uninformative persuasion, coercion, paternalism has been the norm of cancer screening programs from the 1970s to almost the last five years. We have–even to the present day–I mean, we force people to do this. I mean, we do not have discussions. We just put you in the machine and the widget comes out the end. People who are listening will have gone to doctor’s appointments where they walk out and the last thing it says on the paper–maybe they didn’t even talk about it–is: Show up for a mammogram and go to the lab and get your PSA. That is paternalism. That is the doctor telling you that, ‘We need to do this. And, it pretty much doesn’t matter what your thoughts or preferences are on this issue.’

That, to me, is a problem. That’s the root problem of cancer screening, because we could have a broader philosophical debate on paternalism. I’m actually, I’m something in the middle where I think there is a role for paternalism wometimes in medicine. People sometimes want you to make the decision for them, especially in tough times. I mean, that’s been the case. I think sometimes people burden–I mean, sometimes somebody has a tough situation in cancer and I see the trainee putting everything on the family, burdening the family with a choice that the family can’t make, and that we need to shoulder the burden and go in there and say, not ‘This is a choice,’ but ‘Here is my recommendation as a doctor.’

Having said that, this is a place where I think paternalism is off the charts inappropriate, cancer screening. Because it’s healthy people, and the evidence is so disputed for decades by many experts in top publications that I think the only acceptable answer is some shared decision-making.


Russ Roberts: And I think–I love when you said: Well, it depends on kind of person they are, what their preferences are. There’s no right answer here. It’s really important. In case situations of uncertainty and unpredictability, you are not the average person. No one is. You’ll either get cancer that kills you or you won’t. When something comes up, you don’t–you either get a turtle or a bird or a rabbit and you can’t–you’re unique, you’re you. We don’t really have any way to customize yet; and hopefully, we hope someday in the future. But, it reminds me a little bit of the trolley problem, which we’re not going to digress on it in any great length, but for those who know it already, and we may return to this in a future episode, people feel differently about things that they initiate versus things that happened to them passively.

And, I think one of the reasons people screen, even when it’s, quote, “not rational,” or the expected value is zero, is they want to be proactive. They don’t want to have to regret later. They’re afraid that if they don’t screen and then they die and they get terminal cancer, they’ll feel like they were foolish and they made a mistake.

And I think for loved ones, also–it’s why I asked you; it’s a personal question–if you advise your wife not to get a mammogram, you’re putting yourself in a position where you might feel the rest of your life a terrible burden because you counseled something that ended up being, had a bad outcome.

The other kind of outcome–which is, you counsel screening, and that turns out badly–I think that’s easier for people to live with. All the kind of bad things that come from that–the perforated colon in the case of a colonoscopy or the side effects of the treatment. I think a lot of people cut other people slack when those things happen because they’re trying to fix it.

And, this is–the last thing I’ll say on this–it’s an example of, in economics, what we call the seen and the unseen. The direct effects of things are front of mind and at the center of our minds, and the things that are the unseen side effects tend to be minimized because you can’t be blamed for them; and you don’t feel like you’re blame-worthy, unless you’re an economist. [More to come, 52:31]

Rayna Prime

Rayna Prime

Rayna Prime Editor