Ep 90: Fabricated images threaten the integrity of Alzheimer’s research (with Charles Piller)
Credit: Randy Pench
What happens when potential fraud is detected in research papers on major medical issues?
In this episode, we talk to Charles Piller, an investigative journalist who published a shocking story in Science magazine in July this year laying out compelling evidence for misconduct in multiple journal articles on Alzheimer’s disease. This misconduct appears to have occurred in recent papers involving the experimental drug, simulfilam, as well as older, foundational papers in Alzheimer’s research.
Charles’s story focuses on the sleuthing of Matthew Schrag, a neuroscientist and physician at Vanderbilt University who studies Alzheimer’s disease himself. In an extensive (even heroic) effort, Schrag identified over 100 potentially manipulated images in multiple major research papers. We talk with Charles about the consequences of those seemingly fraudulent images for the field and for public trust in science. We also talk about the potential consequences for whistleblowers like Schrag, and what journals and funding agencies are doing to support integrity in basic research.
Cover photo: Keating Shahmehri
-
SPEAKERS
Charles Piller, Art Woods, Marty Martin
Introduction
AW: Memory is an essential part of our lives.
MM: Without memory, it’s impossible to go about a normal day. Finding things, understanding where you are and what you’re doing. Imagine the moment-to-moment confusion and frustration you would feel if you couldn’t do that.
AW: Take those rage filled moments searching for your keys when you're late and multiply them by a million…
MM: On a personal level, memory is key to our identity and place in the world. Most of us don’t think of it often but being robbed of this ability is catastrophic as it prevents us from learning, sharing experiences and forming relationships.
AW: Yet, this is the situation for roughly 6 million Americans who suffer from Alzheimer’s disease. Alzheimer’s is a neural disease characterized by slow decline in memory and overall cognition, eventually leading to death.
MM: But before that end comes, patients are robbed of their ability to remember and to think, and their families are robbed of the person they once knew.. The burden of care for these families is immense, both emotionally and financially.
MM: To better understand the personal impacts of an Alzheimer’s diagnosis, listen back to our season 4 episode “A tattoo on the brain” with Dan Gibbs, a retired neurologist who not only spent his career studying Alzheimer’s but who was, seven years ago, diagnosed with it himself.
AW: Because Alzheimer’s is so devastating and common, governments and companies have collectively spent billions of dollars looking for a cure. But, of course, to find one, we must first understand what causes the disease.
MM: Enter the amyloid hypothesis, an idea that has dominated research efforts for the past several decades.
AW: The amyloid hypothesis encapsulates the concept that Alzheimer’s is caused by the accumulation of beta-amyloid in the brain. Beta-amyloid is a peptide and the product of the cleaving of a larger protein into two parts. In healthy individuals, these two beta-amyloid chunks are easily broken down.
MM: Not so for people with Alzheimer’s in which amyloid accumulates, eventually forming plaques.
AW: Do the plaques cause the disease or are they a by-product of some other causal agent or process? That’s a key question, and for over 20 years, research into the amyloid hypothesis and anti-plaque treatments have received a lot of funding.
AW: Today on the show, we talk with Charles Piller, an investigative journalist who broke a shocking story in the magazine, Science, in July of this year. In it, he laid out compelling evidence for misconduct in major journal articles that underlie support for a recent Alzheimer’s drug.
MM: And worse, he suggests that some of the key papers that boosted support for the amyloid hypothesis, publications starting more than 15 years ago, contain manipulated or falsified data.
AW: Piller’s story focuses on the heroic work of Matthew Schrag, an early-career professor at Vanderbilt. Schrag recently identified potentially manipulated images of Western blots in papers funded by Cassava Sciences.
MM: Cassava is the maker of one of the most important current drug for treating ALZ, simufilam, and Schrag’s work on the simufilam papers eventually led him to look for manipulated images in a broader array of Alzheimer’s papers. And boy did he find some…
AW: it appears that multiple papers authored by an important Alz researcher named Sylvain Lesné contain serious red flags. Piller and Science magazine conducted a 6-month investigation into these papers, the result of which Piller says, “provides strong support for Schrag’s suspicions.”
AW: This is a troubling story for many people: the funding agencies and companies who’ve poured millions into the amyloid hypothesis, the students who’ve staked their research careers on data that now appear questionable, the millions of patients and their families whose hopes have been pinned on the success of this work.
AW: A quick note and apology about the audio quality on this one - we recorded this one on the road and ran into a few issues – should be back to normal soon!
AW: I’m Art Woods
MM: And I’m Marty Martin
AW: And this is Big Biology
[Music break]
Art Woods 00:00
We're here to talk about this blockbuster article that you recently wrote for the journal Science that appeared late in July this year and it tells this kind of amazing story about potential fraud and a set of papers about the underlying causes of Alzheimer's disease. And a lot of the story focuses on detective work analysis done by a scientist named Matthew Schrag who's at Vanderbilt, we want to really talk about his contributions to this whole set of issues. Before we get to Schrag, let's start with you. And just ask about how you got into science journalism and sort of what your background in biology is, and how did you come to be a journalist for science.
Charles Piller 00:41
So first of all, thanks very much for the invitation, I'm pleased to be with you. I have been writing about science and technology for many years, really, I don't have advanced training in the sciences beyond bachelor's degree. I found that a person who studies hard and talks to a lot of smart people can learn a few things about science along the way. I did a lot of work writing about science as a freelancer years ago, wrote a couple of books about, investigative books, about science, one about biological and chemical weapons, and another one about public attitudes towards science and technology. And subsequent to that, I branched out and got into writing about high tech for a while and then eventually ended up at the Los Angeles Times, working both on technology writing, and then eventually science and primarily focused on investigative reporting of various kinds. So my career has been varied. I then work for another newspaper, The Sacramento Bee doing investigations, both concerning scientific issues and completely different issues. And I then went to the health website, STAT, which some of your listeners may be familiar with, and did investigative reporting and other enterprise reporting with them on a range of issues, including clinical trials, medical developments, and a range of research issues. And now I'm at Science Magazine, doing exclusively investigations, where I'm happily engaged in a variety of topics, and the most recent one being the Alzheimer's work.
Art Woods 02:17
So when you do these investigative reports, like just maybe give us a flavor of how you go about it, are you out, calling up scientists going to visit them? Reading the papers really carefully, like people coming to you with information? How does it progress?
Charles Piller 02:31
Well, it's all of the above. So, as an investigative reporter, I rely on both my own instincts and also tips from sources. So, a lot of my stories come to me from people who think they have an important discovery of either malfeasance or fraud, or perhaps error that deserves some public scrutiny. And I'm finding that in many cases, the tips are pretty interesting, but really not for me. I'd say about one and 10, is worthy of close scrutiny and smaller number of those really work for me as a story for Science Magazine. We really try to focus closely on research issues, primarily. A story might be a very worthy look, say at a medical problem or the behavior of doctors, for example, it may or may not work for Science, because, you know, we're looking at the community of scientists who have research interests as one of our most important target audiences.
Marty Martin 03:33
Charles, can you say something about the period between the you mentioned bachelor's in science, but I don't think I heard you say what discipline of science that was. What was that? And then did you get into the investigative journalism that you do now based on some sort of deliberate planned series of steps? Or were there just fortunate surprises that happened along the way that brought you to where you are now?
Charles Piller 03:57
Well, no, I didn't do it from a planned series of steps, per se. Now initially, I studied psychology in college. So it wasn't really that closely related to a lot of the kinds of stories that I do now, in fact, pretty much unrelated. But I was inspired by the generation of journalists who uncovered a lot of the problems associated with Watergate and malfeasance in the 70s and early 80s. And because I always had an interest in scientific and technical subjects, I gravitated in that direction. So I kind of got in on the ground floor in my early days as a journalist doing freelancing on issues related to biology, genetic engineering, and their possible relationship to the development of biological and chemical weapons and also to the defense against those weapons. And because of the time I started doing that, in the 80s, I was kind of in on the ground floor, you might say and it really stimulated my interest in the powerful social and political issues associated with the use of scientific research and kind of launched me in the direction of trying to do work that had the possibility of having impact on people's thinking and on the culture of science and also more broadly in the culture as a whole.
Art Woods 05:17
Well lets move on to talking about some of the meat that's in your recent science article. And maybe you could describe who Matthew Schrag is, and what his sort of broad set of claims are about his set of papers that are about Alzheimer's disease. And maybe let's start in focus on just the Cassava Sciences issues here at the beginning,
Charles Piller 05:39
Right so, where this story begins with Matthew Schrag, is that last summer, so August of 2021, Matthew Schrag was contacted by a colleague of his who was associated with some well known experts in neuroscience, who had raised some concerns, and were discussing with their other colleagues about a drug called Simufilam which was being tested by a small company called Cassava Sciences. And this drug is designed to benefit Alzheimer's patients as so many new drugs have been attempting to do. The scientists who were raising concerns felt that the drug maybe was based on faulty science of various kinds, has been described in various ways, but the word fraud has been introduced by them in a document they filed with the FDA voicing some of their concerns about Simufilam, this drug from Cassava. And at the time, they were planning to deliver, what is called a Citizen's Petition to FDA. And this would be a way of asking FDA take a look at this place, we have real concerns about the way in which the underlying science behind this drug development was conducted. Our concerns are serious and we think you should consider pausing clinical trials with the drug in order to take a closer look. The missing piece that these scientists had was that they were not experts in assessing possible image manipulation associated with the underlying basic research that went into the development of the drug. That's where Matthew Schrag comes in. He's a guy who has this dual expertise. He's a neuroscientist, a physician, and he studies Alzheimer's disease among other neurological ailments. And what Matthew Schrag was able to do, from his own many years of detailed experience, looking at some of the fundamental images created for these kinds of studies, he was able to identify possible fabrications or possible manipulations of images that would tend to support the underlying hypothesis of the experiment, but perhaps, were improper or incorrect. Let me just say from the outset here, Cassava denies any inappropriate contact, any inappropriate work on this basic research underlying their drug, they also deny that any scientific misconduct has occurred among affiliated scientists. So I just want to make sure people understand that these are claims that are being made. And I can also describe later if you'd like, why you can't definitively call this for example, fraud, at this point.
Marty Martin 08:30
Yeah, let's definitely circle back to that. But I think, you know, we're a biology podcast, it's almost imperative that we talk about some of the nuts and bolts of Alzheimer's disease. Can you briefly walk us through what Alzheimer's is, how it manifests, the amyloid hypothesis that's so central to the rest of this story?
08:48
Sure. So Alzheimer's disease was named more than 100 years ago, when a German scientist, Alois Alzheimer, in an inspection of the brain of a deceased dementia patient, found a combination of what are called plaques, these are kind of sticky protein deposits within the brain. And another kind of protein deposit called tangles. So this is the simple definition of Alzheimer's disease. In recent decades, the definition of these plaques and tangles, the way in which they're described scientifically has become much better understood. And the plaques in particular, are a form of protein called amyloid beta. These plaques have for many years been thought to be the primary cause of Alzheimer's disease, and in a way, the primary manifestation as well, and so enormous work has gone into drug development to remove these plaques from the brain with the understanding that if you do so you will improve cognition or you will slow the rate of cognitive decline in Alzheimer's patients. That's a critical part of the idea behind the so called amyloid hypothesis, which is the dominant hypothesis in this field. There are other ideas about what might cause Alzheimer's or how it might be treated, including work on inflammation in the brain, something called neuro inflammation, drugs are being developed to try to attack that. There are also beliefs that infectious agents might affect, Alzheimer's disease might cause it. And also there's another protein called tau, which is part of the puzzle, again, a drug target for some companies and some scientists. So there's a range of ideas about what might cause Alzheimer's. But in the last several decades, the amyloid hypothesis has been dominant. It's received the lion's share of the funding and the attention scientifically.
Art Woods 10:47
So let me just maybe restate: you laid out a range of hypotheses here. And one idea is that the amyloid plaques are not a cause of Alzheimer's, that they're some side effect of some other process that may be associated with Alzheimer's, but they're not causative of the disease. And so the amyloid hypothesis says yes, they are the cause these other hypotheses focus on other potential underlying causes, right?
Charles Piller 11:12
Well, I mean, different scientists view it in different ways. The amyloids are considered both cause and effect by some, you know, and primary or secondary by some. So for example, there may be an interrelationship between more than one biological process that goes into the clinical designation of Alzheimer's disease. One of the one of the really vexing problems has been that until recent decades, it's been impossible to know whether an individual has these plaques and tangles in their brain until an autopsy was done. Now, more recently, PET scanning has maybe not completely solved that problem, but largely solved it. So you can have a pretty good idea about whether a patient has these plaques and tangles in their brain during their lifetimes, not just after death. And I might add that one of the other vexing elements of this amyloid hypothesis is that there have been many cases of individuals who after death in an autopsy are found to have enormous concentrations of plaques, yet didn't suffer from dementia symptoms during their lifetime. So it's, it's complicated.
Art Woods 12:21
So it's an imperfect correlation at best.
Charles Piller 12:24
Yes, exactly. Right.
Marty Martin 12:27
Without getting into the details too much, Charles, can you tell us about this drug Simufilam, that Cassava made? And how that's supposed to interact and help with the tangles and plaques?
12:39
Yeah so, Cassava has described its drug in a variety of ways. So I'm a little bit at a loss as to say precisely how they think it works. They describe it primarily as working on a particular protein called filament A, that they say is misfolded, and therefore causes the accumulation of problems leading to dementia, including the concentration of plaques. And they also describe it as having a beneficial effect on another form of amyloid beta protein that we will certainly talk about later in this conversation called oligomer. So these are, unlike the sticky plaques, oligomers are soluble amyloid beta forms that are also thought to be toxic. So it's a little confusing with Cassava, because in different papers, they describe the mechanism of action of their drug differently. And so it leads scientists to ask well, exactly what is going on here? Why did they describe it differently? Things like that. So I can't give you a definitive answer on that.
Marty Martin 13:42
Okay. I mean, this this is obviously a great big area of research. I mean, that's the crux of what we want to talk about. But right now, is it the case that most drugs are targeting these plaques, or their progenitors or processes that lead to their formation.
Charles Piller 13:57
In fact, there's drugs being developed in a wide range of areas related in a general sense to dementia and Alzheimer's, and of course, Alzheimer's is the leading cause of dementia. But I would say when you look at the concentration of money, and of NIH grants, and of interest in the field, there's no doubt that the amyloid hypothesis has been dominant, but it's not singular.
Art Woods 14:21
Well let's get back to Matthew Schrag and the sort of heroic work that he's been doing to try to uncover what's going on here. So you mentioned image manipulation. And maybe let's just dig in for a moment to what those images are. They're their western blots, right, which are sort of maps of proteins. What did he notice about these images that led him to be suspicious?
Charles Piller 14:45
Okay so, Schrag looked at a variety of images, not just western blots. But in this field, western blots are critically important because when you're looking at proteins, this is a principle method been used for many years to separate proteins by size by molecular weight. What's so great about them is they create this kind of visual, almost a fingerprint, of the range of proteins in a sample. So people grind up sample of brain tissue, they put it through this device called a Western Blot, and it's a kind of a sieve, a gel base sieve, that separates out the sample into bands of protein. So you have these things called bands that are kind of like blobs when they appear photographically, different shapes, but generally kind of more or less oval shaped. And the entire set of bands is called a Western Blot. So it's a great way for understanding what's in your sample. And why it's important is that different proteins are being identified as possibly associated with things like Alzheimer's disease, so you got to see what you have in your sample. The problem with western blots is that they are very easily manipulated using digital tools. It used to be in the old days, you could manipulate them by cutting and literally taking photographs and cutting and pasting these photographs. And there are many instances when that was discovered in the scientific community. But those primitive days are pretty much over now. And what people do are using sophisticated tools, including Photoshop, but also including other tools, even some digital imaging tools that are used by NIH and endorsed by NIH as is appropriate to try to create the best clearest images from your experiments. So that's the problem, you can have cut and paste different bands to put them in different areas to make it appear that they appeared somewhere that they didn't appear, you can change the intensity, the contrast of these images in such a way that it appears that you have a higher concentration of a particular protein, or a lesser concentration of another one, you can move whole sets of bands from one image to another. Often, I think people familiar with the field see that journals have limited real estate in their pages. They don't print the entire uncropped original western blot very often, they print clipped portions of it to save space and to highlight the particular features that are most pertinent to the experiment. The problem is that that method can lend itself to manipulation because it opens up possibilities to move things in the wrong places.
Art Woods 17:27
You're sort of already clipping out parts of it. And so it's too easy to start putting things back in where they shouldn't be.
Charles Piller 17:34
A temptation. It's a temptation, I think, for some people who are unscrupulous.
Marty Martin 17:38
Yeah. And I think just to drive home, I don't know, probably a lot of our listeners have seen a western blot and at least a picture of it if they haven't performed one themselves. But for those that haven't, I mean, this is a very low tech looking thing. And you cast it as an oval. But to me, it looks like just an ink blob, right? I mean, it really doesn't have that much structure. This is not a sophisticated thing with lots of complicated edges and making it up wouldn't really be that hard, even for somebody that's technically not so coordinated, such as myself.
Charles Piller 18:07
Yeah, people who do this kind of manipulation, I think it's kind of axiomatic that no one ever does it once, you know, if you're going to cheat, you cheat a lot of times, usually, and there can be a lot of telltale signs of cheating when people are manipulating such blots, there are the so called clip marks, cut and paste marks associated with moving these things around, there can be halos around these bands within the western blots that don't appear to be a natural part of the experimental photographic experimental process. There can be areas of strange markings that look like something was erased with an eraser tool in Photoshop, there's no limit to the kinds of digital manipulations that can be done a lot of times images are reversed. So they just flip something over and put it somewhere else. And it looks like a normal blot. But you have to flip it back over and show where it came from in order to show that it was copied and pasted from somewhere else. There's a lot of cleverness that goes into this. And there needs to be a lot of cleverness and creativity in reverse engineering possible manipulations, which is something that Matthew Schrag developed quite a talent at. That's not himself alone. Many others have figured out how to do this as well.
Art Woods 19:26
So if we just focus on this, the scope of what he found in these Cassava related papers, so is this just like a single instance or a single figure? Or is it multiple figures and multiple papers? Like what's the scope of what we're talking about?
Charles Piller 19:39
Well, we're talking if you can include both the specific papers produced by scientists who work directly involved with Cassava and their own people, plus scientists who they paid or who are affiliated with them via their grants or studies that involve their money, talking about dozens of papers, hundreds of apparently manipulated images. Schrag created a dossier about Cassava, it's 100 pages long. And it has image after image, paper after paper, apparent manipulation after apparent manipulation. Now, of course, as a journalist, I didn't just accept his findings about Cassava, I knew that I had to validate the findings that he made, and to understand whether they were well founded and well considered and thoughtfully constructed. So I showed his dossier to a number of leading experts in the field. In other words, Alzheimer's experts who have viewed these images thousands and thousands of times over the course of their career, and had an eye for it. And I also delivered it to professional forensic image analysts. And I basically asked them, here's some conclusions being drawn of apparent or possible manipulations in these images. What do you think, do these ring true to you? They didn't all agree, they didn't agree on every image. You know, literally, we're talking more than well over 100 images, more like 150 images. And so not every expert agreed on every image. But overall, they all agreed this was a compelling, thoughtfully done, correctly designed analysis of these papers. And that without question, there were enormous red flags associated with many, many of the papers that no one disputed. In other words, I felt, going forward, that I was on solid ground that had been verified and validated in a number of ways by people who had the knowledge and ability to do so, including I might add, skeptics of the amyloid hypothesis and very strong supporters of the amyloid hypothesis. I felt that was critical to getting that range of views.
Art Woods 21:50
Yeah so regardless of your conceptual, like starting point. So what are the consequences for Simufilam? I mean, it seems like they would be dire.
Charles Piller 21:59
It does seem that way at a glance. So well, it's a bit of a long story, that that story is still unfolding, we'll just have to see. The drug is still being tested in phase three clinical trials. So some scientists, of course, including Schrag and the detractors that were associated with the FDA petition, find that this is very unfortunate state of affairs, because they don't believe that the basis for the science is correct. And the FDA has declined the request to pause those trials, but is reserving its right to look further at the issue, and to possibly force a halt or a pause in those experiments at a later time. So those trials are underway, but they're going to be many months, if not years before the final results come through. So we're waiting. And Simufilam and Cassava are going ahead with the trials.
Marty Martin 22:53
Does FDA have to give a reason for its declining of pausing trials, or just sort of makes that decision, we don't really know on what that decision was based?
Charles Piller 23:02
FDA actually is a bit opaque in their process. I happen to know that they're looking into this further, because just from sourcing, but what they'll do about it is very much a big open question. They work very slowly, the regulatory process, both at NIH and FDA for looking at the possibility of misconduct and research is a kind of machinery that turns at a snail's pace and can be very frustrating at times.
Marty Martin 23:34
Well let's go back into sort of the scope of this issue, you made a clear point that the number of images up to 150 or so that have issues of concern to a lot of experts, some of which support and some of which are against the amyloid hypothesis. But there's something I think, important that we need to point out here too, the literature in science is gigantic, there are very prominent journals, and then there are very obscure journals. And then there's a whole lot of other great literature. But among this set of 150 images, is included one of the most impactful papers of you know, sort of the foundations right, because the 2006 paper that has been cited 3000 times, that has been fundamental to the development of this idea in the first place. So can you talk about that paper? What was found there and its role?
Charles Piller 24:22
So let me let me take a step back, Marty, if I can, to just slightly correct a possible slight misunderstanding. So the story of Schrag continued after this Cassava issue. What happened was that he was continuing to try to refine his skills in understanding better how to assess these images. And there's an important and interesting tool that many biologists use, it's an online forum called Pub Peer. And this forum is place where people post concerns that they might have, not just about images, but about other issues associated with scientific papers and are often there in the form of: I see this image looks funny in this way, could the authors provide the original, uncropped, unedited versions of these images so that we could understand better whether we're seeing something that's just an innocent artifact of the publication process? Or is there some other bigger concern here that needs to be looked at? And this is a place where people can post anonymous comments. So you have to take everything with a grain of salt, there can be accusations that are incorrect up there, or concerns raised that might serve to undermine people improperly. But there's also an enormous amount of great information up there. And what Schrag was doing is he was saying maybe I can use this as a tool to refine my own instincts and to refine my own abilities to understand better how to assess his ongoing look at the Cassava Sciences work. So he went up to Pub Peer, and he just searched for the term Alzheimer's to see what came up. And up came a bunch of studies from this researcher at the University of Minnesota by the name of Sylvain Lesné, and these were images associated with a particular type of amyloid protein called oligomers, which I mentioned earlier, a soluble form, a toxic soluble form of amyloid. And these papers caught his eye because they had been recently flagged on PubPeer, he started to look at the images that had raised concerns among the anonymous inquiries on the website. And he thought, okay, well, this is interesting, these images do look to be problematic. Then he looked at other images in the same papers that had not been flagged on PubPeer. And he thought, oh, here are a few others that look a little funny to me, too. Then he thought this is an interesting situation. The common thread in the papers was this guy, Lesné, although there were other, of course, many other authors on the papers, as well as obviously typical in biology. And he then did a bigger search, just did a PubMed search for papers, by Sylvain Lesné. And up came a bunch of other papers that he started to look at in a more methodical way. And one of these was a paper that was published in Nature, one of the most important scientific journals in 2006. And it was a paper that had been enormously influential. So if you'll forgive me, I'll just digress for a second to explain, why was this paper influential? So the problem was that the amyloid hypothesis had been the dominant way of thinking about how to look at Alzheimer's for a long time, certainly, since amyloid beta proteins were well defined in the 80s. This hypothesis, though, has been coming under some stress by the early 2000s. And the reason is that a lot of drug development had gone into attacking amyloids, and it's been even shown over and over that you could remove amyloid proteins from the person's brain, but not have an effect on the cognition of that patient. So what does that mean? Well, people didn't know they still felt so sure that amyloid proteins these plaques and tangles in particular, that were at the heart of Alzheimer's, would be important to remove from the brain. Yet, this lack of drug development success was troubling. Then came along this 2006 experiment by Sylvain Lesné at the University of Minnesota with his mentor, a woman by the name of Karen Ashe, who is a quite illustrious biologist and physician who made enormous contributions to the field herself, completely separately from Lesné. And their experiment was fascinating. Because what they did is they were able to isolate in the brains of these transgenic mice that I might add had been invented by Karen Ashe, a very important animal model for this entire field, transgenic mice that produced enormous amounts of amyloid protein, they isolated this particular protein from the mouse's brains and this protein they dubbed amyloid beta star 56 (Aβ*56). Now, the number 56 refers to the molecular weight of the protein, kind of on the heavy side of these amyloid proteins. Then they injected this into rats and almost immediately found that rats exhibited something you could call the memory loss symptoms akin to Alzheimer's disease. So really, for the first time, you had a direct apparent cause and effect relationship between a particular amyloid protein and the symptoms of Alzheimer's disease and it was a watershed moment.
Art Woods 29:50
Now it seems like a smoking gun there for that hypothesis.
Charles Piller 29:54
It sure does. It sure does. And it was enormously interesting to the scientific community. So people in this field know that probably 10s of 1000s of basic research papers have been published on Alzheimer's disease since 2006. Since this article appeared, I did an analysis that showed that out of all of those papers, 10s of 1000s of papers, this paper by Ashe, Lesné, and colleagues has been cited by more than all but four other basic research papers in the field during that period. It was a big deal, because people incorporated it into their thinking. It encouraged them to continue to look in those directions. And it helped form the foundation for the belief that people were really on the right path in looking both at the amyloid beta issue, the amyloid hypothesis and the specific soluble toxic oligomer aspect of that, so it was a very big deal. That's the backdrop for why this paper caught Matthew Schrag's eye, he sees this paper and he realizes, wow, this paper was a big deal. This has been cited 1000s of times, but there's something fishy about it. And so what he did is he started to look carefully at this paper and what he found was that in image after image in this paper, there were suspicious markings, crop markings, apparently duplicated blots. Bands, I should say within blots, apparently whole sections of blots, moved around to other blots, weird differences in the intensity that couldn't be attributed to a normal western blot, mechanical process and photographic process of creating these images. And he was deeply concerned about it. So he started to do a more detailed analysis of Lesné's work, both with Karen Ashe, this mentor of his and separately, and he found about 20 papers associated with Lesné, 10 of which were specifically about this famous star 56 protein that appeared to have image after image paper after paper, apparently, doctored images. Now I say apparently, let me explain why. Why do I not say he knew with absolute certainty that they were doctored? The reason he couldn't say that with certainty, is that you need to see the full uncropped unedited images, which he didn't have access to, only the investigators have access to those, and sometimes they give them to the journals. But he wasn't able to gain access to them, all he could say was, these look really funny, they look doctored, someone needs to take a look at this. And until they do, this should really be regarded as suspect. Now no one knew about this yet, basically, except me because he shared his information with me. This was way back in December 2021 or January of this year.
Art Woods 32:59
So it's like I get that point about needing to see the original images. But why would that even resolve it? Because couldn't whatever files are produced also have been doctored? And so why isn't seeing all of these marks and these sort of suspicious statistical anomalies and possible duplications in the published western blots? Why isn't that enough? Just to say, yeah, that's fraud.
Charles Piller 33:22
The reason is actually pretty simple. Sometimes in the file conversion process for publication, either online or in the printed magazine of these journals, there can be what are called compression artifacts. So when files are converted into one format to another, they can cause little anomalies that appear sometimes to be improperly manipulated, but they may have an innocent explanation.
Art Woods 33:47
So and you avoid that by getting the raw files out of the camera directly.
Charles Piller 33:50
Exactly right. And the problem is, of course, that when you reach an extreme case, where there are so many apparent manipulations and so many ways to test them, ultimately, you don't really need the original images, you can come to a very high level of certainty without those original images, but it always helps to have the originals to be absolutely certain.
Marty Martin 34:10
So beyond the issues of the images, something that is pretty conspicuous that we'd love to hear your perspective on, while other people could get these transgenic mice or they could collaborate with these folks and better recreate the work. What's happened with other people that have tried to work on A beta star 56?
Charles Piller 34:28
Yeah, fascinating question, Marty. The short answer is that many have tried, few have accomplished even validating that A beta star 56 can be independently confirmed. So there are a couple problems associated with this. One is that this oligomer form of A beta is very difficult to refine. It's kind of messy, it's very hard to really closely separate these out. Many people think that It's almost next to impossible. And so right off the bat when someone claims to have completely purified this star 56 protein, some people's eyebrows have raised. In fact, one of the leading investigators and supporters of the amyloid hypothesis, a guy at Harvard by the name of Dennis Selkoe, who I interviewed extensively for my article, Selkoe tried and failed twice to do this. And Selkoe also was extremely skeptical of the stated method of refinement used by Lesné and Ashe. He basically said, this does not seem plausible to me, scientifically. And he shared it with many colleagues. And he said there was a lot of discussion in the scientific community, how could they have done this, it doesn't make sense scientifically. And consequently, there was enormous skepticism. Now, there have been a few very small cases where people have said that they were able to some degree refine and test out the A beta star 56. But reproducibility is critical in science. And this particular protein never really met that bar. That said, you know, how the scientific community is, it loves these sorts of disputes, and it plays out over a long period of time. But what we can say with certainty is, this has never been an easy protein to find, and very few have been successful in doing so.
Art Woods 36:31
So if this really is fraud, and it sounds like suspicions are running pretty high that it is, what are the consequences for how we should think about the amyloid hypothesis?
Charles Piller 36:42
Well, let me make it clear that one of the consequences of my story is that it received global attention in the media, and a lot of that coverage was excellent, and some was not so good. So there were some right wing media pundits, including Tucker Carlson on Fox News, who were in essence saying, Well, if this research was faked, then nothing that's been done in the field for the last 16 years is worth anything. And by extension, no science is worth anything. So I mean, it was kind of a crazy argument that was repeated by others. But what I can say is that even the critics of the amyloid hypothesis, think that amyloids may play a significant role in Alzheimer's disease, what they object to, in a sense, is the exclusion of other important possible causes, because of the dominance of that hypothesis. They don't necessarily view a singular cause for Alzheimer's. And it may be that more than one thing is going on. So I would say I'm certainly not someone who says that the amyloid hypothesis is worthless and should be completely rejected and not studied. Of course, it should be studied. But I think many skeptics of that hypothesis would say, let's broaden our horizons. Let's put more concentration in other realms. Let's try to create a scientific cornucopia of ideas that attack other possible and perhaps very promising avenues.
Marty Martin 38:12
I'm a little bit hesitant to touch on this given we don't want to potentiate skepticism for all science based on this one case, even if its expansive, but what kind of money and resources are we talking about, what's the scale of the problem on a financial level with this particular case?
Charles Piller 38:33
Yeah, so of course, the NIH has supported most of the work associated with Ashe and Lesné. But I think that's a really a small percentage of what the financial issue should pertain to. So let me just look at the NIH spending separately from the drug development, because the drug companies are putting literally billions of dollars into development associated with drugs attacking the so called toxic oligomer forms of amyloid beta, that Ashe and Lesné studied. So that's billions of dollars. The NIH has also put in enormous amounts of money into studying these things. So just to give you an example, in 2006, when this seminal Nature paper appeared, they were spending next to nothing on these toxic oligomers. In 2021, they spent $287 million dollars on studies supporting that sort of work. And of course, you've got all the years in between where it was ratcheting up. So you're talking literally billions of dollars from NIH on studying these things, more than a billion anyway. So that's an enormous amount of money. Now, of course, was all that money misspent because one set of studies associated with oligomers might have been faked? No, I don't think so. I think some of that was interesting research that was worthy of doing. I think the bigger point and the point that I think was made a far better than I can by this guy Tom Sudhof, who's a Nobel Laureate, who has studied Alzheimer's and other neurological ailments for many years at Stanford, what he told me was the immediate obvious damage is wasted on NIH funding, and wasted thinking in the field. Because people are using these results as a starting point for their own experiments. That is one of the big hazards associated with this. When you have a very influential study that can skew ideas about what's important. Let me just give you an example that to me was very profound, I mentioned this guy, Dennis Selkoe, who's an eminent scientist, and has been one of the most influential figures in the field for many years, a big supporter of the amyloid hypothesis. Two of his early papers, he tried and failed to show the presence of A beta star 56, this famous protein, and wrote about that, however, in many subsequent papers, at least 13 that I've identified, he cited this paper as evidence for furthering study in the toxic oligomer field. So you have to ask yourself a question, he was skeptical of their technique, he found it implausible, he was unable to reproduce it himself, he was unable to identify these proteins in his own experiments. And yet, it still had influence in how he considered and thought about the field. So that's just an example. There are dozens and dozens of other famous researchers who also made references to this article, in an overall view of the importance of looking at this realm. So does that mean all their conclusions were wrong? Of course not. It just means that it had a kind of influence that could be regarded as pernicious in the field.
Art Woods 41:47
Let me ask a different, I guess this is a narrow question about economic consequences of what you've described, and that is circling back to thinking about who wins and who loses economically with Cassava story. So the two neuroscientists that got this ball rolling were short sellers of Cassava stock. And so they stood to gain if Cassava came under really heavy fire. When I was preparing for talking to you, I checked the Cassava stock trajectory over the last year. And it looks like it hasn't taken much of a hit or any at all, if you know, it's sort of hot, maybe as close to this yearly high. So maybe just have any insight into what's going on with the Cassava situation.
Charles Piller 42:30
Sure. Thank you for raising that, Art. I should have mentioned that they were short sellers. That's an excellent point. So yeah, they have a vested interest in this, very important to note that. Second, I should also mention that Schrag, the guy who I've been working with who has supplied a lot of the image manipulation data, he was paid $18,000 by the attorney representing those short sellers, for his services, I should say that he spent many hundreds of hours on these projects. And that's the only money he's received from it. So just to be clear, he's not making a fortune doing this. But that said, I think you need to look farther back at Cassava stocks, as Cassava is a company that was valued at about $6 billion before any of this broke, because they were talking about and beginning to test what was regarded by some as a promising Alzheimer's drug, after the news came out, more than a year ago. So if you're only looking back a year, you're not looking back far enough, more than a year ago, when the news came out that the basis for their research might have been manipulated images and improper experiments, the stock plummeted down to about a billion valuation. So it lost about five sixths of its value.
Art Woods 43:48
Oh, wow. Okay, yeah, I didn't look back far enough.
Charles Piller 43:49
It's recovered a bit. So it's a little it's been a little bit higher. There's a lot of speculation in Cassava stock. If you look at this stock over the last few months, it's gone up and down and up and down. For example, when my story appeared, it plummeted for a while, then it crawled back up. So we'll see. We don't know what's going to happen with that drug. But we're not going to know for quite a while and we don't know if the FDA ultimately will intervene.
Marty Martin 44:13
So that's sort of the financial consequences for multiple different entities. But what about the adequacy of responses by the other players in this process? What have the journals said? What has NIH said? What do they consider to be their roles in what's happened?
Charles Piller 44:28
Let's take it one at a time. So let's start with the journals. The journals have been mixed. A number of Cassava related papers have been retracted since Dr. Schrags findings were presented to them, because he sent his findings to the journals, not just to me, some to the journals before I saw and some cases of retractions have occurred. In other cases, statements of concern by the journals have been posted on their websites and consequently, they're continuing to inspect the data, just like I mentioned, they asked the principal authors normally for the original uncropped images from the western blots, or from the microscopic imagery of, for example, brain tissue that are often appearing in these papers as well, which I might add are also easily altered using Photoshop and other digital tools and have been altered in many occasions. So this process can be incredibly slow, the journals are extremely conservative, there's the embarrassment factor. There's the lack of expertise on their own staff, depending on the quality and sophistication of the journals. And I might add, even top journals lack sophistication in this area. So how do they do it? Well, a lot of times, they don't, they just do a terrible job, they just don't scrutinize these images well, and then they get in these horrible situations. Other times, they have been starting in recent years to use AI programs that look at digital images and try to find possible elements of them that have been improperly changed. And these tools have some capabilities, I've viewed some of them. And I'm looking actually closely at that now. But I should say that they catch some things and not others. And consequently, they can do a review of a variety of say, Western Blot images and say, Okay, well, we don't see anything bad here. But someone who has a keen eye, a human being, not just using an algorithm and an AI program, can see other things can try other techniques to tease out where the problems might exist. And it does take a bit of a seasoned eye to be able to see these things, I'm sure these tools will improve, just as the ability to change, the images will improve. So we might see a bit of an arms race. So I'm not totally blaming the journals for being terrible at that, although they are terrible at it, by and large. I'm saying that it's a hard problem that they are not very good at solving.
Art Woods 47:09
Let me just say, this relates to what you said about AIs and the sort of continued supremacy of the human eye and the human insight, because I would have thought, I mean, it just feels like that's the kind of problem that an AI ought to be pretty good at solving. And it's interesting that, you know, there's still has to be a human element in there to catch to catch some of them.
Charles Piller 47:27
So far, that's the case. I'm hopeful that these AI programs will improve over the years. And I think I think over time, this will likely become a little less of a problem, at least for a while, because I think these AI programs are going to improve. They're not there yet. But they can already do some things well.
Marty Martin 47:48
So I want to hear about the NIH response. But while we're on AI, what about flipping it over? Once these things are made really good to scrutinize and detect problems? It's not long before they were exploited to create the perfect images that are fakes, but really difficult to detect. How are people worried about that? Is that part of the conversation as well?
Charles Piller 48:07
They're not worried, they're not paying attention.
Art Woods 48:13
Okay, so the NIH response?
Charles Piller 48:15
So Schrag also submitted his results, both for the body of work associated with the underlying research for Cassava and its drugs and also the whole body of work associated with Sylvain Lesné, Karen Ashe, and studies associated with AB star 56 and other work that Lesné conducted, and these dossiers went to NIH, in the same way that he presents them to the journals. These look like possible misconduct associated with NIH grants, you should look into it, please look into this and do an investigation. So what happens is, the federal government has a process where NIH will take an initial look, and they'll say, okay, maybe there is something worth looking at here or not, if not, they'll just drop it. If they think there's something possibly improper there, they will give it to another sister agency, a tiny little outfit within HHS, called the Office of Research Integrity. And this group within the Health and Human Services Department will then do its own look at the images and they'll say, Okay, if it passes muster with them as something that's worth looking into, they will then deliver their questions. And in this case, presumably the dossier that Matthew Schrag created, to the universities where the research was conducted. And they'll say to the universities, we think there's something going on here that needs to be looked at, you should do a look at this yourselves and report back to us and then they would review what the university comes up with.
Art Woods 49:56
So it seems like a potentially very long, slow process.
Charles Piller 49:59
You've caught onto problem one. There are at least three really big problems with this process. One is it's opaque, completely opaque to everyone, to the public, even to the requester, Matthew Schrag, he can't know what if any of those steps are being followed, but we presume that there are steps being followed because both the University of Minnesota, and in the case of Cassava, the City University of New York, where a lot of the original research was conducted in the lab of one of the scientists there, in both of those cases, the universities have acknowledged that they're conducting investigations into these concerns. So I presume that it's followed that normal NIH Office of Research Integrity, step by step approach. The problem is that the universities have the least to gain and the most to lose if misconduct is concluded to have occurred. And there's no independent overview of how they're going about their investigation. And so, consequently, what we've seen over and over and over and over and over, first of all, it takes forever, as you guys have noted, it can take months or even years to go through this. In the meantime, the scientists are continuing with their work, even scientists whose work has been deeply questioned as potentially tainted. Second, the conclusions reached by the university, then would undergo a new set of reviews at the Office of Research Integrity, they would come to some conclusion about, you know, if, for example, no misconduct was found by the university, they would have to review that and they could either concur or reject it. Again, it takes more months, more assessment. Eventually, you might have a situation where misconduct was formally found and announced by the federal government, only in that case, would you have this formal record of it that was publicly available, and that's a small minority of cases, even where the evidence seems somewhat clear to outside observers. Terrible, it's a terrible process. I think many people agree, very inadequate. Look, due process is really desirable and important. We shouldn't just assume that because there's some evidence of possible misconduct, that the scientists are doing improper activities, it has to be investigated. But the system that's in place for doing that, I think many people would agree, is lacking in teeth, and it's lacking in vigor.
Art Woods 52:27
Seems like it's due for a big overhaul. Well, a last question about consequences. And that's about the consequences for Matthew Schrag himself, you know, consequences now and sort of consequences in the future. This is a, you know, really kind of bold thing that he's doing and wondering what you think about what the risks and possible benefits are for him.
Charles Piller 52:48
I think that it's important to keep a couple things in mind with a guy like Schrag. First of all, he's a junior professor at Vanderbilt. And he did some very courageous things. One is he challenged some leading figures in his own field, he challenged leading journals on whom he might be depending for his own publications to advance his career and questioning their decisions publicly, he has also questioned the process, and the decision making of providing grants to these investigators on the part of the NIH, the funding agency on which he depends for his sustenance, as a scientist. He also was taking the risk of incurring the wrath of internet trolls, and the supporters of companies like Cassava Sciences, who have come after him quite aggressively on Twitter and other social media, as they have against me as well, of course, but for journalists that comes with the territory. For a scientist, you know, you don't really want to get into that sort of fight obviously.
Art Woods 53:55
A little more shocking for us, I think.
Charles Piller 53:58
Yeah. And so there's all kinds of reasons why he had reason to be cautious. He took a long time, months of discussions with me about whether or not to go public with his findings. And ultimately, you know, look, I was obviously going to not make his name public, if he didn't want to go public. It's so important to honor the wishes of sources in that regard. But in his case, he finally concluded and this, of course, this was my position all along, is that this is a story that had some drama, and some impact in the field that could best be told as a journey that he traveled through his eyes, the risks he ran and the discoveries he made as they unfolded. And I think that's why, in part why the story grabbed a lot of public attention is because people understood that this is a saga, that science needs to understand, that scientists need to grasp of when they're within their own world. I mean, let's face it, it's a human endeavor. Science, like any human endeavor, is going to have some corruption. Some people who are cutting corners, some people who value their own careers, over the purity of the scientific record, and the obligation they have, not just to the research community, but to patients and to the public as a whole and to funders. And even though in my view, there are relatively few scientists who are genuine cheaters, there are enough that we need to remain vigilant about it. And it's my responsibility as a journalist to call attention to it when I see it. And it's the responsibility of scientists like Schrag. And I think they're rare to sometimes go out on a limb and and take some risks and be able to provide the community with some insight.
Marty Martin 55:54
Well, Charles, thank you so much for your time to tell the story. Thank you so much for your time to write the story, the article and do all the research in the background. Obviously, I wish we could have had a conversation about a brighter, happier topic. But this is important stuff to platform. So thank you for that. We always end giving you the chance to add to the conversation, something some question that we didn't put out there. Is there anything that we didn't touch? Is there any other point that you would like to raise?
Charles Piller 56:24
I just want to end on maybe a slightly more positive note, which is to say that science has the ability to self correct. It has the ability to intervene in its own affairs effectively. And I think the model that is presented by Matthew Schrag's work suggests a way that that can happen, a way that scientists can kind of refresh their own understanding of the importance of, of self correction and also of self policing. It requires all of us journalists, scientists, and the general public to ask questions. And it's asking those questions, raising skepticism, not just being the recipients of the official wisdom that's so important in this process,
Marty Martin 57:12
And it's definitely a great place to wrap up. Thank you so much, again, for coming on.
Art Woods 57:16
Thank you, Charles. Really great conversation.
Charles Piller 57:18
My pleasure. Thank you.
[Music break]
MM: Thanks for listening to this episode! If you like what you hear, let us know via Twitter, Facebook, Instagram, or leave a review wherever you get your podcasts. And if you don’t, well we’d love to know that too. All feedback is good feedback!
AW: Thanks to Steve Lane, who manages the website, and Ruth Demree and Brad van Paridon for producing the episode.
MM: Thanks also to interns Dayna De La Cruz, Daniella Garcia Almeida, Kailey McCain, and Kyle Smith for helping produce this episode. Keating Shahmehri produces our awesome cover art.
AW: Thanks to the College of Public Health at the University of South Florida, the College of Humanities and Sciences at the University of Montana, and the National Science Foundation for support.
MM: Music on the episode is from Podington Bear and Tieren Costello.