“The James Randi Educational Foundation (JREF)”
by Craig Weiler
JREF is a skeptical organization supposedly devoted to promoting critical thinking regarding claims of the paranormal. In reality it is an advocacy organization, known in politics as a pressure group. They lobby media and science organizations to dissuade them from taking parapsychology and psychics seriously. Like other pressure groups, they occasionally perform high profile publicity stunts to attract attention.
In fact, they are a magnet for controversy and scandal. The president of JREF, D.J. Grothe, has recently been accused of “misogyny and disrespect for women coworkers,” and, “constant duplicity, dishonesty, and manipulation” by a female employee. James Randi’s significant other, Deyvi Pena, was convicted of Identity Theft; a disgruntled million dollar challenge applicant put out a $100,000 reward to anyone who could prove that the challenge was legitimate, and there is a long list of complaints by people who have either applied for the challenge or taken it. The challenge itself is the subject of unending criticism:
“Psychic offered a million dollars to prove his abilities.” How many times have you seen that headline? James Randi, a magician, offers a million dollars to any person who can prove they possess psychic abilities. This is done through the James Randi Educational Foundation (JREF for short), and is referred to as the million dollar challenge (MDC for short).
Every few months a story pops up in a prominent magazine about a prominent psychic who has been challenged to prove their abilities by taking the MDC. Celebrity psychics such as the late Sylvia Browne, James Van Praagh and John Edward have all been goaded at various times to apply for the challenge. All have declined.
Because the MDC is perceived in the media as a legitimate way to test psychic ability, declining to take the challenge is promoted as proof that the psychics are actually charlatans. Over the years, many people have applied for the challenge, a very tiny percentage has been tested, and no one has passed even the preliminary part. Skeptics point to this as proof that psychic ability does not exist.
Mr. Randi is a very popular skeptic and the million dollar challenge is easy to understand and seems to provide a clear and easy way to establish the truth about psychic ability. Because the MDC is rather popular in mainstream periodical literature, it merits serious investigation.
A Review of the Literature
Parapsychological literature sheds little light on the workings of this challenge. There are no scientific papers reviewing the MDC and it is mentioned only briefly in some books about parapsychology. The most influential book in parapsychology, The Conscious Universe, by Dean Radin, spends only a sentence on Randi:
“They [Geller or Randi] are actually so irrelevant to the scientific evaluation of psi that not a single experiment involving either person is included among the thousand studies reviewed in meta-analyses.”
In the two most influential books that specifically address parapsychology skepticism, the JREF million dollar challenge gets only the briefest mention.
Chris Carter, in his book Parapsychology and the Skeptics, devotes a mere four and half pages to Randi without examining the challenge except to say this:
“The problem with this test is that Randi himself acts as policeman, judge and jury. Given his countless disparaging and insulting remarks concerning parapsychology, and his financial stake in the debunking movement, he can hardly be considered to be an unbiased observer.” 
Robert McLuhan, despite the title of his book being Randi’s Prize, has even less to say about the challenge, devoting only a few paragraphs to it:
“Randi himself laments that none of the stars in the psychic firmament – John Edward or Uri Geller for instance – has entered for it, (…) Another view, of course, is that, unlike the naïve individuals who actually do apply for the prize, they have more sense than to put themselves in the hands of a crusading sceptic who considers them to be the scum of the earth. (…)
“To offer an analogy: the difference between parapsychology and Randi’s prize is the difference between a fleet of boats heading out to sea equipped with radar and large nets, and one man sitting beside a muddy stream waiting for fish to jump in his net.”
What is apparent is that scientists and scholars of parapsychology feel that the challenge is so insignificant as to not merit any significant consideration. Some serious examinations of the challenge do exist in blog posts on the Internet. Greg Taylor at The Daily Grail, published a very thorough article that examined it:
“First, and perhaps the most important, is the effect size required to win the challenge. While the JREF says that ‘all tests are designed with the participation and approval of the applicant’, this does not mean that the tests are fair scientific tests. The JREF need to protect a very large amount of money from possible ‘long-range shots’, and as such they ask for extremely significant results before paying out – much higher than are generally accepted in scientific research (and if you don’t agree to terms, your application is rejected).”
Both the blog post by The Daily Grail, and another by Michael Prescott , questioned the rules for the challenge, pointing out logical errors and draconian terms in the application. For example, rules #4 and #8 allow JREF to use information as it sees fit and the applicant surrenders all rights to legal action. In other words, if the organization decided to lie and cheat the applicant cannot sue for damages.
Over the years, I have published a couple of articles on the Million Dollar Challenge on my blog, which have made their way into various on line discussions about the merits of the challenge.
My main interest has been in the workings of the challenge. According to Wikipedia:
“In the October 1981 issue of Fate, Rawlins quoted him [James Randi] as saying “I always have an out”.  Randi has stated that Rawlins did not give the entire quotation. Randi actually said “Concerning the challenge, I always have an ‘out’: I’m right!“ . Randi states that the phrase “I always have an out” refers to the fact that he does not allow test subjects to cheat.” 
Examining James Randi’s Character
As the organization bears his name, this invites questioning about James Randi’s character. Does Randi have an out? Is there some method he uses to make sure applicants never win? All of the criticisms of the challenge that I’ve read don’t address this. They point out, correctly, that the challenge is unrealistically hard and that Randi, who is considered to be far from impartial, totally controls it.
The skeptical point of view is that Randi needs to control the challenge in order to prevent alleged psychics from cheating, and that he is qualified based on his considerable experience in magic to expose frauds. James Randi is a very accomplished magician and this does qualify him to expose people who are posing as psychics but are actually using magic tricks to dupe people. However, Randi has no scientific education, self-taught or otherwise. His critics contend that total control over the challenge allows Randi to cheat, or to create unrealistic rules that no one could satisfy in order to win.
In order to deal with the efficacy of Randi’s challenge, we have to examine the character of James Randi. If he has a genuine interest in the truth, we can rely on his good judgment. So we look first at what his critics have to say.
While psi proponents acknowledge his considerable magic skills and that he has exposed a few frauds posing as psychics, he is widely regarded as deeply biased and more interested in publicity than the truth. Biologist Rupert Sheldrake, a target of Randi’s criticisms, uses this widely circulated story to illustrate that point:
“The January 2000 issue of Dog World magazine included an article on a possible sixth sense in dogs, which discussed some of my research. In this article Randi was quoted as saying that in relation to canine ESP, “We at the JREF [James Randi Educational Foundation] have tested these claims. They fail.” No details were given of these tests.
I emailed James Randi to ask for details of this JREF research. He did not reply. He ignored a second request for information too.
I then asked members of the JREF Scientific Advisory Board to help me find out more about this claim. They did indeed help by advising Randi to reply. In an email sent on February 6, 2000 he told me that the tests he referred to were not done at the JREF, but took place “years ago” and were “informal”. They involved two dogs belonging to a friend of his that he observed over a two-week period. All records had been lost. He wrote: “I overstated my case for doubting the reality of dog ESP based on the small amount of data I obtained. It was rash and improper of me to do so.” 
Randi also claimed that in a tape of a dog experiment that Sheldrake had performed, the dog was responding to every passing car. He was later forced to admit that he had never seen the tape.
It is safe to say that no parapsychologist or paranormal investigator would ever work alongside Randi. In one telling instance, he was banned by the family and the investigators from entering a house where poltergeist activity was supposedly occurring. Any testimony to Randi’s integrity and honesty will not be found in the opinions of his opponents. Psi proponent Victor Zammit goes so far as to write:
“In fact his conduct shows him to be a conman, a mind-manipulator and someone who himself admits – and this is a matter of public record – to being highly skilled in deception, trickery and conning.”
In Will Storr’s “The Heretics” he gets a stunning confession from Randi:
“Is James Randi a liar? I begin gently, by telling him that my research has painted a picture of a clever man who is often right, but who has a certain element to his personality, which leads him to overstate.
‘Oh, I agree,’ he says.
‘And sometimes lie. Get carried away.’
‘Oh, I agree. No question of that. I don’t know whether the lies are conscious lies all the time,’ he says. ‘But there can be untruths.’ 
We next turn to skeptics’ perceptions of Randi. Does the skeptical community hold him in high esteem? Some do. In 1986, Randi was awarded a MacArthur Foundation Genius award for his work on exposing frauds.
Some skeptics are less generous. Former parapsychologist and CSI fellow Susan Blackmore reviewed Randi’s book, The Supernatural A-Z, and commented that the book “has too many errors to be recommended.”
Ray Hyman, a longtime skeptic and leading CSI fellow who has contributed more to the field of parapsychology than any other skeptic, noted:
“Scientists don’t settle issues with a single test, so even if someone does win a big cash prize in a demonstration, this isn’t going to convince anyone. Proof in science happens through replication, not through single experiments.”
Hyman and Blackmore are scientists who are among a very tiny handful of skeptics who have actual expertise in parapsychology and have made contributions to the field. While they do not criticize Randi directly, they lightly regard his scholarship and grasp of science.
Randi has been caught red-handed plagiarizing from skeptics on his own forum. He took comments from a forum user known as “Hawkeye” and changed the wording. When confronted, Randi responded with this comment:
“Chris: I admit, I shamelessly took your comments and dropped them in as part of SWIFT, simply because they exactly reflected my observations. I could have changed the wording, but getting SWIFT together each week – amid all the other duties that keep me here at least 60 hours a week – calls for some corner-cutting every now and then. Mea culpa…
“I see two real problems with Randi plagiarizing or otherwise cheating for any reason at all. The first is that the nature of his life’s work demands that he act with 100% honesty and integrity, because that’s the standard he’s demanding from those he exposes. Why else would Randi pursue cheats unless he thinks cheating is bad? So why then is it OK to give the excuse “oops, you caught me in a blatant cheat, I was busy that week?”
Or are we saying that cheating is OK as long as you admit it when you get caught? I hope we’re not saying that. 
The most serious damage to Randi’s integrity came from a long-term case of identity theft. Randi, who is gay, has a significant other, actually named Deyvi Pena, who went by the name José Luis Alvarez for twenty years before being caught in 2012. Somehow, Randi mistook Deyvi Pena, a young man from Venezuela on a student visa to study at the Art Institute of Fort Lauderdale, for a teacher from the Bronx.  Either Randi was duped by an obvious con right under his nose for many years or he knowingly conspired with Pena to hide the truth.
So in regards to whether James Randi has impeccable character, the answer is clearly no, he does not. He is apparently willing to abandon honesty and integrity when it suits him. He does not have enough personal credibility to be taken at his word and his detractors have legitimate reasons for not trusting him. The million dollar challenge is somewhat suspect on this issue alone, although he’s not in charge of it anymore. For that reason it is necessary to look at how the challenge is run.
Examining the Challenge
How exactly does the challenge work? What is the procedure for taking an applicant from start to finish? This information is not readily available and I have seen no formal explanation from JREF explaining this process in detail.
On the surface, the million dollar challenge seems legitimate. It seems as though skeptics work on a protocol with psychics until a final procedure is hammered out. But Randi’s people working on the protocol are not vetted in any meaningful way. There is no requirement that they understand scientific protocol or be able to conduct a scientific test. In the challenge forums I visited, no one, for example, seemed to take the experimenter effect seriously. There seemed to be an attitude that psychic ability was something that should function on demand, and testers did not have the specialized knowledge in parapsychology which would be necessary to design proper experiments.
The most glaring problems with the million dollar challenge come from rules that can change on JREF’s whim.
Scientific testing of psychic ability is statistical. That is to say, an effect is considered real when it is shown to not be due to randomness (or problems in the protocol). You do this by calculating the odds that something might occur due to chance. In the results you get from repeated tries, the higher the odds are against chance, the more likely it is that psychic ability is in play.
I’m explaining this because this crucial information is missing. You won’t find it on the application or the FAQ for the million dollar challenge. It should say somewhere that the preliminary test must overcome odds against chance of approximately 1,000 to 1, but it doesn’t. And rumor has it that to win the challenge the applicant must overcome odds of one million to one. This kind of information is crucial to understanding how hard the challenge is.
An analogy would be to have a jumping contest to discover whether jumping was possible, but to not state anywhere how high a person had to jump in order to win.
Investigating through the forums
When I initially investigated the challenge, this is what I found: Applicants for the challenge were given their own thread on a sub forum specifically for the challenge, which is the only way for an outsider to track an applicant’s progress. The forum appeared to be run by volunteers and it was done very much on the cheap. Much of the information passed through the forum, some of it went through the mail and some of it was emailed. It was clear from reading the forum posts that the challenge process was a disorganized mess. The applicants dealt primarily with the volunteers, except when dealing with the staff, who apparently didn’t always notify the volunteers about what they were doing. (The volunteers sometimes found out what had transpired from the applicants.)
Randi could swoop in at any moment and change whatever he wished without notifying anyone or giving any justification for what he did. The volunteers seemed to be left to fend for themselves and had no authority to move an applicant forward in the process no matter how much work they’d done with that individual. One of the signs this was badly run was that very few applicants ever got to the testing stage.
And the application process is dreadfully slow. A process that takes a year to two years is not unheard of. In the course of the Ziborov attempt, Startz (a forum name for a JREF MDC volunteer) made this comment:
“In fairness to Pavel, he has presented statistically sound protocols. JREF has been rather unresponsive as to what objections they have so that Pavel can revise them in accord with JREF’s wishes.”
Let me be more pointed than, as a fan of JREF, I wish were necessary. JREF has asked for communications to be done by email. When I have done as JREF has asked, JREF has not had the courtesy to return emails. If JREF were one of my PhD students, rather than an organization with a long, successful track record, I would say this in a less pleasant way.
Remie has sensibly pointed out that negotiations are better done by email than through this public forum. Following this wise advice, I have (on Pavel’s behalf) sent in protocols by email (while posting informational copies to the forum). JREF’s responses have been through the forum. There is no reason this could not have been settled in a week of back-and-forth email messages. Nearly all the delay time has been on JREF’s end, not Pavel’s. 
By forcing all the applicants to make a specific claim and set up a protocol, JREF is making the process difficult for people who have no experience in doing such things. Psychics are not scientists.
There is almost no transparency in the process and no attempt is made to satisfy outside objective observers that the testing is fair. There is no log of people who have been tested. A complete report on the individual testing that would explain in detail what occurred does not exist. 
PR Stunt or Serious Inquiry?
The easy way to tell a PR stunt from a serious inquiry is the way in which information is handled. A PR stunt requires important information to be controlled, and a serious inquiry requires all important information to be disclosed. The JREF million dollar challenge is replete with information being controlled and makes no full disclosures. Here is an example:
The question of exactly how many people have been tested is obviously deliberately obfuscated. In the FAQ for the challenge one finds:
“(3) To date, how many persons have been tested for the million-dollar prize offered by JREF?”
That’s not a simple question to answer. Many hundreds have made application, and most have had to be instructed to reapply sometimes several times because they did it incorrectly or incompletely. There are, at any given time, about 40 to 60 applicants being considered, but from experience we know that the vast majority will drop out even before any proper preliminary test can be designed. Of those who get to the preliminary stage, perhaps half will actually be tested, and some of those will quit before completion.”
It is a simple question. Just add up the number of people who have been tested. (It appears that no more than a couple of people a year are actually tested, so I would put that number between 20 and 30.) Hiding this number is an obvious attempt to make the number look bigger than it actually is. This subterfuge is especially apparent on the Wikipedia entry for the MDC:
“To date, over 1,000 applications have been filed but no one has passed a preliminary test, which is set up and agreed upon by both Randi and the applicant. 
These entries into Wikipedia are done under the auspices of an organization known as the Guerrilla Skeptics, who I’ll cover in a later chapter. They work closely with JREF, so it is no accident that applications have been confounded with actual testing. It is safe to say that the confusion of who has applied versus who has been tested is deliberate.
Investigating a Legitimate Applicant:
Sometime in 2009 I started investigating an application for Pavel Ziborov that had just played out. At the time, the discussion forum for challenge applicants was open to public viewing, so I was able to follow what happened. I discovered foul play by Randi and wrote it up in my blog. Sometime after that, the applicant forum was closed to public viewing. What is there now is nothing more than a brief summary, which, in Pavel Ziborov’s case, is inaccurate.
Pavel Ziborov’s Attempt to be Tested
Around May of 2007. Pavel Ziborov contacted JREF to apply for the challenge. After two years and almost 900 forum post plus emails and letters, he and the volunteers had agreed on a straightforward protocol. Ziborov was to determine whether envelopes held a black or white piece of paper (50% chance) with 100 trials. It was agreed that he needed 67% correct answers to win (odds of that being chance are 1,000 to 1). When this was submitted to Randi, it was changed to 20 trials with this explanation to Pavel:
Thank you for your continued patience. Now that the TAM dust has cleared, we can again take a look at your protocol. As I said before, I asked other JREF staff to weigh in on whether or not they believed your proposed protocol was workable.
Mr. Randi said:
Suggest that he merely identify for us which of two photos are in an envelope, 20 times. We cannot satisfy each and every whim, and it’s too expensive.
I’d say, if he refuses, he’s refused to be tested.
What do you think of simplifying the protocol to that level? Is that a possibility?
If not, I will do as Mr. Randi has suggested, and close your file.
James Randi Educational Foundation”
This is straight up unethical. In addition to framing Pavel’s possible refusal as chickening out:
1. Randi violated his own rule that applicants have eight hours to complete their challenge.
2. Pavel still had to comply with the 1,000 to 1 odds, so in order to achieve this with 20 trials he would have had to have a success rate of 80%, where he had claimed to be able to achieve 67%. He was being asked to succeed at something he never said he could do.
3. The take it or leave it demand violates the condition that “both parties have to agree to the protocol.”
4. All of the JREF volunteers who had worked with Pavel were thrown under the bus as JREF blithely disregarded the protocol they had come up with.
The skeptic volunteers who worked on this application had worked with Pavel through several iterations of how to conduct tests where they had to teach him a bit of science. They did this because they believed in James Randi and JREF. His volunteers were betrayed.
In fact, many skeptics could not turn a blind eye to such an obvious miscarriage of justice. One of the people involved in the Pavel application wrote:
“I realize that there is almost no interest in holding Randi and the MDC to the standards that they claim for themselves. I’ve always been in a ridiculed minority when I make these suggestions. It is clear that the Challenge is not about allowing people to demonstrate their claims, but rather about providing examples for our ridicule – partly for education, partly for group-bonding (my guesses). I am in the process of moving on from the idea of trying to persuade anyone to care to that of trying to get the JREF and Randi to be more upfront about this instead, in order to thwart criticism. I fully realize that this will be a futile effort as well.
This comment was buried in a forum thread only a handful of people will ever read. Rather than show their warts, JREF has provided a handy little synopsis of the outcome of the Ziborov application:
“In accordance with the suggestions from other JREF staff, Pavel was given one last opportunity to simplify his protocol. He has declined, and his Challenge file has been closed.
Pavel will have the opportunity to re-apply for the Challenge in one year, assuming he qualifies under the guidelines governing the Challenge at that time. 
This kind if dissembling is an indication that the JREF organization doesn’t take their own challenge seriously. While they have acknowledged that it is a publicity stunt, it is this sort of organizational behavior that demonstrates something worse: outright dishonesty.
Other Challenge Applicants
I can’t begin to list all the applicants who have complained about the challenge or investigate whether their protestations are valid. But here’s one example of how irritated applicants have become over how they were treated. Homeopath John Benneth, who claims to have been stonewalled by Randi in his attempt to get tested, has issued a $100,000 challenge to anyone who can prove that the JREF million dollar challenge is legitimate.  No one has come forward to attempt to claim this prize.
Most of the complaints revolve around not getting tested at all despite numerous attempts. However, one complaint had to do with Randi’s behavior during testing.
One contestant who actually got to the challenge and had some initial success was 11 year old Natalya Lulova. In a trial where she was demonstrating that she could “see” without her eyes, Randi responded by claiming that the bridge of her nose was special, allowing her to see underneath her blindfold. He then did a massive taping job that, according to her guardian, left her in tears and unable to perform.  The website referenced in the footnote has pictures and a narrative of the event.
What about the Actual Test?
How about the actual tests that have been performed that have not demonstrated any psychic ability? Are these legitimate tests of psychic ability? If psychic ability exists, why has no one passed the test? In 1991, The TV show, James Randi : Psychic Investigator, employed many tricks to prevent actual demonstrations of psychic ability.
Astrophysicist Sam Nichols, who attended one of the shows, enumerated a long list of obvious deficiencies including this significant one:
“Never let the psychic get comfortable enough to feel settled; the guests were more or less dragged on stage with barely an introduction and then expected to exhibit psychic marvels.
The idea behind the challenge is to expose frauds and delusional people who don’t have the ability they claim to have. However, if you make the challenge impossible to fulfill, then you haven’t proved anything, as noted by “Cuddles” on the Ziborov forum thread:
“- It is therefore extremely important for the JREF to ensure that a test is fair to an applicant by ensuring that there is a high probability of success should they actually possess the claimed ability.
Short tests are more difficult than longer ones for psychological reasons. The pressure to succeed on any individual attempt is much higher and the applicant has no chance to relax. Patricia Putt had ten trials.  Pavel was allowed to have twenty. A 1979 test had three trials for four dowsers , and a recent test with Derek Ogilvie had ten trials. 
The test is high profile, as a failure will be highly publicized. Psychic ability also declines in the presence of skeptics. This has been proved in scientific testing and is known as the experimenter effect. In Randi’s tests, the applicants are surrounded by people who hope failure will occur.
Even if an applicant freely signs up for this sort of setup, as Patricia Putt did, a short, high pressure, high profile test run by skeptics is no legitimate way to test for psychic ability. That people fail, often in spectacular fashion (Putt did not get a single trial right), doesn’t say anything about the abilities of the people who were tested except that they didn’t succeed under these very adverse conditions.
In Putt’s case, the problem had to do with the way she was tested. She normally speaks, but was required to write out her psychic readings. She was being asked to succeed at something she had no training at and had not claimed that she could do.
The test protocol was presented to her as take-it-or-leave-it. This was a violation of standard scientific testing protocols. She normally does under five readings a day and she was required to do ten. Portions of her readings were blacked out by Prof. Richard Wiseman who was conducting the test. Doing ten readings in one day is far more than most mediums do. She reported being mentally exhausted by the eighth reading . Other problems included having all young, female students as target subjects, making their experiences and personalities difficult to tell apart. Like other applicants, she did not have much control in the study design.
To sum this up, while the test had good controls against cheating, it was very poor at providing elements that were favorable to psychic functioning. An analogy would be putting a seed on a shelf to see if it will grow. The experiment is perfectly controlled, but is guaranteed to fail.
All you can say is that the test demonstrated nothing. Patricia Putt cannot be said to have failed the test because the design was completely inadequate to test anything. That did not stop the press releases of course.
To put this in perspective, the closest comparison to this test is probably The Afterlife Experiments conducted by Gary Schwartz, which claimed to demonstrate statistically significant positive results from psychic mediums. The tests ran over several days and involved multiple mediums. Various versions of the same tests were run and all the details and results were published. The experiments have been refined and replicated over the years and notable skeptical scientists have reviewed the literature and commented on it and it has become a part of parapsychological literature.
No reports of the psychic testing done for the million dollar challenge has ever made it into parapsychological literature. Zero. Not one.
All that seems to happen is that when one of these tests is performed, the news travels around, a few newspapers pick it up, a few people blog about it and argue about it on the forums, and then it fades into history. Because it’s not important.
There are obvious reasons for their insignificance:
1. Very few serious applicants for the challenge: With the age of the Internet upon us, potential applicants are much better informed than in years past. Even the most cursory Internet search will yield damning information about James Randi and the challenge. So only the most naïve people will pursue the prize.
2. No overall methodology: Every once in a while someone braves the system and actually gets tested with an experiment that is unique, with very few trials and no replications. This renders the results useless as a measurement for anything.
3. Lack of data. To seriously discuss the relevance of any particular test requires that all details of that test be available for examination and critique.
The only thing you can say about the JREF million dollar challenge is that it is a publicity stunt that does not do what it claims to do: legitimately test for psychic ability. Because of the poor way that testing is managed, the tests themselves are not indicators of anything, much less psychic ability. In its present form this challenge is, for all practical purposes, unwinnable.
It begs the question: if JREF is so sure about psychic ability not existing, why do they have to resort to this deceit?
Chapter From the Book:
TED, Wikipedia and the Battle for the Internet: The Story of a Wild and Vicious Science Controversy . . . that Anyone Can Join!
Craig Weiler, CreateSpace, 2013
© 2014 The Association for Skeptical Investigation. All rights reserved.