"The 'effect' researchers are really talking about pertains to the old hypothesis many atheists still hold, which is that people not smart enough when it comes to logical reasoning are also the people not smart enough to reject the notion that there is a God."
What Does Math Skill Have to Do With Faith?
Critical Meditations on the Study, "Divine Intuition: Cognitive Style Influences Belief in God"
in ref to:
Shenhav, A., Rand, D. G., & Greene, J. D. (2011, September 19). Divine Intuition: Cognitive Style Influences Belief in God. Journal of Experimental Psychology: General. Available online.
Kudos, great men of the elitist halls of Harvard, on publishing yet another study that atheists probably can’t wait to get their hands on. Under the guise of an “objective” science, you have unleashed more scientism for both fellow scientists and the American public. I guess we live in an age when absolutely nothing can escape a scientific evaluation of it, but surely you’re each Bright enough to have realized beforehand that you have a piece of literature here that holds the potential to deal believers a serious blow, yes? Ethically, if you claim to be studying the psychology of faith, I would hope you would be a bit more sensitive how you design your studies and present your “predictions” about the way people who believe in God will behave. But that is apparently too much to ask when you are so overconfident about your research you gloss right over what you don’t really know. Please, allow me just enough arrogance to critique your study, not that I expect you to care now that (after what can only be rigorous peer review) it’s been forever awarded a place in the Journal of Experimental Psychology. Heaven forbid someone might be offended by the way truth is revealed by science. Count me among the disgruntled. I’m not going to address all the problems I have with this report, but I would like to start with a careful review of Phase 3 (because even the order in which the researchers present their phases skews your perspective in favor of supporting their hypothesis).
Phase 3: The "Success" of Inducing Intuition to Cause Belief in God
Phase 3 of the report is an experiment (2 x 2 factorial design) in which participants were randomly assigned to one of four conditions. Before being asked whether they're convinced God exists, the participants are first (supposedly) "primed" by being instructed to write a paragraph about one of the following:
Group 1. How intuition/"first instincts" in a situation led to a positive outcome.
Group 2. How using careful reasoning in a situation led to a positive outcome.
Group 3. How intuition/"first instincts" in a situation led to a negative outcome.
Group 4. How using careful reasoning in a situation led to a negative outcome.
From these conditions, it was predicted that by "inducing a mindset" focused on intuitive thinking that led to a positive outcome, (group 1 above), and by doing what researchers believe is "opposite" this: inducing a mindset focused on how careful reasoning in a situation (or so-called "reflective" thinking) led to a negative outcome (group 4), there would be a higher number of people in these groups who agree they're convinced God exists compared to the other two groups. The point here is to make intuitive thinking seem favorable to half the participants. Researchers believe that intuitive cognition is somehow linked to belief in God, and here they are trying to show intuitive thinking actually causes people to agree they believe there's a God.
The subjects of most concern are the ones asked to write a paragraph about when intuitive thinking led to a good outcome, which according to the researchers' theory on "divine intuition" should have the strongest effect. I have to stop at this point and ask: How is it that researchers are determining who believes in God? From the study's graphs, we see that participants are asked if they agree with the statement:
"I have had an experience that has convinced me that God exists."
What exactly does it mean when someone answers "yes" to this ambiguously worded question? Does it mean a) there has been at least one time in my life I've been convinced God exists but I don't anymore, b) there has been at least one time in my life I've been convinced God exists and I still believe God exists, or c) there has been at least one time in my life I've been convinced God exists but I'm not sure God exists? The "forced-choice" format of the original question demands participants to say yes or no to a question that has at least three different interpretations. However, in support of their hypothesis that intuitive thinking causes belief in God, researchers have decided that when anyone agrees with the statement it only means "b" above. Why is this a problem if all participants are asked the same question? Because we don't know exactly what researchers are measuring. On top of this, the measure of the dependent variable is not "belief in God," as written into the title of the journal article. The measure of the dependent variable is reported belief in God, and there can very well be a degree of difference between actual belief in God and reported belief in all phases of the study. To what degree? We simply don't know. However, throughout the journal article researchers use the two (reported vs. actual belief) as if they are simply interchangeable.
I also have to stop at this point to ask, What specifically is being primed when participants are asked to write a paragraph about how intuition led to a good outcome? Are they priming 1) a cognitive mode of intuitive thinking, 2) a high appraisal of intuitive thinking, or 3) given it's a task that still involves verbal intellect--are participants still using logical thinking but to write about intuition? Researchers seem to think they are only priming intuitive thinking. Are they correct? We simply don't know.
Assuming the priming was as successful as researchers see it, was there an overall (main) effect of either the one independent variable, cognitive style (intuitive or reflective), or the other independent variable, outcome of the written paragraph (positive or negative) on the dependent variable (belief in God)? The researchers make good of immediately telling you that a higher number of people agreed they believe there's a God if they wrote paragraphs that "vindicated intuition," but then they admit after they did the factorial analysis, "Neither factor exhibited a main effect." No main effects means that overall, when evaluating individually the effect each independent variable had on the dependent variable, the independent variable had no effect on the dependent variable. Hence, at this point the results indicate that it's not certain whether or not inducing a mindset that favors intuitive thinking causes a person to more likely say they believe in God (hold on and I'll tell you what they do have evidence of). The researchers still over-confidently manage to suggest in the abstract (which is what most people only read anyway), "Three studies support" that intuitive thinkers are more likely to report they believe in God, but you have to read much further than the abstract to understand exactly how did the experimental phase provide "support." As usual, the news media uncritically accepted their perspective so that when it comes to not simply reported belief in God but belief in God itself, as Yahoo News now sees it, the matter is as simple as "triggering intuitive thinking boosts belief" ( see http://news.yahoo.com/belief-god-boils-down-gut-feeling-104403461.html ), which is particularly misleading based on what evidence was found in the experimental phase of this study.
What if researchers found a way to demonstrate that priming intuition can cause more people on average to report they believe in God, but only if the priming is carried out in a very precise way can you actually get an effect? Would this now be compelling evidence for the one true brain process explaining belief in God? Would it have also demonstrated why the people who say they don't believe there's a God are more likely to be correct in their use of a careful "reflective" reasoning in attempts to apply this to the existence of God? In case you can't tell from how confident researchers are in reporting their results, if I can be allowed to make a humble prediction of my own, the focus of future research will be guided by a burning desire to demonstrate that an intuitive thinking style is part of a causal chain for belief in God. (At the moment, they have some people believing it's the primary cause.) Scientists studying human behavior tend to use a cookbook recipe approach: this plus that plus a variable like "intuition" equals belief in God. What they never tell you is that they don't know if they're leaving out hidden ingredients that are vital for the cake to fully rise. While they focus on how much flour you need to bake the cake, their end product comes out of the oven flat and and already stale. Perhaps worst of all, through their overgeneralizations about how human beings think and behave (sometimes called laws of uniformity) they also leave out what spices might alter the taste of the cake for individuals. Underscoring this mess left in the kitchen are the cooks who think no phenomenon of interest concerning human behavior can escape a scientific investigation of it. Experimental analogs are devised that don't necessarily fit the real world any better than a metaphor. They turn up the heat with hopes that it will make the cake rise, and if you can't stand the heat (not to be confounded with the truth) you know what you can do (get out of the kitchen).
The other thing many scientists don't concern themselves with when they report their research results is the problem of first causes. We demonstrate that A causes B, but what if there's something important that causes A that we are completely overlooking in our understanding of covert behavior such as belief in God? (Please don't tell me about the "God gene.") As we will also see in another phase of the study at hand, researchers assume that an intuitive approach causes a person to get a math problem wrong, but this is because to solve a math problem "correctly" requires logical thinking and mathematical skills. What actually caused the people in this study to approach a math problem using intuition? It doesn't matter to researchers because they think they need not concern themselves with first causes. But they go to another level in their inferences when they find something like the people who didn't use logical reasoning to solve a math problem also just happen to be the people more likely to report believing there's a God, and this is not a simple matter of the "facts." It's a matter of inferring that if people use intuitive thinking to solve a math problem, then their approach regarding the existence of God is done in the same simple manner. Conversely, researchers suggest that if people use "reflective" logical reasoning to solve a math problem, their approach regarding the existence of God will not be done in the same simplistic (or "primitive") manner as the people who use intuition. Indirectly, this also suggests that the people using the kind of intelligence that favors logical reasoning will also get the correct answer regarding the existence of God (i.e., God doesn't exist).
The Special "Effect" of Interaction
What Phase 3 of the research at hand did find is a significant "crossover interaction" between the two thinking strategies (intuitive or reflective) and the two outcomes (positive or negative). In general, an interaction occurs when the effect one independent variable (IV 1) has on the dependent variable (DV) depends on the level of another independent variable (IV 2). Does the effect of writing a paragraph about using one of two cognitive styles to solve a problem--intuitive or reflective (inducing mindset, as researchers call it, which is IV 1), change the degree subjects report they believe in God (DV) but depends on the outcome written in the paragraph being good or bad (IV 2)? Yes, it does. In fact, in reverse fashion (hence the authors claiming there was a "crossover interaction"). Their results show that the effect of priming intuitive or reflective cognitive style depended on which outcome (positive or negative) participants had to write about , or said the other way, the effect of the outcome in the written paragraph (positive or negative) depended on whether it was for an intuitive or reflective cognitive style. All this means is that the way cognitive style made reported belief in God go up was if the intuitive-positive outcome or reflective-negative outcome was the subject of writing, and conversely, cognitive style made reported belief in God go down when the intuitive-negative outcome or reflective-positive outcome was the subject of writing. The interaction itself has more to say about how writing about intuitive vs. reflective thinking needs to be precisely engineered (perhaps in future research) if you want to use it to influence a measure such as reported belief in God.
I'll say this again another way because I'm sure someone out there wants to go so far as to hack into my computer to give me a lesson on factorial analysis. If we focus for the moment only on the two levels in the intuitive priming condition, the direction (increase or decrease) of reported belief in God depended on the outcome written into the paragraph. When in the intuitive-positive outcome condition participants were significantly more likely to report belief in God. But when in the intuitive-negative outcome condition the opposite occurred, i.e., there was less reported belief in God. This shows you that it is indeed no simple matter to induce an "intuitive mindset" that will lead to reporting belief in God. The "effect" of doing so in the present study was statistically significant (i.e., didn't occur by chance alone) but the increase in reported belief required inclusion of an additional variable so that, on average, among the intuitive cognitive style only those writing about a positive outcome had more reported belief in God. The difference between the two intuitive conditions is profound? Compelling? It's only about 13% (see the bar graph at Figure 1.) When you look at the numbers, statisticians might feel content saying the reason for no main effects is because of finding a crossover interaction (i.e., the way the numbers cancel each other out in the factorial analysis), which is true. However, had the experimental design been one that yielded a main effect (for intuitive vs reflective priming) in addition to the interaction there would be less room for doubt regarding the simple notion that intuitive thinking affects reported belief in God, and I can't stress enough that the interaction they found required the study to be designed with two possible outcomes to the written paragraph. The added variable made it more likely that they could obtain statistically significant difference of some kind between groups and use it to somehow support their causal hypothesis.
There's yet a third way to look at this in which you can see the way researchers interpret their results is just a bit deceptive, and this is in regard to something known as accepting or rejecting a null hypothesis. I know this might seem a bit confusing, but I didn't invent scientific research methods. The predicted outcome of an experiment is called the research (or alternative) hypothesis. A null hypothesis is the predicted outcome of an experiment stated in the opposite way. The wording of a null hypothesis is quite crucial. You risk a certain type of error if you reject a null hypothesis when you shouldn't (Type I Error), or if you don't reject a null hypothesis but you should have (Type II Error). The question is which null hypothesis do researchers have sufficient evidence to reject:
A. Priming intuitive thinking does not cause more (reported) belief in God.
B. Priming intutive thinking does not cause more (reported) belief in God if the outcome in the written paragraph used to prime intuition is positive.
Stated as plainly as it is, null hypothesis A says the independent variable, intuitive priming by itself, will not cause more reported belief in God. Null hypothesis B above is one that is conditional, i.e., it depends on more than just the independent variable of "intuition." Had this experiment found a statistically significant main effect for primed intuitive cognitive style across both levels of this independent variable (positive or negative outcome), it would allow them to reject null hypothesis A. But because researchers found only a significant interaction with no main effect, this only allows for rejecting B. Because an interaction is one that relates to a conditional hypothesis that depends on the presence of more than one variable, the null needed to have been written as B above. Without having found a statistically significant main effect, rejecting null hypothesis A is an example of a Type I Error. The researchers undoubtedly think they have enough statistical power in their study to reject both A and B as null hypotheses not merely in terms of reported belief in God but in terms of belief in God itself. Much closer to the truth is that null hypothesis A is lacking in precise terminology and is far too vague to reject given the outcome of their experiment.
Remember this too: An experiment such as this is supposed to be an analog of how people think in the real world, and nobody is going about manipulating people's minds by telling them to write about the virtues of logical reasoning or intuition. The more the analog deviates from the real world the more ecological validity (how your experiment pertains to the real world) becomes a real issue. Beyond using the tool for manipulation found in the present experiment and its effect on reporting belief in God , it's still yet to be shown that when you simply "induce the mindset" of intuitive cognitive style it causes a person to believe in God.
Figure 1. The bar graph to the left is how researchers chose to depict results of Phase 3 in which there were no main effects between the two levels of independent variables (priming conditions and outcomes) on the dependent variable (reporting belief in God), but they did find an interaction between the so-called "primed" cognitive styles (intuitive or reflective) and the outcome of what participants had to write about (positive or negative). The choice of a bar graph allows researchers to emphasize what still looks like a large effect regarding how intuition apparently favors reported belief in God. A line graph (right) is what other researchers typically use to graph results when finding an interaction. Note how the line graph does not play up the differences between groups and in an unbiased manner emphasizes only what was found, which is a crossover interaction.
Looking at the chart they provide (Figure 1. left), for the dependent variable (in the form of asking participants to agree or disagree with the statement, "I have had experience that has convinced me that God exists") after priming intuition as a favorable outcome, 63% of the participants agreed, but that means 37% of these primed subjects still said they don't believe in God. There's no way of knowing if, despite randomization, this group just happened to have more individuals who believe in God, but if so the effect of priming still did not cause more than one third of the participants in this group to behave according to predictions. Do these 37% represent the exception to the rule for intuitive thinking supposedly causing belief in God? I know, the alternate interpretation is that the "mindset" of intuitive thinking wasn't reinforced enough for them, right?
In light of this hypothesis that a "divine cognition" is actually CAUSING belief in God ("...a causal relationship between cognitive style and belief in God, a relationship for which we tested experimentally in Study 3," as researchers say on p. 3), the chart indicates participants who wrote about reflective reasoning leading to a positive outcome (predicted to be turned into doubting Thomases) were not quite moved in the direction of denying belief in God by vindication of careful reflective reasoning. They didn't exactly act to "support" the research hypothesis or act in an opposite way compared to the subjects primed to favor intuitive thinking if almost 45% still agreed they're convinced God exists. Again, the alternative interpretation is that the mindset of reflective thinking just wasn't reinforced enough for them, and no doubt experimental conditions could be created (if not through priming certainly through operant conditioning) in which experimenters ram so much logical thinking into subjects brains it causes them to report God does not exist. Historically, torture has been shown to be very effective for getting individuals to recant their beliefs.
However, it's questionable if the other group of interest--the participants who wrote about a reflective analytic type of strategy that led to a negative outcome--actually "vindicated intuition" because the process of writing about reflective reasoning is not necessarily the "opposite" of intuitive thinking (nor are the processes of intuitive thinking itself). Researchers have built this dichotomy of human cognition into all three phases of their study, but that doesn't mean subjects' cognition automatically conformed to this model in phase 3. I know it's hard to believe, but they just may have been thinking with the whole brain!
So then, how might we begin to explain why you find these results in Phase 3? Here's a quote from the instructions for participants assigned to write about an intuitive cognitive style (the so-called intuitive priming condition):
"Please write a paragraph (approximately 8–10 sentences) describing a time your intuition/first instinct led you in the right direction and resulted in a good outcome” (p. 4).
"First instinct," which isn't quite the same to some people as "intuition," might have led some participants to write about being impulsive. To others, intuition means doing something as if natural instead of methodical. Still, others may have interpreted the instructions as pertaining to an act of morality. If the whole point was to prime subjects with a "mindset" of intuitive or reflective thinking mode, you don't quite know what exactly you primed for in the intuitive condition. More importantly, as I say above, a writing task that involves verbal skills still relies on logical reasoning and doesn't necessarily coax a person into an intuitive mode of thought. Yet, researchers uncritically assume (because of a fabricated dichotomy of cognition): If we're not priming for intuition we must be priming for analytical reasoning. If participants in the intuitive-positive outcome group were primed to be impulsive, they may have more hastily agreed they believe in God. The same is likely if the priming influenced these participants to think "right direction" and "good outcome" was associated with moral behavior. Subjects in the intuitive-negative outcome group were not instructed to focus on something "good," and this helped create the contrast you see between the two intuitive conditions that also had an impact on the final analysis of all conditions (a crossover interaction). If the other set of participants-- the primed "reflective" type-- had no such bias in play, this may explain why the reflective-negative outcome numbers didn't add up to more reported belief in God, which created a contrast with the reflective-positive outcome condition, and had an impact on the type of interaction effect as well.
Different forms of bias have been known to plague researchers in ensuring accuracy from survey results, there's something known as "response bias" in which a person anticipates what researchers want them to say and say this instead of revealing their true beliefs. I can also tell you based on my own experiences both conducting research and from being a participant, in self reports and questionnaires people often try to think what is the "right" answer to a question. Even when there is no correct answer the answer that's "correct" is the one experimenters probably want me to provide. In Phase 3, subjects who wrote paragraphs about being a more "reflective" analytical type (with the good outcome) may have thought the "right" answer that experimenters wanted in reference to their faith was to say no to God. It's an example of response bias in the form of not wanting to admit they are believers (and it's suspect that nobody cares whether participants answered THIS question in a hasty "first instinct" manner). If subjects asked to write about intuition leading to a good outcome associated the instructions with being moral (read that quote again) their response bias may have been one in which because they were told to write a paragraph about moral behavior they thought the "correct" answer when asked about their faith was to say yes to God. You may believe I have no reason to suggest the above bias was working in favor of the hypothesis researchers were trying to support, but there's no check in place to see that the bias wasn't present, much like there's no check in place for being able to tell if what was being "primed" in this group was an intuitive cognitive style. In both cases, we simply don't know.
Before discussing the first phase of the Harvard report, I'd like to draw your attention to the first thing you see, which is the title of the study, "Divine Intuition: Cognitive Style Influences Belief in God." The authors could have created any title they wanted but instead decided to reveal a bias, which is in the form of putting under scrutiny only one side of the believer/non-believer dichotomy. Instead of calling the article something like, Atheist Reasoning: Cognitive Style Influences Disbelief in God, right in the actual title is the focus on how intuition--a notoriously faulty form of thinking to most scientists and logical thinkers--is supposedly linked specifically to believing there's a God. The strength of this claim regarding the internal validity of the study depends on how you operationalize "intuition, " and in phase one of the study what they call "intuition" is assumed to be one in the same as the thought pattern that leads to a mistake in logical reasoning.
News media and researcher distortion: Phase 1
Phase 1 of this research was a survey, not an experiment. In an online format, subjects were given a Cognitive Reflection Test (CRT) in which they were asked three questions intentionally designed to mislead people into making an error that required logical thinking to solve (math problems). Afterwards they were asked questions including if they've had an experience convincing them that God exists (as in Phase 3). Researchers then correlated the number of people who reported they believe there's a God with the number of questions they got wrong, i.e., defined as the answer that researchers assumed could only mean they answered the question in an intuitive manner. Again, the question is who is an intuitive thinker, who is a "reflective" analytical thinker, and who is more likely to say they believe in God. Questions were of this type:
Bill and Sally’s age totals 28 years. Bill is 20 years older than Sally. How old is Sally?
If you answered the question with 8, the researchers didn't just assume you made a mistake. You were placed into a category that assumes how you approach problems favors intuition over logical reasoning. To answer the question correctly required this form of logic: Sally's age is X. So, 28 = (X + 20) + X. When you solve for X you eventually get 2(X)= 8, and the answer is 4. Indeed, 24 plus 4 is 28. The researchers already knew that many people were likely to gloss over how the math problem says Bill is 20 years "older" than Sally, but what they don't say is that this ALSO leads to a form of logical thinking (20 + 8 = 28) but it gets you the wrong answer. What's questionable is that researchers insist this error in logic is the same thing as the indicator that a person has relied only on intuitive thinking.
There's something else I'd like to question:
"A bat and a ball cost $1.10 in total. The bat costs $1 more than the ball. How much does the ball cost?"
In reporting this research, Yahoo News (under the outrageous title: Belief In God Boils Down To a Gut Feeling) explains, "The intuitive answer to that question is 10 cents, since most people's first impulse is to knock $1 off the total. But people who use "reflective" reasoning to question their first impulse are more likely to get the correct answer: 5 cents."
How could anyone possibly know from the design of this study that those who got the correct answer are among those who "question their first impulse?" Researchers have created a dichotomy of cognition, and the purpose of the questions is to separate subjects into those who used one form of thinking (intuitive)or another (reflective). However, "reflective" in this case actually is the skill to use logical reasoning to solve the math problems and not necessarily dig deeper and think harder about the questions, as the chosen terminology suggests. When it comes to using logical reasoning or intuition in matters more diverse and complex than a math problem (such as belief in God) most people probably apply both. Still, the researchers claim:
"By intuitive judgments we mean judgments made with little effort based on automatic processes, and by reflective judgments we mean judgments in which the judge pauses to critically examine the dictates of her intuition(s), thus allowing for the possibility of a less-intuitive or counter-intuitive conclusion. Reflection is typically assumed to be more effortful than intuition..." (p.1)
Do you see how this explanation contains just a little hidden suggestion that people who say they believe in God will be predicted to also be people whose judgements are "made with little effort," as if sloppy thinkers in general? Gee, I guess it's not the researchers' fault. They only designed the experiment and it's "fair" use of words appraising the people predicted to be less likely to believe there's a God as making "reflective judgements...in which the judge pauses to critically examine the dictates of her intuition." All this from answering a math test correctly?
The questions in the CRT were chosen because they typically mislead some people. Mathematically, in order for the bat and the ball to be $1.10 total the bat needs to be $1.05 and the ball needs to be $.05. However, the so-called "intuitive" thinkers may read the original question like this: If a bat and a ball both cost $1.10 in total, and the bat costs $1, how much did the ball cost? They are more likely to leave off the $1 "more" part of the information. But if you're the "reflective" (smarter) thinker you get the right answer because you weren't fooled by the question's misleading format. Once the data comes in, we take all the people who were fooled by the question and call them "intuitive" thinkers, call all the ones who got the correct answer "reflective" thinkers, and then when we correlate the results with reported belief in God, the people more likely to have gotten the wrong answer are the ones who agreed they believe in God. You may notice something a bit dubious here.
How do we know if any answer was the kind of quick hasty one that "intuition" apparently always manifests itself as? How come you can't say there were people who got the question wrong because they're poor readers? Why not careless? Uninterested? People who value working as quickly as possible (an important cultural value)? Why aren't they people who have trouble with math problems in general? Might they be easily intimidated? Phobic about math or statistics? No. None of these can apply because all subjects must be made to fit neatly into only one of two discrete categories: intuitive or reflective.
Why not call the "reflective" thinkers "mathematical" or "logical" or "analytical" thinkers? In contrast to "reflective," these other labels don't hint to an American audience that there might just be some advantage to being the type less likely to believe in God. These labels suggest a cold-hearted scientific type. This research is put out into the news media as supposedly results of an "objective" science but a much more objective science would be labeling the two groups Group A and Group B, which would allow the reader to decide what further labels they'd like to assign to each group given their performance on a math problem. The researchers can't go back and change those labels because they know that would weaken their (soon to become popular) underlying hypothesis, which is that the more "intuitive" your thinking style is, in contrast to a more "reflective" and logical form of reasoning, the more you will believe there's a God. The choice of labels is the confirming (more like condemning) evidence provided through the design of the research itself.
Let's consider the results from Phase 1. According to the chart (Figure 2.) with the focus on people who all agreed they've had an experience that's convinced them God exists, when you compare a) the number of people who had zero intuitive type answers (assumed to be the least intuitive thinkers) with b) the group in which all three questions were answered with intuitive responses (assumed to be the most intuitive thinkers), this number rises from about 38% to not higher than about 58%. That's only a 20% difference. Plus, in the group that indicated no intuitive thinking (assumed to be the most reflective thinkers), 38% still said they've had an experience that's convinced them God exists. In addition, as much as 42% of the people who said they don't believe in God still relied on intuitive thinking to answer the questions (i.e., 100% - 58% in the group with 3 intuitive responses)! So, even though the data allowed researchers to obtain statistically significant correlations between what they call intuitive cognition and belief in God, there were actually some important nonuniformities. I realize this is a moot point to both statisticians and the people who want to believe that as long as the correlation is significant there's no reason to consider inconsistencies in the results. I like to remind people it's not the data it's how you interpret it that often matters the most. You can't help but wonder why are these researchers skewing results from a correlational design in a causative direction when they say, "The effect was also robust" (p. 2) after they statistically controlled for other factors (age, income, conservatism, etc.). No matter how you fudge it, the design of Phase 1 does not allow for anyone to infer that people who use intuition to solve math problems also use intuition when it comes to belief in God and thus intuition causes belief in God.
In Summary
1. In general, researchers have constructed a dichotomy of cognition, intuition vs. logical reasoning, and in a biased manner suggest both levels of cognition apply equally to a) how to solve a math problem and b) how to respond to being asked whether or not God exists. They assume one or the other cognitive style is not merely linked to believing God exists but that it is the cause. However, intuition is not necessarily a "first instinct" approach when it comes to answering questions about God's existence, but it may make it more likely you incorrectly solve a math problem. Logical thinking might get you the correct answer on a math problem but have little to do with trying to understand if God exists.
2. In general, bias is present in the language and charts used throughout the research as well as in the title of the article itself to scrutinize how intuitive thinking is the cause of believing in God rather than presenting the research in terms of either how analytical thinking is the cause for a reluctance to believe God exists or in more neutral terms that don't emphasize a believer/nonbeliever dichotomy. This is employed to skew the reader's judgment in favor of the researchers' hypothesis regarding causality rather than providing a more objective presentation of the findings.
3. In Phase 3, researchers only found statistically significant change in the dependent variable (reported belief in God) through interaction between the two independent variables (intuitive or reflective cognitive style vs. positive or negative outcomes), which was possible only because the variable of "outcome" was built into the experimental design. In support of the hypothesis that intuitive cognition causes belief in God (not merely reported belief), this type of causal evidence is less convincing than finding a significant main effect (for intuitive vs. reflective priming) in addition to an interaction. Yet, they confidently call their finding "support" of the hypothesis that intuitive thinking is not just linked in some way to reported belief in God; it actually causes more belief in God, and they say this based on one way to manipulate reported belief in an experiment that may have little to do with the real world.
4. Researchers equally reject the null hypothesis, "Priming intuitive thinking does not cause more (reported) belief in God," and the more precisely stated null hypothesis that should have pertained to the experiment in Phase 3, "Priming intutive thinking does not cause more (reported) belief in God if the outcome in the written paragraph used to prime intuition is positive." Having found a statistically significant interaction allows for rejection of the latter null but not the first. The wording of the first null hypothesis is too vague and overgeneralized to apply to this experiment.
5. Questionable instructions. In Phase 3, there's no way to check if researchers are specifically priming intuition when participants are asked to write about a time that intuition/ first instincts led them in a "right direction" with a "good outcome." Participants may have interpreted this in terms of morality, which may have led to response bias in the form of thinking when experimenters asked to say yes or no to the existence of God it was a matter of morality, and the "correct" answer was to say yes. Similarly, the association of the words "careful reasoning" in the reflective-positive outcome condition before being asked whether or not you believe in God may cause response bias in the form of being expected to hesitate, in which case the better answer is no. There's no way to know if the "effect" found in Phase 3 was the result of response bias.
6. In Phase 3, there's no way to know if experimenters were priming a) intuitive cognitive style, b) high appraisal of intuition, or c) neither of the two because a writing task still requires both verbal intellect and logical reasoning. They assume "a" with no check in place.
7. An ambiguous question. When someone agrees with the question, "I have had an experience that has convinced me God exists," does this indicate someone who a) has had the experience but no longer believes there's a God, b) has had the experience and still believes there's a God, or c) has had the experience but isn't quite sure whether there's a God? For the sake of gathering only the evidence that supports their hypothesis (intuition causes belief in God), researchers assume a "yes" to the question only indicates "b." Rather than use the clearly stated "I believe God exists" format, an ambiguously stated question provided them with ambiguous answers. Hence, they don't know what they are really measuring.
8. Confounding reported belief in God with actual belief in God. (It's interesting how I wasn't going to address this one because I thought it was so obvious. But then I started to read what other sources were saying about the research article and, once again, I can see it has to be made explicit). There is a world of difference between saying intuitive thinking causes reported belief in God and saying intuitive thinking causes belief in God. The dependent variable in Phase 3 (and in all phases) is a self reported belief in God. It is unknown who actually believes in God but said no, and who doesn't believe in God but said yes. Hence, we have empirical observations that pertain to a measure based on a self report at best assumed to be an accurate indicator of not only underlying belief, but enduring belief despite the possibility that priming may have only led to temporarily altering self reported belief in God. Despite this fact, what is the title of the journal article? Is it "Divine Intuition: Cognitive Style Influences Reported Belief in God" OR did researchers somehow manage to get away with the title, "Divine Intuition: Cognitive Style Influences Belief in God?"
9. Phase 1 of the study was a survey, not an experiment. Researchers found a significant correlation between getting an incorrect ("intuitive") answer on a math problem and reporting more belief in God, but there can be any number of third factors for the actual cause of this relationship. No matter how strong the correlation, the design of Phase 1 does not allow for anyone to infer that people who use intuition to solve math problems also use intuition when it comes to belief in God and thus intuition causes belief in God. Yet, the survey was used in this manner in support of the hypothesis that intuitive thinking is responsible for the belief in God (this includes in the abstract of the original report, which will be most widely read).
10. Overconfidently reporting the results of both Phase 1 and 3 of the study encouraged the news media's distortion of the actual findings.
"Instead of calling the article something like, Atheist Reasoning: Cognitive Style Influences Disbelief in God, right in the actual title is the focus on how intuition--a notoriously faulty form of thinking to most scientists and logical thinkers--is supposedly linked specifically to believing there's a God."
The New "Science" of Faith?
I'm not certain about the research at hand, but I wouldn't be surprised if the field of cognitive psychology as applied to the study of faith is attracting atheists. It's already been said in different ways that science has been hijacked by an atheist's agenda. Written all over this study is the assertion that belief in God is something that is associated with a rash, illogical, and erroneous form of thinking. It reminds me of the early attempts to reify intelligence in accordance with IQ tests, which involved very questionable interpretations of research if not downright dishonesty and fabrication of "facts" (see Stephen Jay Gould's book The Mismeasure of Man).
The authors are dangerously close to compartmentalizing faith into the confines of the single cognitive process of intuitive thinking (and according to a "first instinct" definition of intuition). Given their two categorical labels, researchers are also assuming there's only two modes of thought: the quick and sloppy thinking of the intuitive type and the deeper more reflective type associated with getting the right answer to questions that involve logical reasoning. The cognitive processes associated with spirituality do not necessarily fall exclusively in either of the two (i.e., intuition or logical reasoning). Depending on the situation, an individual's conceptualization of spirituality can be a matter that is affected by accepting or rejecting cultural values because of the pressures of social integration. There are people who reject religious beliefs and confound them with belief in God. There are people who have unrealistic expectations about things like prayer. There are people who place themselves in circumstances in which logical thinking becomes paramount, and they get good at math because of what the environment demands no matter if they believe in God or not. Unlike the very determined perspective the study in question presents (i.e., intuitive thinking causes belief in God), whether or not a person leans toward relying on "intuition" or not in their belief in God is in a dynamic relationship with many other factors.
How could Phase 1 of the study have possibly been biased if they gave everyone the chance to get the "correct" answer? To get the correct answer was the basis on which groups were divided. In Yahoo News, much was made about the statistical analysis in which intuitive thinkers are 1.5 times more likely to say they believe in God compared to the "reflective" thinkers who answered the math problems correctly. The way Yahoo News sees it, if roughly 40% of the participants (actually closer to 38% as mentioned above) who got all three math problems correct ("reflectives") said they believe there's a God and this rises to roughly 60% for the participants who got no answers correct ("intuitives") it is indeed 1.5 times more. However, it's misleading in the direction that intuitive thinking is responsible for belief in God. We can also report from the same percentages by saying: Reflective thinkers were one-and-one half times more reluctant to believe in God. This does not mean all reflective thinkers said they do not believe in God. It means for every 10 people categorized as reflective thinkers who believe in God, there are 15 categorized as intuitive thinkers. Yahoo News took this article's information and proclaimed "Belief In God Boils Down To a Gut Feeling" when the results obtained from the online survey completed by 882 adults can't be generalized to the public and don't pertain to a cause and effect relationship (i.e., a "gut feeling" doesn't cause belief in God).
It's almost hypocritical for a researcher to use the labels and the wording you find in this study and then claim (likely for the sake of public relations), " 'Intuitions are important and reflection is important, and you want some balance of the two. Where you are on that spectrum affects how you come out in terms of belief in God ' " (from the article "Try God" in the Harvard Gazette http://news.harvard.edu/gazette/story/2011/09/intuitive-try-god/ ). They do not have evidence that indicates when you are on the extreme intuitive end of their fabricated dichotomous spectrum you are likely to believe in God! Can't someone have "balance" between being intuitive and logical and still believe firmly that there's a God? Intuitions are "important" but not when it comes to solving a math problem! We don't need to do an experiment to know that logical reasoning can explain why many people deny there's a God. The "effect" researchers are really talking about pertains to the old hypothesis many atheists still hold, which is that people not smart enough when it comes to logical reasoning are also the people not smart enough to reject the notion that there is a God.
If there's one word I can't stand in American lexicon it's the use of the word "debunk." I admit I have highlighted exaggerated claims in the study above, but I am not trying to "debunk" the current research. I'm trying to set realistic limits in interpreting the research rather than run wild with overgeneralizations about a facet of human behavior. Being a critical thinker requires much more than reading and then summarizing what was written onto the page of a scientific research article like an intelligent parrot. We live at a time in which there is a general reluctance to remain aware of what limits there are to understanding ourselves through a scientific method while there's a general focus exclusively on what we believe are the benefits of this approach. It's a practice that doesn't simply use science as a tool. We construct a climate of scientism, and this leads to the abuse of scientific methods. There's more need now than ever before to be skeptical of research rather than accepting of it at the surface level, and this does not constitute the act of being close-minded.
I'm not saying faith is so mysterious that science can never understand anything about it. I'm saying science does little more for understanding faith or belief in God beyond an approach that tries to turn the world into one that is 100% logical. Scientism is the notion that all we can know about ourselves and the world must come from scientific methods and an empirical investigation, and this has become a cultural phenomenon pervasive in the USA. Whatever underlying atheistic agenda a psychology of faith may or may not have in uncovering the thinking processes of who is and who is not a true believer, you can bet as long as it's stamped with the scientific seal of approval the public is likely to be misled into believing it's God's honest truth.
Another version of the review above can be found here: http://biodeterminism.blogspot.com/2011/09/critical-meditations-on-study-divine.html
© Peter Kurtiak, 2011. All rights reserved.
