Based on the book The Demon Haunted World by Carl Sagan
The following are suggested as tools for testing arguments and detecting fallacious or fraudulent arguments:
Wherever possible there must be independent confirmation of the facts
Encourage substantive debate on the evidence by knowledgeable proponents of all points of view.
Arguments from authority carry little weight (in science there are no "authorities").
Spin more than one hypothesis - don't simply run with the first idea that caught your fancy.
Try not to get overly attached to a hypothesis just because it's yours.
Quantify, wherever possible.
If there is a chain of argument every link in the chain must work.
"Occam's razor" - if there are two hypothesis that explain the data equally well choose the simpler.
Ask whether the hypothesis can, at least in principle, be falsified (shown to be false by some unambiguous test). In other words, it is testable? Can others duplicate the experiment and get the same result?
Additional issues are
Conduct control experiments - especially "double blind" experiments where the person taking measurements is not aware of the test and control subjects.
Check for confounding factors - separate the variables.
Common fallacies of logic and rhetoric
Ad hominem - attacking the arguer and not the argument.
Argument from "authority".
Argument from adverse consequences (putting pressure on the decision maker by pointing out dire consequences of an "unfavourable" decision).
Appeal to ignorance (absence of evidence is not evidence of absence).
Special pleading (typically referring to god's will).
Begging the question (assuming an answer in the way the question is phrased).
Observational selection (counting the hits and forgetting the misses).
Statistics of small numbers (such as drawing conclusions from inadequate sample sizes).
Misunderstanding the nature of statistics (President Eisenhower expressing astonishment and alarm on discovering that fully half of all Americans have below average intelligence!)
Inconsistency (e.g. military expenditures based on worst case scenarios but scientific projections on environmental dangers thriftily ignored because they are not "proved").
Non sequitur - "it does not follow" - the logic falls down.
Post hoc, ergo propter hoc - "it happened after so it was caused by" - confusion of cause and effect.
Meaningless question ("what happens when an irresistible force meets an immovable object?).
Excluded middle - considering only the two extremes in a range of possibilities (making the "other side" look worse than it really is).
Short-term v. long-term - a subset of excluded middle ("why pursue fundamental science when we have so huge a budget deficit?").
Slippery slope - a subset of excluded middle - unwarranted extrapolation of the effects (give an inch and they will take a mile).
Confusion of correlation and causation.
Straw man - caricaturing (or stereotyping) a position to make it easier to attack..
Suppressed evidence or half-truths.
Weasel words - for example, use of euphemisms for war such as "police action" to get around limitations on Presidential powers. "An important art of politicians is to find new names for institutions which under old names have become odious to the public"
Above all - read the book!
Further resources:
The Critical Thinking Community
CSICOP/Skeptical Inquirer
Australian Skeptics
Quackwatch
Carl Sagan Productions Ann Druyan's comment on this web page:
"I have no problems whatsoever with your efforts to spread the word on critical thinking. It was Carl's dream and mine that each and everyone of us would have that baloney detection kit inside our heads. I salute your efforts in this direction.
With best wishes,
Ann Druyan"
12 Nov 2000 Project Voyager: OneCosmos represents the realization of a shared lifelong hope to organize and lead a team that will deliver the vision of Cosmos through every available screen: spectacular living Internet, engaging television and cinematic works of art.
Examining the role of think tanks by Sharon Beder, Engineers Australia, November 1999.
Innumeracy.com.
Faith-Based Reasoning - Scientific American June 2001. In one case [global warming], the president invokes uncertainty; in the other [missile defence], he ignores it. In both, he has come down against the scientific consensus.
InConcept.
Scientific American, Nov 01: Baloney Detection: How to draw boundaries between science and pseudoscience, Part I. Part II.(updated URLs)
Scientific American May 02: The Exquisite Balance - It seems to me what is called for is an exquisite balance between two conflicting needs: the most skeptical scrutiny of all hypotheses that are served up to us and at the same time a great openness to new ideas.... If you are only skeptical, then no new ideas make it through to you.... On the other hand, if you are open to the point of gullibility and have not an ounce of skeptical sense in you, then you cannot distinguish the useful ideas from the worthless ones - Carl Sagan, 1987.
Skeptic.com
The Skeptic's Dictionary by Robert T. Carroll.
Cable Science Network - new TV service dedicated to science.
21 Feb 06 New Scientist: Algorithm detects Canadian politicians' spin - Spin, in this case, is defined as “text or speech where the apparent meaning is not the true belief of the person saying or writing it”...
9 Mar 06 National Geographic: Was Darwin Wrong?
11 May 05 Karl Kruszelnicki: Mysterious Killer Chemical - We live under the illusion that we understand the world around us...Dihydrogen Monoxide FAQ - a common household compound can be hazardous
27 May 06 SciAm: Up the Lazy Creek - "motivational deficiency disorder"...numerous news outlets picked up the BMJ press release and ran it without a hint of skepticism. That's just motivationally deficient journalism. BMJ abstract: Scientists find new disease: motivational deficiency disorder + People are easily duped about new diseases, conference is told.
27 Jun 06: By coincidence, I received the 24 June copy of New Scientist and July copy of Scientific American on the same day. In New Scientist Richard Koch and Chris Smith ask "why is science under attack like never before?" (subs!). They suggest that the rest of society is now much more critical of science, which has revealed a darker side such as atomic weapons and "poisoning of the planet". There is another, more likely reason for the demise of science that is revealed in the article "The Political Brain" by Michael Shermer in Scientific American. He describes MRI studies of the brain that have revealed how the brain suppresses the rational, reasoning portion of the brain in favour of emotions that reinforce confirmation bias - "whereby we seek and find confirmatory evidence in support of already existing beliefs and ignore or reinterpret disconfirmatory evidence".
Science, of course, is built on that rational, reasoning function of the brain and is founded on skepticism. This does not bode well for politicians, religious fanatics or marketers of consumers products who utilise emotional responses to ply their trade. Is it any wonder that that science and skepticism are discriminated against when these same groups now have a huge influence on the media?
6 Oct 06 New Scientist (subs): Mind fiction: Why your brain tells tall tales (see 27 June item above)
Ig Nobel prizes for 2006!
9 May 07: Perpetual Motion and the Big Wither.
25 Feb 08 New Scientist ($): Interview: The man who would prove all studies wrong - "People aren't willing to abandon their hypothesis. If you spend 20 years on a specific line of thought and suddenly your universe collapses, it is very difficult to change jobs." + Comment: Why peer review thwarts innovation
25 Feb 08 SciAm: Adam's Maxim and Spinoza's Conjecture - "...we should reward skepticism and disbelief and champion those willing to change their mind in the teeth of new evidence. "
Jun 08 Kids.net.au: Scientific method.
Anti-Quackery Ring
[ Join Now | Ring Hub | Random | << Prev | Next >> ]
Less serious sites:
Journal of Irreproducible Results
The Annals of Improbable Research. (with the Ignobel Awards)
Dihydrogen Monoxide FAQ - a common household compound can be hazardous ;<)
Prepared by Michael Paine for The Planetary Society Australian Volunteer Coordinators.