Wednesday, August 29, 2012

Results Guaranteed - Free Recast.


 
In the 2012 Fall Sellers update eBay banned a list of items from being traded on their site. This included home businesses and recipes. It also included Psychic and Tarot readings, Spells and Potions. EBay gave the following reason for the prohibitions ; “Transactions in these categories often result in issues between the buyer and seller that are difficult to resolve.

Now having twice been ripped off through eBay (buying a tent and phone rather than spells) I definitely don’t want to endorse this company or accidentally advertise them. What I want to do is explore how eBay’s ultimately practical rather than philosophical decision has anything to do with hot topics like “freedom of religion” or “the division of science and religion”.

When I learnt about eBay’s decision (from this site) I recalled wandering into a new age fair a few years ago. At that fair people sold a range of items which made different claims. Some were very specific. One item was a crystal which when attached to your phone reduced the electromagnetic radiation from said phone.  

This is a fairly clear example of a “scientific” claim because it is what scientists and philosophers refer to as “falsifiable”. I can measure the electromagnetic radiation output before attaching a crystal to my phone and after. It is possible, if certain measurements are obtained, that I can say that the crystal does not reduce the electromagnetic radiation (or if it does, it does so as much as any stray pebble stuck to a phone would). That capacity to evidentially disclaim a statement or “falsification” is one of the central principles of science.

By contrast a claim made at another stall was that their particular brand of very expensive colour therapy would enhance my mood. That’s a much harder to falsify claim. No matter how depressed I might be despite using colour therapy, my therapist could argue that I would’ve been still more depressed without their therapy. It’s fairly hard to prove otherwise. One way to do that is to conduct large scale trials of people using the “proper” colour therapy and compare them with people using a placebo version of colour therapy (just random colours which are said to be the real deal) while maintaining a third control group who receive no therapy at all. Certain measurements of happiness, well-being and health (such as higher scores in the control or placebo groups) would be able to at least weaken the claim that this colour therapy enhances mood. 

While it is also true for the phone crystal it’s easier to see with the colour therapy that we could encounter disagreements about measurement which confound any proof of efficacy. What exactly is an enhanced mood? Is self-reporting an adequate way to find out about it? As I argued in the post Measure Me, measurement of some kind is crucial to science but also to any meaningful critical conversation. Measurement is how we evaluate. As a consumer on eBay or at a new age fair then taking responsibility for your spending means not allowing the seller to define how results are measured. It’s your money and you are evaluating the product so you get to decide what an enhanced mood means to you. If the seller tries to avoid any definition, preferring to keep it so vague it can’t be measured, (or if they claim that only they have the special tool or skill to measure the effect of their therapy or product) then I would quickly walk away.

What also makes the colour therapy’s claims hard to falsify is the level of complexity involved in its method. Treatments and interventions which are based on very solid science still have wildly divergent effectiveness between controlled and real life settings. In controlled trials the birth control pill is effective at reducing pregnancy. In the real world it has less effectiveness among some populations (ie. people with poor memory or chaotic lives) because regularly remembering to take any pill is harder for them. What this means is that if a person reports a negative outcome from using a complex therapy, their deviation from the instructions can be to blame for the failure rather than the therapy (as performed perfectly) being to blame itself.

This problem with complexity can be exploited if I want to make a therapy immune to being properly tested. If a therapy is so complex that there is always something to go wrong then any negative results can be dismissed. It’s a way of hiding bad data, which renders your claim unfalsifiable while it still sounds falsifiable (and thus still scientifically credible). We’re usually wise to this however. If a therapy sounds too complex to complete correctly we guess we will fail it and so don’t buy it.

The real trick is to add complexity that still sounds achievable. The easiest way to do this is to introduce an immeasurable user component as an essential part of the therapy. The classic choice is faith (or “winners attitude”) in the outcome. Faith is impossible to self-measure because you can’t ever view your own from outside it. You have to inhabit faith for there to be any faith. Yet you have to drop faith in order to ever say if the therapy has succeeded or failed. That creates a catch 22 which ensures the therapy only fails when the user fails it by withdrawing their faith to say “hey this is not working”. Everyone who succeeds proves the therapy. Everyone who doesn’t has no impact on the results because they eventually lost faith and thus didn’t complete the treatment.

Not all claims make any attempt to be falsifiable. The claim that a lamp will beautify your home is not falsifiable. There is no universal measurement of beauty that would allow me to disprove that statement. Or rather I can disprove it easily for myself just by looking at the lamp but as it’s a matter of taste I can’t generalize from my experience to you. That’s why ebay doesn’t have any problem with a lamp that’s as ugly as sin being sold as attractive. It’s spin and we all ought to know it’s spin and check the lamp for ourselves. Mind you, if the lamp is not pictured then a buyer may assume that there is a generally accepted standard of attractive being referred to. The buyer may assume that saying the lamp is attractive is actually a falsifiable claim. A few philosophers might agree with them by arguing that beauty is objective rather than opinion, but I wouldn’t. I'd say, “Pics please”.

What eBay has done by prohibiting metaphysical products such as spells and potions, is to remove an area where we generally hear claims as falsifiable but they regularly aren’t. A potion or spell tends to say things like “you will attract money into your life” or “you will cause someone to fall in love with you”. Some of their claims are harder to falsify such as “your sex life will improve” or “increases courage” and some are impossible such as “cleansing an aura” but even there spells and potions sound like they are promising specific benefits with measurable outcomes. However all the tricks of obscured measurement, complexity and reliance on faith are in employ to create non-falsifiable claims masquerading as falsifiable ones. This is what is called Pseudo-science.

A very interesting question is how much of all religion and theology is a similar pseudo-science itself. We can easily put certain faith healing and holy amulet purchasing into that category. They make clearly falsifiable sounding claims that work just like spells. However does theology come with the promise that studying it will bring you closer to God? Is that also an ultimately untestable claim (who measures it) that still sounds falsifiable? Does prayer come with the guarantee of results but only if your faith is strong? Is prayer then ultimately a form of pseudo science too? And if we can ask these questions of theology and prayer what about philosophy? Does it come with a promised outcome? Of what though; Wisdom?, Truth?

Against the position that religion and philosophy should be condemned as pseudo-science is the criticism that science shouldn’t be allowed to be such a totalizing paradigm that everything is subjected to its way of thinking. Science is also increasingly wedded to a consumer attitude to the world as it serves to evaluate products for our purchase. The world view of a scientific consumer is a cultural phenomenon and we should be asking ourselves whether we want everything to be evaluated within it. Perhaps a lot is lost if we do.

To me the resolution to this argument lies in identifying what territory you are attempting to occupy with your claims. When spells and prayers and readings and the like put themselves up on eBay they are very clearly entering into a commercial marketplace. Marketplaces are not wild territories without rules but have standards of communication and definition. Consumers have socially constructed rights. If you expect to advance your theology or philosophy in a commercial marketplace then you can’t get all huffy about such rights infringing on your religious expression. Those marketplace rules have to apply to everyone precisely in order to define a marketplace. You’re happy after all that they apply to your own purchases of electrical goods for example.

Similarly science is a space with socially constructed rights and rules. Opinion is not evidence. People have the right to know your methods and repeat your experiments. Definitions of measurement must be published with results. Beliefs must ultimately be able to be overturned based on results. If religion, theology and philosophy don’t want to play by these rules then that is perfectly legitimate. There is argument for validity outside of the culture of science just as all life shouldn’t be contained by commercial culture. Maybe your religion ought to be non-scientific.

However you can’t make use of the cultural weight of science for religious claims and assert a special religious exemption from its rules. To do so would negate what science is; where all claims must follow the same rules. If you claim to have proof of a magical outcome or make clearly falsifiable sounding claims then it is dishonest to object to the standards of proof and falsifiability that give those terms meaning. Ultimately you ought to admit that you are not scientific if you are not going to be. If you don’t, then it’s incumbent on those following the rules to ask you to leave. This is all eBay did in regard to metaphysical products like spells.


1 comment:

  1. Generally i like it. Nice work.
    Couple of points (as always).
    Your example of the blinded colour test still would not be valid. You need a double blinded study. This is where both the participants and the experimenters are both ignorant of the placebo group. This removes any bias from creeping in, ie if i know i'm doing the fake stuff, you could probably pick up clues from me and bias the result.

    ReplyDelete