Onus Probandi, Assertionism and Peer-review.

Good morning, ladles and jellyspoons.

In the wake of the hyperbolic, hysterical travesty that was the Boaty McBoatface saga, I have, after many years of resistance, succumbed to the double-edged Damoclean temptation of a sojourn into the Twitterverse. I've avoided it precisely because I have the worst case of SIWOTI syndrome in the recorded history of the universe since the invention of Tim Berners-Lee, especially when the WOTI is, in my esteem, harmful.

It should come as no surprise to learn that I've managed to find some WOTI in fairly short order, in this case in the form of a column on the website of the Daily Telegraph and a resulting series of tweets and other interactions.

I had wanted to use this post to continue on pre-Planck cosmology, but thought it worth picking this apart a little.

The article itself is this one, headlined Forget my son - I'm the one in exam stress hell, penned by Allison Pearson, known to some as the author of I Don't Know How She Does It.

Much of the article is unobjectionable. Liberally sprinkled with apposite anecdotes (see below for caveat regarding this), and a tone of exasperated but good-natured humour, with some reasonably decent advice for parents enduring the slings and arrows of teens going through one of the most stressful times in their young lives. Then, however, comes the following:

When they made the exams easier to pass they also made perfection attainable. Diligence is all. Fine if you’re a swotty, well-organised girl, not so good for testosterone-plagued slugabeds.

Now, the diligent reader will spot the issues with this without direction. Setting aside the notion of the attainability of perfection, which has some logical problems all its own, the real issue lies in the latter portion. I'll come back to that shortly, but here's the opening salvo:



A couple of assertions in there, not least the one saying 'all the evidence suggests...'

There's an extremely common fallacy committed in public discourse known in rigorous circles as onus probandi, closely related to the fallacy of bare assertion, both of which are discussed in a previous post. Onus probandi is the shifting of the burden of proof. In essence, this fallacy is committed when, rather than supporting one's own claim, the claimant insists that the skeptic prove it false.

It's up to the person erecting a claim to support that claim. Failure to do so commits one of the two above fallacies, dependent on the circumstances. The reason for this should be fairly obvious, namely that skepticism is the rebuttable position for any given claim.

In the first assertion above by Ms Pearson, no good reason has been given to accept the claim 'that boys lag behind girls in organisation skills and exam results'. This commits the fallacy of bare assertion. It's easy to see how this is related to the onus probandi, because the fallacy of bare assertion is essentially onus probandi by proxy.

Dr Weinstein then raises the motivating point for her initial interjection, and it's a good one. She asks Ms Pearson whether she thought the veracity of this claim (not contested for the moment) was sufficient reason to promulgate stereotyping in a national newspaper.

Stereotyping is another common fallacy (hasty/sweeping generalisation), but this one actually has measurable negative consequences in the real world, and the more people who buy into a given stereotype, the more destructive it is.

In this instance, she's voicing a commonly-held perception that girls are generally better organised and less indolent than boys, and that this is reflected in their exam results. Whether this assertion is actually true is almost beside the point for the purposes of Dr Weinstein's point, which is that perpetuating such a stereotype can be harmful and, worse, stereotypes are self-perpetuating^{[1] (this is how it's done, incidentally: Ed)}.

Ms Pearson defends her article, saying that 'it isn't stereotyping', which reveals another problem with stereotyping, which is that we often don't even realise we're doing it!

Of course, the scientifically literate among us realise that this is just another manifestation of our exceptional pattern-seeking software, which has had deep consequences to our evolutionary history, not least that it's lasted as long as it has, but I digress.

She goes on to assert 'it's science'. Here's the fallacy again, but this time with a smuggled in argumentum ad verecundiam (appeal to authority).

Moving on to the follow-up:




This is a quite beautiful fallacy committed by Ms Pearson, the argumentum ad populum in all its glory, along with a glorious feedback loop of 'reasoning' whose radius is related to its circumference by a multiple of 3.14159 (excuse the tautology, it was for emphasis).

It's true because mothers loved it, and mothers loved it because it's true. Can you say 'circular'? I know you can!

 

Of course, at this point the inner gobshite took over, and I interjected:

 

SIWOTI! That's my excuse and I'm sticking to it.

 

Anyhoo, there was another exchange, this time on another platform:


 And this is what motivated me to write this post. Ineedmoresleep needs, I think, a little more thought. Perhaps too much sleep (or too recent; we can all be groggy) has undermined this person's ability to reason temporarily but, given that this is such a common fallacy, quite probably not, which is a worry. First denying that there's a stereotype, then defending that stereotype as accurate! Which is it? 

Now, it's possible that this is a semantic issue and, always willing to apply Hanlon's Razor, I must entertain the notion. Stereotyping is having a set idea about what something or someone is like. For fans of the argumentum ad lexicum, most dictionaries will add a clause to the effect that this is especially with ideas that are wrong, but this isn't actually necessary for stereotyping to occur. It only requires that it be generalised.

Some stereotyping is useful. We all do it. The fact that I stereotype cars as dangerous means that I look both ways before I cross the road. Stereotyping certain patterns as dangerous in aeons past allowed our ancestors to recognise the pattern of a tiger in the grass closely enough to be able to escape it, thereby avoiding a terminal destiny as an entrée (this is an example of a stereotype that will often be wrong, yet still confers a survival advantage).

That said, reaching a conclusion about something based on sweeping assumptions is always deductively fallacious. In an earlier post, I dealt with how different types of reasoning are employed in science. See that for more on this. Stereotyping of the sort that gives you a head start on the tiger is abductive and, in that sense, it's fine (you're far less likely to come a cropper if you respond to the warning sign than if you don't). However, when you use such sweeping premises in deductive arguments, you're doing it wrong.

More important, and the source of Dr Weinstein's chagrin, is that young people going through their exams have a tough enough time of it, and really don't need this additional pressure (I wonder how Ms Pearson's son feels about being described as 'indolent' or a 'testosterone-plagued slugabed', not to mention the fact that she's comparing her suffering to his in all this).

Before moving on, I just want to add  something from a head teacher on this:

 

Let's briefly look again at the interaction with Dr Weinstein:


This is the worst form of onus probandi. This doesn't merely shift the burden of proof by suggesting that the skeptic prove the arguer wrong, it actually asks the skeptic to demonstrate the veracity of the arguer's argument.

Even Sagan, in his brilliant exposé on the failure of critical thinking in society The Demon-Haunted World: Science as a Candle in the Dark didn't foresee this. I have an invisible dragon in my garage, now prove I'm right...

What?!! 
 
One thing I didn't cover in the previous post was what follows from the eight steps of logic of the scientific method: Peer-review.

When you've worked really hard and covered every bit of ground you can think of in trying to break your hypothesis, and you're satisfied that you've thought of every possible means to falsify it, it's time to publish. For the scientist, this is a fraught time. 

You go over your findings, make sure you've got every equation in place, all nicely LaTeXed up. You make sure all your graphs and charts are an accurate representation of the data, and that the error bars are included. You go over every bit of mathematics and make sure that the logic is flawless, and that all your assumptions are properly stated, with supporting citations from the primary literature so that those assumptions can be cross-referenced for rigour. You make sure your abstract accurately describes what the problem is that you're attempting to solve, and give a brief précis of how you hope to solve it.

Then, when you've got all your little ducks lined up, you submit to a journal.

You can think of this as like a driving test. When you've passed your driving test, you have a token that says 'I have mastered the basics of controlling this machine, and now I am allowed to go and learn to drive'. That's what your paper is now doing. 

In this analogy, the author of the paper is the driving instructor, and the paper itself is the trainee driver.  When the instructor thinks his pupil is ready, he refers her to a test centre (science journal) for a driving test (publication). The test centre will now assign an examiner (an anonymous expert or group of experts in your field - the referee), and the examiner will then test your driving ability to destruction. It's in the driving examiner's best interest to make sure he's harsh, because he doesn't want to let unsafe drivers on the road, so he's actively looking for reasons to fail you. 

Similarly, a referee will test a paper to destruction, and it's in his best interest to work as hard as he can to demonstrate that the conclusion drawn is not supported (see falsification and falsifiability in my earlier post). Also, the journal doesn't want to publish papers that don't stand up to critical scrutiny, because that's damaging to the reputation of the journal, and affects its impact factor (a measure of its influence on science at large, loosely speaking). 

If the referee can find no problems with the paper, and has failed to falsify it, he will pass the paper and it will be published in the journal. Now, the paper has passed its driving test and is ready to go out there and face the rigours of traffic in the real world. Some papers will get honked at by other drivers for cutting them up, others will shake fists at it, or advise them where the accelerator pedal can be found, and the paper makes its way. 

OK, so I'm running a bit too far with this analogy, but the point is, I hope, well taken.

The point is that the scientist must work really hard to get her assertions through this process, and it doesn't stop at publication because, once published, it's fair game for anybody who wants to challenge it. Sometimes, those challenges will comprehensively falsify the paper, such as happened with the BICEP-2 result, because the reviewers spotted a flaw in the reasoning, namely that it hadn't ruled out a known source of the readings they were getting prior to attributing them to the mechanism behind the underlying hypothesis. Note that the hypothesis itself wasn't falsified. More on that here.  

So, this is what scientists are used to seeing with claims. It's a good process; not flawless, but it works (in fact, in the long run, it pretty much IS flawless, because shoddy or incorrect papers are always weeded out in the long run, because science is self-correcting). 

The important point is that, given any assertion or hypothesis, it's always up to the one evincing said hypothesis to support their claims and to provide the data upon which any conclusions are based. Now, I'm sure that no scientist I know expects such a level of rigour from everybody, non-scientists included, but there are some basic standards of discourse that absolutely should be met by everybody, and among them is the principle that countering the rebuttable position with supporting data is the responsibility of the claimant. These standards are not being met in the above exchanges, and this is all-too prevalent in the social media sphere and the world at large.

So, if you're going to get into it with a scientist (or with anybody else, for that matter), be prepared to provide the data upon which your assertions are based. Failing to do so is every bit as effective in getting your point across as saying nothing at all, except that now you've pissed somebody off, to boot..

That which can be asserted without evidence can be dismissed without evidence - Christopher Hitchens.

And so it should be. 

 

I hear ya, sister! 

[1] On the Perpetuating Nature of Social Stereotypes - Snyder 1981 (cited from Hamilton 2015)