Courtesy of Bronte Capital
I seem to have annoyed some statistics fundamentalists with the last statistics post. So let me spell it out from first principles…
The current gold-standard in epistemology is Karl Popper's view that all knowledge is provisional – and you are entitled to believe your educated guess so long as it agrees with experiment. However when someone does an experiment that contradicts your "knowledge" you are no longer entitled to your educated guess. Your educated guess goes into the book of failed theories.
To continue to believe your educated guess is non-scientific.
Popperism applies not only to your theories but to your meta-theories – your theories as to how to test knowledge.
A simple example suffices. People – good, clever people, for centuries believed that the only way to determine whether something was right was to look in the Bible. If the Bible supported it then it is right.
Nowadays we have developed several well-tested theories which are in direct contradiction to the Bible (see deep time in geology and evolution for the best examples).
Not only is the creation myth in Genesis falsified but so is the meta-theory that the appropriate test for knowledge is that it is printed in the Bible.
The FDA's meta-theory
During the FDA panel hearing for Lemtrada (an MS drug under development from Genzyme/Sanofi), the panel voted no on the proposition that the trials are adequate and well-controlled, and yes on the proposition that applicant provided substantial evidence of effectiveness of alemtuzumab [Lemtrada] for the treatment of patients with relapsing forms of MS.
The FDA staff asserted that you could not possibly vote no on whether the test was adequate and well-controlled and yes on effectiveness.
In doing this the FDA staff were asserting a meta-theory – the theory that the only test of effectiveness is an adequate and well-controlled test.
I observe that adequate and well-controlled test is something defined in legislation (see here). And that parachutes as a device to prevent trauma and death to people who jump out of planes have never been subjected to an adequate and well-controlled test as defined in the legislation.
Yet we know that parachutes are effective despite the absence of an adequate and well-controlled test.
In other words we have a contradiction to the meta-theory. The meta-theory demands an adequate and well-defined test as the only method of knowledge and the parachute example is a direct counter-example.
The FDA's meta-theory has been contradicted.
However, much to my surprise, and several years after the publication of the famous parachute paper the FDA staff (and some of my blog readers) still asserts their meta-theory.
I have now categorized the FDA staff in the pseudo-science camp – those who – like their fellow-traveller creation scientists – support their meta-theory in the face of direct falsification.
Hopeless. Stupid too.
