Archive for the ‘science’ Category

Your astrological sign can maim you, kill you or make you insane (or not?)

April 24, 2014

Can day of birth (as in here, here too and there) or  astrological sign affect later outcomes?

These are essentially the same question – astrological sign is determined by  the day of one’s birth but our reaction to the question is different. Asked in the first way, this is a legit question about an Epidemiological signal known since antiquity and is officially known as “season of birth effect” (SBE). (Historical trivia: the first use of this term may be found in Ellsworth’s Huntington book: Season of Birth: Its Relation to Human Abilities published in 1938). The SBE appears to be relevant for many diseases, outcomes and human populations and countries. Since we westerners are more likely to bite the bullet from cardiovascular disease or cancer, rest assured that the association has been noted for the former and the latter (though not everyone agrees)

Asked in the second way, the same question can immediately bring ridicule and shame on the scientifically oriented person who dares to ask it. In fact there is a small cottage industry of papers that use astrological sign in order to caution/warn/ridicule/discredit subgroup analyses in clinical datasets. The most famous of this is a subgroup analysis of the ISIS-2 but there are certainly others.

So what makes us think that the Season of Birth Effect is real and worthy of study across disciplines, while the astrological sign association is bogus and should be discarded any time it is detected? The answer may be found in the framing effect, a form of cognitive bias in which our reaction and even answer to the same question critically depends on the way it is framed. Though the framing effect is particularly relevant in sociology, it is also operational in medicine particularly affecting the choices patients make depending on the format of presentation of the same information.

So if framing effects are also operational in scientific discourse (at least in the medical field) how do we deal with them? The first way is to acknowledge their potential presence so as to minimize their impact by appropriate use of neutral terminology and avoid sensationalism when describing associations and effects. As the season-of-birth effect example shows one can present the same information to evoke sharply different responses, so one should always be on the look for emotions when reading a story in either the scientific or the lay press.

So to come back to the initial question: does one’s day of birth or astrological sign affect later outcomes? The associations seem to be there and are relatively consistent and reproducible so some phenomenon must be at play. Whether this phenomenon is real or a statistical analysis artifact (e.g. could age-period-cohort effects and the troubled history of the 20th century be at play here) is something I don’t know. But I would certainly be willing to find more about it, irrespective of whether the information is presented by month of birth or astrological sign 🙂

Failed Randomization In A Randomized Trial?

November 5, 2013

We will continue the saga of the three-arm clinical trial that is giving the editors of the prestigious journal The Spleen a run for their money. While the polls are gathering digital dust, let’s see if we can direct this discussion to a more quantitative path. To do so, we will ask (and answer) the question from a frequentist point; according to this approach we raise the red flag if the event under examination is rare assuming a hypothesis about the state of the world (null hypothesis H_0) is true.

In this case the null hypothesis is that the investigators at Grand Fenwick Memorial did run a randomized control trial under a simple randomization scheme, in which each patient had equal chance to be given one of the three interventions: GML, SL or MBL. To calculate the rarity of the observed pattern, we need to define an appropriate event and then figure out its rarity (“long-term frequency”) in many repetitions of the randomization allocation scheme used in the trial.

Considering the number of patients in the three arms of the trial, 105/70/65, v.s. the expectation of 80/80/80  it would appear that the most influential factor in determining the “rarity” of the observed pattern is the difference in size between the largest and the smallest arm in the trial.  On the other hand a difference of 5 between the second largest and the smallest arms would not appear to be worthy of consideration, at least as a first approximation. To determine the long term frequency of the event in a trial with 240 patients, we will use the R language to carry out a large number of these hypothetical allocations and figure out the number of those in which the difference in size between the largest and smallest arms exceeds 40:

 event<-c(105,70,65)  ## observed pattern
## computes the difference in size between arms
frequentist2<-function(x,l1=40) {
x<-sort(x,decreasing=TRUE)
I((x[1]-x[3])>=l1)
}
set.seed(4567) ## for reproducibility
## hypothetical trials
g<-t(rmultinom(500000,sum(prob),c(1,1,1)))
## flags the repetitions of the studies in which a rare
## event was observed and calculates the frequency (in %)
res3<-apply(g,1,frequentist2);mean(res3)*100
 

This number comes out to be 0.5%. In other words, 1 out of 200 randomized trials that assign patients with equal probability to three arms will generate an imbalance of this magnitude.
But is this the answer we are trying to obtain? In other words the situation that the editors of The Spleen face is to evaluate the likelihood that patients were not randomly assigned to the three interventions. This evaluation is only indirectly related to the rarity of observing a large size difference in the arms of a trial that did not cheat. By not considering directly the hypothesis of foul-play (unequal allocation probabilities in the three arms), both the investigators and their accusers will find themselves in endless quarrel about the interpretation of rarity as a chance finding v.s. an improbable one indicative of fraud.

Is this evidence of scientific fraud?

October 30, 2013

(the names of countries, journals, interventions and sample sizes have been changed to protect the potentially innocent and to avoid cognitive biases, invocations of  stereotypes and accusations that could lead to World War Z)

Two years ago the prestigious medical journal The Spleen published a randomized controlled trial that evaluated three therapies: genetically modified leeches (GML), standard leeches (SL) and mechanical blood-letting (MBL, standard of care) for acute complicated absentmindedness (ACA). This single center study randomized, in 1:1:1 ratio , 240 patients who presented at the Emergency Department of the Grand Fenwick Memorial Hospital and concluded that GML was associated with a 90% improvement in outcomes relative to SL and MBL. The lead author, a prominent absentmindedneologist and President of the corresponding Society of the Duchy of Grand Fenwick, concluded that GML should become the new standard of care and that SL and MCL should not be offered except as second line treatment for patients failing GML.

(more…)