## Archive for the ‘hypothesis evaluation’ Category

### Failed Randomization In A Randomized Trial?

November 5, 2013

We will continue the saga of the three-arm clinical trial that is giving the editors of the prestigious journal The Spleen a run for their money. While the polls are gathering digital dust, let’s see if we can direct this discussion to a more quantitative path. To do so, we will ask (and answer) the question from a frequentist point; according to this approach we raise the red flag if the event under examination is rare assuming a hypothesis about the state of the world (null hypothesis $H_0$) is true.

In this case the null hypothesis is that the investigators at Grand Fenwick Memorial did run a randomized control trial under a simple randomization scheme, in which each patient had equal chance to be given one of the three interventions: GML, SL or MBL. To calculate the rarity of the observed pattern, we need to define an appropriate event and then figure out its rarity (“long-term frequency”) in many repetitions of the randomization allocation scheme used in the trial.

Considering the number of patients in the three arms of the trial, 105/70/65, v.s. the expectation of 80/80/80  it would appear that the most influential factor in determining the “rarity” of the observed pattern is the difference in size between the largest and the smallest arm in the trial.  On the other hand a difference of 5 between the second largest and the smallest arms would not appear to be worthy of consideration, at least as a first approximation. To determine the long term frequency of the event in a trial with 240 patients, we will use the R language to carry out a large number of these hypothetical allocations and figure out the number of those in which the difference in size between the largest and smallest arms exceeds 40:

``` event<-c(105,70,65)  ## observed pattern
## computes the difference in size between arms
frequentist2<-function(x,l1=40) {
x<-sort(x,decreasing=TRUE)
I((x[1]-x[3])>=l1)
}
set.seed(4567) ## for reproducibility
## hypothetical trials
g<-t(rmultinom(500000,sum(prob),c(1,1,1)))
## flags the repetitions of the studies in which a rare
## event was observed and calculates the frequency (in %)
res3<-apply(g,1,frequentist2);mean(res3)*100
```

This number comes out to be 0.5%. In other words, 1 out of 200 randomized trials that assign patients with equal probability to three arms will generate an imbalance of this magnitude.
But is this the answer we are trying to obtain? In other words the situation that the editors of The Spleen face is to evaluate the likelihood that patients were not randomly assigned to the three interventions. This evaluation is only indirectly related to the rarity of observing a large size difference in the arms of a trial that did not cheat. By not considering directly the hypothesis of foul-play (unequal allocation probabilities in the three arms), both the investigators and their accusers will find themselves in endless quarrel about the interpretation of rarity as a chance finding v.s. an improbable one indicative of fraud.

### Argumentation between people of different convictions: Practical Implications of the Lindley Paradox

July 9, 2013

The Lindley paradox  can teach us some deep lessons about the conduction of our everyday business: one cannot (and in fact should not!) expect to win over an argument when one is confronting a person with much stronger convictions, even if that person is wrong!! Furthermore when we witness such an encounter, one cannot (and in fact should not!)  always attribute this outcome to other reasons e.g. deference to authority, lack of determination or even outright bullying. Even in the absence of these factors a rational exchange between two individuals of widely different convictions would lead to the one with the stronger ones coming out of the debate a winner. This is not particularly troublesome if the person is right, but it turns out to be problematic when that person is outright wrong. (more…)

### On the futility of discourse between two people of different degrees of conviction : Reflections on the Lindley Paradox

July 6, 2013

In an exchange between two people, it is the one with the stronger conviction,not the better grasp of reality, who will win.

The phrase above snapped into my head when I watched an honest and very extensive dialog between two business partners who were examining an issue. Partner A had come into the table with a very strong conviction about the issue and what needed to be done, while partner B was not so sure and was open to entertain a number of possible courses of action. Over a course of 45 min, they examined the data available to them and decided to proceed with A’s opinion. What is surprising is not the decision per se , but the fact that before the meeting A had confided to me that the available information contradicted his belief; B had also said that A’s position was rather implausible given the same data. Yet, when they sat down and considered what was in front of them they reached a rather different conclusion!