Is a pilot study useful to determine the size of a cluster randomised trial?

Talk Code: 
2A.3

The problem

Pilot studies for cluster randomised trials are used to estimate the intracluster correlation coefficient (ICC), a key component in the sample size calculation for the main trial. However there is little guidance on how large a pilot study for a cluster randomised trial should be in order to make reasonably accurate estimates.

The approach

We considered trials where the intervention effect would be estimated as the difference between 2 means. We estimated the distribution of the expected number of clusters for the main trial under different assumptions about the natural size of clusters, the ICC (0.01, 0.05, 0.2) , the eventual cluster size in the main trial (the same as in the pilot or larger), and cluster size in the pilot (2, 5, 10, 50, 200), using simulation. We assumed the pilot study would be estimating both the standard deviation of the outcome and the ICC. We chose ICC values and pilot study size to broadly reflect those commonly reported in the literature.

Findings

The median sample size calculated from the simulated pilots was lower than the true value in many cases and also the values were very widely spread. If we assume the pilot and main study will have 50 individuals per cluster and 480 participants were required for an individually randomized trial , then the number of clusters required for an ICC of 0.05 would be 34. Our simulations show that using a pilot study with only 5 clusters , 50% of the pilot studies will give an estimate of 27 clusters or less required for the main study, 10% of pilot studies would lead to a main study having 10 or less clusters and 10% would give lead to 59 or more clusters. Changing the ICC and cluster size did not change the conclusionsDiscussion: While pilot studies may give useful insights into the processes of the trial, they are of limited value in estimating the ICC. In many cases results from a pilot study will lead to the main trial being too small. ICCs for different outcomes follow known patterns, depending on whether they are patients outcomes or process measures and the size of the natural cluster. Primary care ICCs for patient outcomes are usually less than 0.05. This information should be used alongside any information from the pilot study

Credits

  • Sally Kerry, Lancaster University, Lancaster, UK
  • Sandra Eldridge, Lancaster University, Lancaster, UK
  • Ceire Costello, Lancaster University, Lancaster, UK
  • Brenan Kahan, Lancaster University, Lancaster, UK
  • Gillian Lancaster