Blog

Longer write-ups of my papers, with the methods, the R software and the numbers behind the conclusions, and practical notes on statistics in biomedical research.

Posts

Where a confidence interval comes from: the bootstrap, the sampling distribution and why the simplest bootstrap interval is too narrow in small samples

A confidence interval is read off a sampling distribution, and the bootstrap estimates that distribution by resampling the data. Three interactive figures show how, why the simplest bootstrap interval is too narrow in small samples, and how often the percentile, BCa, bootstrap-t and t intervals cover the truth. Tables cover what to resample in each design and how many resamples are enough, and a checklist covers what to report.

bootstrap confidence intervals resampling

Where a p-value comes from: the permutation test, the null distribution and the distribution of the p-value itself

Every p-value is a tail area of a null distribution. Shuffling the group labels builds that distribution from the data. Three interactive figures show how, how the t distribution approximates it and where it fails, and how the p-value itself varies from study to study. Tables cover how many shuffles are enough, what a shuffle assumes and what to shuffle in each design, and a checklist covers what to report.

p-values permutation tests null distribution

The p-value, the effect size and the confidence interval: what each tells the reader, and how to report them in a paper

A p-value, an effect size and a confidence interval answer three different questions about one result. Three figures (two of them interactive) show how they behave, and tables show how to read an interval against the smallest effect that matters and what to report for common analyses.

p-values effect size confidence intervals

Leakage defence in layers: how splitGraph, fastml and bioLeak divide the work of honest performance estimation

Leakage in cross-validation comes from study structure, from procedure, and from artefacts only the data reveal. This methods note shows how three R packages take one source each, how they compose into one pipeline with a feedback loop, and what each layer buys in a 30-replicate simulation.

data leakage cross-validation R packages