Delete or trim? Yuen’s test, data exclusions and what we estimate
Extreme observations raise different questions. Measurement validity: is a record erroneous or ineligible? Estimation: if an observation is valid, do we want the arithmetic mean or a location estimator less sensitive to extreme magnitudes? Justified exclusions may address the first; trimmed means are one option for the second. Neither approach is automatically correct. Explore how these choices affect the estimand, standard error, and inference. The independent-groups example is simulated; the paired mode also includes the publicly available 92-person water-tasting data.
1. The example
A two-independent-groups Stroop experiment: participants complete incongruent trials under cognitive load (n-back; group A) or without it (control; group B). The outcome is each participant's mean response time (ms). The data are simulated for illustration. Choose which kind of extreme values the sample contains; every section below updates. Important: this illustrates independent-samples Yuen (WRS2::yuen()), not a paired test. Paired observations, such as two ratings from each participant, require a paired method and an explicitly chosen paired estimand.
Explorer: trim and Winsorise a separate toy sample (n = 20)
Drag the slider or press Play. Top row: sorted data, trimmed points in amber. Bottom row: winsorizing pulls those points in to the nearest kept value.
2. Delete or trim? First establish validity and the estimand
Delete: the value is invalid
- Implausible given the instrument (participant mean RT below 150 ms: anticipatory responding or recording error)
- Documented failure: software or equipment fault, protocol violation, failed preregistered attention check
- Uncorrectable data-entry or coding errors; ineligible or duplicate participants
Prefer prespecified, independently justified data-quality rules, applied without reference to the desired result; document any later-discovered faults transparently. Report exclusions and sample sizes by group.
Trim: the value is valid but extreme
- Genuinely slow responders (lapses, fatigue, mind-wandering); RTs and durations are right-skewed by nature
- Heavy-tailed or contaminated-normal distributions; plausible individual differences
- Anything you can show is odd only by its size
Retain valid observations in the source dataset, then choose an estimand deliberately. Options include the arithmetic mean with appropriate inference, trimmed means, or a suitable distributional model. A bootstrap changes inference but need not change the estimand.
Not sufficient justification for deletion
- "It is far from the mean" or exceeds a post hoc ±2.5–3 SD rule: magnitude alone does not establish invalidity, and extreme values can inflate the SD used to flag them
- "The result is not significant yet", or "a reviewer finds it odd"
- "Cut the same number of points that trimming would" and run an ordinary t test: this does not provide the usual SE for the trimmed population estimand (section 4)
Ask: is there independent evidence this record is invalid, or are we merely uncomfortable with its magnitude? If it is valid, deletion changes which observations define the analysis; consider what quantity you mean to estimate. A prespecified magnitude threshold can be defensible for a specific construct (e.g., physically impossible or anticipatory responses), but requires justification.
3. What trimming, winsorizing and deleting do to the data
Trimming drops g = ⌊γn⌋ observations per tail; the trimmed mean Mt averages the remaining h = n − 2g. Winsorizing replaces the g lowest values with the (g+1)th lowest and the g highest with the (n−g)th, keeping n and capping each extreme's influence. Yuen's test trims for location and winsorizes for spread:
SE(Mt) = √[(n−1)·SDw² / (h(h−1))] ≈ SDw / ((1−2γ)√n)
t = (Mt1 − Mt2) / √(d₁ + d₂), Welch–Satterthwaite df with h in place of n
The Winsorised variance, adjusted for the retained fraction, estimates the sampling variability of a trimmed mean, whose cut points are themselves random sample quantiles. It does not mean the trimmed observations contribute their original numerical magnitudes to the location estimate.
Your data, coloured by validity
Approach 1: deletion (selected rule)
Approach 2: exclude invalid only, then trim and winsorize
4. Why "pre-cut, then t test" is not the same thing
Symmetrically cutting the tails produces the same sample trimmed mean but does not justify analysing the middle h values as an ordinary, unselected sample. Their cut points were estimated from all n observations. For a single sample, the exact ratio of Yuen's estimated variance to the naive retained-sample estimated variance is ((n−1)/(h−1)) × (SDw² / SDkept²). The naive SE can therefore badly understate sampling uncertainty. Below is an independently checked simulation under true nulls (identical populations, 20,000 replications per row, seed 20261115; reproducible script provided):
| Population, n per group, γ | Yuen | Welch on pre-cut data |
|---|---|---|
| Normal, 20, .10 | .050 | .134 |
| Normal, 20, .20 | .050 | .219 |
| Normal, 50, .20 | .051 | .254 |
| Lognormal, 20, .20 | .035 | .209 |
| Lognormal, 50, .20 | .044 | .263 |
In these scenarios, the naive test increasingly exceeds the nominal 5% false-positive rate. This is a problem of selecting by observed rank and then ignoring that selection—not a demonstration that all scientifically justified exclusions invalidate t-tests. See also Bakker & Wicherts (2014).
5. The analyses side by side
Each estimate compares the two independent groups. They may describe different target populations and different measures of location. Do not interpret agreeing p-values as proof that the estimands agree.
Estimand. Welch on all measured data estimates a difference in arithmetic means, although invalid records can make that target scientifically meaningless. Approach 1 estimates the mean contrast in the selected dataset: if exclusions genuinely identify invalid records, this can target the means among valid observations; if they select by outcome magnitude, it generally no longer targets the original unselected population means. Yuen targets the difference in population γ-trimmed means, μt = (1−2γ)−1∫γ1−γF−1(u)du, a location contrast for the central (1−2γ) of each marginal distribution. The mean and trimmed-mean contrasts coincide for symmetric marginal distributions or pure location shifts, but not generally under unequal skew. γ is part of the estimand and ideally should be specified before seeing outcomes.
6. Figures and illustrative APA-style write-ups
Template warning: the dataset and validity flags are simulated. Copying these examples into a real report requires verifying every exclusion rationale, analysis, and numerical result. This is not evidence that your actual study preregistered any rule.
Approach 1: delete, then Welch
Approach 2: exclude invalid only, then Yuen
If you report both
State which was the primary (preregistered) analysis and report the other as a sensitivity analysis, naming each estimand. Do not choose after seeing p values.
R (for the downloaded CSV, independent groups): d <- subset(read.csv("yuen_simulated_stroop.csv"), validity == "valid"); WRS2::yuen(rt ~ cond, data = d, tr = 0.2)
Python (independent groups): scipy.stats.ttest_ind(a, b, equal_var=False, trim=0.2)
7. Paired-data comparison: two different trimmed estimands
For each person, we observe two linked ratings, A and B. The ordinary paired t-test estimates the mean of A − B. But there are two distinct targets for trimmed-mean inference: trim the distribution of each person's differences, or trim A and B separately before subtracting the two marginal trimmed means.
Verified R comparison
lumen::yuenTTest(x, y, paired = TRUE) trims paired differences. WRS2::yuend(x, y) trims the two margins separately, then adjusts its SE using the Winsorised covariance of the still-paired observations. The two tests target different parameters; the difference is not a software bug or a failure to respect pairing.
The three analyses, with 95% intervals
Trim differences — lumen
Compute D = A − B for every person; trim the lowest and highest γ of the differences. Its point estimate is Tγ(A − B). The Winsorised variance is calculated for those differences.
Trim marginal distributions — WRS2
Trim the lowest and highest γ of A and B separately. Its point estimate is Tγ(A) − Tγ(B). The uncertainty accounts for pairing by subtracting twice the Winsorised covariance; it does not treat observations as independent.
See why the selected participants differ
In the original water-tasting data, + means bottled water is rated higher. Gold points mark the pairs with the most extreme differences and therefore trimmed by the difference-based estimator. The diagonal is equal ratings.
Why the two standard errors differ
Reproduce these methods in R
# Vectors A and B must be aligned by participant ID. # Here A = Chaudfontaine, B = tap water. t.test(A, B, paired = TRUE) lumen::yuenTTest(A, B, paired = TRUE, trim = 0.2) WRS2::yuend(A, B, tr = 0.2) # Same trimming fraction; different population quantities.
Data provenance: Wagenmakers & Bartoš, water-tasting study; all 92 publicly shared participant records are included in this teaching view. This dashboard computes the published test formulae in JavaScript; check against the R calls above for formal analyses.
8. What is the estimand? Compare before you conclude
The estimand is the population quantity the analysis is meant to recover. Two analyses can use the same data, the same units, and even the same word “paired”, yet answer different questions. Conversely, two analyses can have different p-values while targeting an identical estimand using different uncertainty calculations.
Changing the estimator or uncertainty
For a fixed estimand, comparing plausible sampling distributions, variance estimators or priors tests inferential sensitivity. A student-t location parameter is not automatically equal to the arithmetic mean when its model is misspecified.
Changing the scientific question
Trimming the data distribution changes the functional (mean versus trimmed mean). Trimming different parts of paired data changes the functional again. These are legitimate different estimands, not contradictory answers to one question.
References
Algina, J., Keselman, H. J., & Penfield, R. D. (2005). An alternative to Cohen's standardized mean difference effect size: A robust parameter and confidence interval in the two independent groups case. Psychological Methods, 10(3), 317–328.
Bakker, M., & Wicherts, J. M. (2014). Outlier removal, sum scores, and the inflation of the Type I error rate in independent samples t tests: The power of alternatives and recommendations. Psychological Methods, 19(3), 409–427.
Tukey, J. W., & McLaughlin, D. H. (1963). Less vulnerable confidence and significance procedures for location based on a single sample: Trimming/Winsorization 1. Sankhyā, Series A, 25(3), 331–352.
Welch, B. L. (1947). The generalization of "Student's" problem when several different population variances are involved. Biometrika, 34(1–2), 28–35.
Wilcox, R. R. (2022). Introduction to robust estimation and hypothesis testing (5th ed.). Academic Press.
Yuen, K. K. (1974). The two-sample trimmed t for unequal population variances. Biometrika, 61(1), 165–170.
WRS2 documentation and source for yuend(); lumen documentation for yuenTTest(); and WRS2 Winsorised covariance implementation.
Limitations: simulated, independent-groups data only; one outcome type; no model of trial-level RT or its measurement errors; effect sizes shown without CIs (add appropriate CIs in a real report); the dt scaling constant approximates the Winsorised SD of a standard normal at the chosen γ. Source data are built into this page and can be exported above.