A study that assigns an intervention to whole teams does not acquire thousands of independent treatment assignments merely by measuring thousands of employees. The assignment happened at the team level. People within a team share managers, routines, and conditions, so the study's design and uncertainty must reflect that shared exposure.
TL;DR: Count both clusters and participants, justify why group assignment is necessary, and evaluate power using the planned analysis. The familiar design effect is a useful first calculation under limited assumptions, not a license to replace missing teams with unlimited people. Recruitment timing, unequal team sizes, attrition, and the target average can change the interpretation before any model is fitted.
A funding panel receives three proposals
Imagine a fictional research program testing a new workflow in customer support teams. The intervention changes the team's scheduling process and shared review meeting. The outcome is a continuous measure of resolution quality collected from individual employees after a fixed follow-up period. The intervention cannot realistically be assigned to some employees while their immediate teammates continue using the old shared schedule.
Three proposals reach the funding panel. Proposal A recruits ten teams with sixty employees each. Proposal B recruits twenty teams with thirty employees each. Proposal C recruits thirty teams with twenty employees each. All promise six hundred employee observations. The applicants describe their samples as equally large, but that statement is incomplete because the number of independent randomized units differs.
For a transparent first comparison, suppose team sizes are equal within each proposal, allocation is balanced between two arms, and the outcome's intracluster correlation is 0.05. The simple design effect is one plus the team size minus one, multiplied by that correlation. It represents variance inflation under the corresponding simplified setup. The cluster trial CONSORT extension explains why cluster size and within-cluster correlation enter planning. CONSORT cluster extension
| Proposal | Teams | Employees per team | Total employees | Simple design effect | Total divided by design effect |
|---|---|---|---|---|---|
| A | 10 | 60 | 600 | 3.95 | About 152 |
| B | 20 | 30 | 600 | 2.45 | About 245 |
| C | 30 | 20 | 600 | 1.95 | About 308 |
The last column is an intuitive effective sample size calculation, not a complete power result. It shows the cost of correlation under the stated approximation. It does not account for every feature of the analysis, the uncertainty associated with a small number of teams, covariate adjustment, missing data, or departures from equal cluster sizes. The panel should not report those values as exact numbers of independent employees manufactured by the design.
Nevertheless, the table changes the budget conversation. With the same six hundred employees, recruiting more teams can provide substantially more information than adding employees to existing teams. The relevant purchasing decision is not simply the cheapest observation. It is the cost of obtaining another independently assigned unit and a useful number of observations within it.
The principal investigator asks for larger teams
The investigator explains that onboarding a new organization is expensive. Surveying another employee inside an enrolled team is cheap. That is a legitimate operational constraint, but it does not alter the statistical structure. The panel needs a cost curve that includes both the fixed cost per team and the incremental cost per employee.
Suppose onboarding costs two thousand currency units per team and measuring an employee costs fifty. Ignoring all other costs, proposal A costs fifty thousand, proposal B seventy thousand, and proposal C ninety thousand. These invented totals follow from team onboarding plus thirty thousand for six hundred employee measurements. The designs differ in both cost and information; neither total sample size nor effective sample size alone determines value.
A sensible redesign might compare additional combinations rather than choosing one of those three automatically. More teams with fewer employees, stronger baseline measurements, or a narrower primary outcome could change the tradeoff. The methodological literature on efficient cluster trial design emphasizes choosing cluster number and size together. Efficient cluster trial design
The panel's first decision is therefore to fund a design exercise, not to approve a headline participant count. The exercise must use the intervention's delivery constraints, realistic recruitment costs, and an analysis that respects assignment. This preserves the practical reason for clustering while making its price visible.
The correlation estimate needs a provenance file
An intracluster correlation of 0.05 may sound negligible. With sixty employees per team, however, the simple design effect is 3.95. Correlation operates across many pairs of employees, so a small value can have a substantial consequence in large groups. The numerical size of the coefficient cannot be interpreted without the cluster size and outcome.
The panel asks where the estimate came from. An estimate for employee satisfaction may not be appropriate for resolution quality. A correlation measured within departments may not describe smaller operational teams. A historical estimate from a different organization may reflect different management practices or measurement procedures. The borrowing decision needs a substantive argument rather than a citation attached to an arbitrary decimal.
The planning file should preserve the outcome definition, population, cluster definition, follow-up timing, and statistical model associated with the estimate. If the planned analysis adjusts for baseline variables, it should be clear whether the planning correlation corresponds to an adjusted or unadjusted outcome model. Mixing these quantities can make a calculation look more precise while quietly changing its assumptions.
For proposal B, changing the assumed correlation from 0.02 to 0.10 changes the simple design effect from 1.58 to 3.90. Dividing six hundred by those values gives about 380 and 154, respectively. That range is large enough to change a funding decision. It deserves a sensitivity table rather than a footnote saying that the correlation was uncertain.
A pilot can help estimate feasibility and provide information about variation, but a small pilot's correlation estimate can itself be unstable. Treating its point estimate as known may simply move uncertainty from the main study into an unacknowledged preliminary calculation. Planning should explore plausible values and explain why the selected design remains useful across them.
The same principle applies to the expected intervention effect. A design cannot be rescued by assuming an unrealistically large effect. The panel should distinguish the effect that would justify adoption from the effect that a small preliminary study happened to estimate. Sample size is a consequence of the scientific decision, not a number selected first and rationalized afterward.
The teams are not actually the same size
The recruitment office returns with a more realistic roster. Some teams have eight employees, others have seventy. The mean remains thirty, but proposal B's equal-size calculation no longer describes the planned sample. The distribution of team sizes matters because a few large teams can dominate the number of observations without creating more independent assignments.
This is not merely a request to replace one formula with another universal correction. The consequences depend on the analysis, outcome, allocation, and relationship between cluster size and outcomes. The study should evaluate the actual or plausible size distribution, including whether small teams have different working conditions or intervention effects. A method that weights every employee equally may answer a different question from one that weights every team equally.
Consider two hypothetical teams. The small team has ten employees and an average improvement of ten units. The large team has ninety employees and an average improvement of two units. An equal-team average of those improvements is six units. An employee-weighted average is 2.8 units: one hundred improvement units from the small team plus one hundred eighty from the large team, divided by one hundred employees.
Neither calculation is automatically fraudulent. They answer different averaging questions. A policymaker interested in the expected benefit for a randomly selected employee may prefer an individual-average estimand. An organization deciding what happens to a typical team may care about a cluster-average estimand. The protocol should identify the target before the observed results make one weighting scheme more attractive.
The example is a demonstration of weighting, not an estimate of a randomized treatment effect. A real trial must construct the relevant comparison between assigned arms with a method appropriate to the design. Its value here is to show why the phrase average effect is incomplete when cluster sizes differ and effects may vary with size.
The panel now requires a recruitment distribution rather than only a mean. It asks how many small, medium, and large teams are available, whether the assignment procedure will balance important size differences, and how the analysis will handle the target population. If the final sample departs from the planning distribution, that departure belongs in the report.
Randomization does not protect recruitment that happens afterward
A second problem appears in the proposed timeline. Teams will be randomized first. Managers will then invite employees to participate, and managers will know which workflow their team received. A manager enthusiastic about the new process might recruit broadly, while a control manager might invite only the most reliable employees. Assignment was randomized, but the measured employees were selected afterward.
The risk is not removed by writing intention to treat at the top of the analysis plan. That principle addresses analysis according to assignment; it does not make the composition of an outcome sample immune to selection. If assignment affects who enters the study, the comparison among enrolled employees can differ from the intended comparison among eligible employees.
Research on cluster trial recruitment describes this timing problem explicitly. The timeline cluster approach asks investigators to show when clusters and individuals are identified, when allocation occurs, and who knows the assignment at each step. A chronological diagram can reveal a weakness that a conventional participant count misses. Timeline cluster method
For the support-team study, one option is to establish an eligible employee roster before randomization and collect outcomes consistently from that defined cohort, subject to the appropriate participation process. Another is to use recruitment staff who do not know assignment where feasible. Neither option should be assumed workable without checking the operational and ethical circumstances.
The team must also distinguish an open cohort from a closed cohort. In a closed cohort, the target employees are identified at a specified starting point. In an open cohort, employees may join and leave during follow-up. Turnover can be part of the intervention's effect or a source of changing composition. The analysis should not quietly switch between those populations because one is easier to observe.
An illustrative recruitment audit makes the issue concrete. Suppose ten intervention teams contain three hundred eligible employees and enroll two hundred forty, while ten control teams also contain three hundred eligible employees but enroll one hundred fifty. The enrollment rates are eighty and fifty percent. Those figures do not prove bias, but they establish a discrepancy that needs explanation and sensitivity analysis.
The audit should retain reasons for nonparticipation and the characteristics available for eligible nonparticipants where collection is appropriate. It should not infer that missing employees had average outcomes. The scientific question is whether the observed sample supports the intended inference, and what assumptions would be required to bridge any gap. Selection bias in cluster randomized trials
The analysis must know how many assignments occurred
The statistician receives a spreadsheet with one row per employee. A standard regression treating all six hundred rows as independent can produce overly narrow uncertainty intervals when within-team outcomes are correlated. Adding a team identifier column to the file does not solve the problem unless the analysis actually uses the grouping appropriately.
Several analysis strategies can be reasonable, including analyses of cluster summaries, mixed models, or marginal models with suitable uncertainty calculations. The choice depends on the outcome, estimand, covariates, and design. The panel should require a specific method and its assumptions rather than approving the phrase account for clustering as though it were a complete specification.
Small numbers of clusters require particular care. Many familiar uncertainty calculations rely on large-sample approximations whose relevant sample size is closer to the number of clusters than to the number of rows. A large employee count cannot automatically make those approximations reliable. Simulation research has examined how model tests and small-sample corrections behave under different cluster counts and covariate settings. Small-cluster model evaluation
The planning simulation should mirror the intended analysis rather than calculate power for one method and report results using another. It can vary the correlation, cluster-size distribution, outcome distribution, baseline adjustment, and missingness mechanisms that are plausible for this study. Results should show both power under meaningful effects and false positive behavior under no effect.
Baseline adjustment may improve precision, but the protocol should specify which variables are measured before assignment and why they are included. Variables affected by the intervention are not interchangeable with baseline predictors. Adjusting for a later consequence of the workflow can change the question being answered and introduce additional assumptions.
The randomization procedure also constrains valid analyses. If teams are matched or allocated within strata, the analysis should respect that design. A permutation approach, when appropriate, should use assignments that could actually have occurred rather than shuffling employees freely across arms. The randomization unit and allowed allocation set are part of the evidence.
One team per arm is an especially revealing failure case. A difference between two organizations could reflect the intervention or anything else that distinguishes those organizations. Measuring more employees inside each does not create independent replications of assignment. A report should not disguise this as a large randomized trial merely because it contains many individual records.
Attrition has two different scales
Losing five employees from each of twenty teams is not equivalent to losing several entire teams, even if the total number of missing employee records is similar. Whole-team withdrawal removes an assigned unit and may change the balance of organizational contexts. Individual dropout changes information within retained units. Both need to be anticipated, but they should not be collapsed into one generic attrition percentage.
Suppose proposal B starts with twenty teams of thirty employees. Losing three employees per team leaves twenty teams and five hundred forty employees. Losing two entire teams also leaves five hundred forty employees, but only eighteen teams. The identical final row count hides a different design and potentially a different selection process.
The reason for withdrawal matters as well. A team that leaves because the workflow is too disruptive provides different missingness information from a team lost through an unrelated organizational closure. Neither can simply be assumed to have the same unobserved outcome as retained teams. The report should describe the timing and reasons at the level where loss occurred.
Retention planning should therefore include organizational commitments, reliable outcome collection, and realistic procedures for staff turnover. These measures support data quality, but they should not become coercive attempts to keep participants in research. The study's participation and oversight arrangements remain part of the design rather than an administrative detail added after the sample size is approved.
When missing outcomes remain, the analysis should describe the assumptions behind its handling method and assess alternatives that could change the conclusion. A sophisticated model cannot supply information that the study never collected without relying on assumptions. Showing the effect of plausible departures is more informative than presenting one completed dataset as though no loss occurred.
A boundary experiment tests what the clusters really contain
The organizational chart is not necessarily the intervention's true boundary. Suppose two nominal teams share a supervisor who runs the new review meeting for both. Assigning those teams to opposite arms may create spillover because the supervisor cannot reliably deliver two separate routines. The original rationale for team assignment would then need to be reconsidered at a higher organizational level.
Alternatively, employees might move frequently between teams or participate in a shared message channel where the intervention's practices spread. Some spillover may be unavoidable and relevant to real deployment. The protocol should describe it and clarify whether the target is the effect of offering the workflow under those conditions or an effect under stronger separation. Those are different policy questions.
A larger cluster boundary can reduce contamination while making recruitment and inference more difficult. If the trial assigns entire offices instead of teams, twenty teams might collapse into only five offices. The design cannot keep using twenty as its count of independent assignments after changing the treatment boundary. This tradeoff belongs in the original justification for clustering.
The panel asks the investigators to map shared managers, physical locations, and cross-team collaboration before fixing the assignment unit. This map does not require measuring every social relationship. It requires enough operational understanding to avoid assigning different labels to groups that receive the same intervention through a common channel.
Why unlimited measurements cannot replace missing organizations
Under the earlier equal-size approximation, a study with K clusters of size m has an effective sample size of K times m divided by one plus the quantity m minus one times the correlation. If the correlation remains positive and m becomes very large, this expression approaches K divided by the correlation. The approximation therefore has diminishing returns as more people are added to a fixed set of clusters.
For ten teams and a correlation of 0.05, that limiting value is two hundred. The sixty-person version had an approximate effective sample size of one hundred fifty-two. Increasing each team to six hundred employees would produce six thousand observations, yet the same calculation gives only about one hundred ninety-four. The additional measurements help, but they do not provide the tenfold increase suggested by the row count.
This limiting calculation should not be mistaken for a universal ceiling on every possible analysis. It relies on the simplified correlation model and does not remove the separate small-cluster inference problem. Its practical role is to expose a budget proposal whose promise of precision depends entirely on measuring ever more people inside a handful of organizations.
It also explains why improving measurement quality can be useful without solving the replication problem. A more reliable employee outcome can reduce some noise and sharpen comparisons. It cannot establish how the intervention behaves across organizational settings that were never independently assigned. The study needs both a credible measurement process and enough relevant units to support its intended claim.
The panel rewrites the deliverables
The revised grant asks for a cluster roster, an employee eligibility definition, an allocation procedure, a recruitment timeline, and a prespecified estimand. It asks for power calculations tied to the planned analysis and sensitivity to uncertain correlations. It also requires reporting the number of clusters and people contributing to each important outcome.
The primary result will include an effect estimate and uncertainty interval with the outcome scale made explicit. The report will describe cluster sizes and losses, the method used to account for grouping, and relevant correlation estimates. These details let another researcher judge the evidence and plan a subsequent study without reverse engineering the design from a participant total.
The panel does not select proposal C solely because its simple effective sample size was largest. It compares feasibility, cost, scientific relevance, and the precision achievable under realistic assumptions. If only a small number of suitable teams can be recruited, the project may be more useful as a feasibility study with appropriately limited claims than as an underpowered definitive test.
The decisive change is that six hundred employees no longer functions as a complete description of the experiment. The design now states who was assigned, who was measured, how observations share conditions, and which population average the result addresses. Those commitments make the eventual estimate interpretable before the study knows whether its headline will be favorable.