Original Research
Part of Original research for digital PR
Recording sample selection and exclusions
Document who could enter a study, how cases were selected, why records were excluded and which denominator supports each result.
Keep a dated record of how people or records could enter a study, and why cases were removed. A final sample size cannot show who was reachable, who took part or which exclusions shaped a result. Begin the record when selection starts, while decisions can still be checked.
Identify the population, source and unit
Name the group the research aims to describe, then identify the actual source: a customer list, online panel, business-record extract or another defined frame. State what one case represents. An account may contain several people, and one person may generate several transactions. Counting rows as people would change what a percentage means.
Record how the source was assembled, its geographic and time coverage, and known gaps. If people volunteered, state where the invitation appeared and who could see it. If cases were randomly selected from a defined list, retain the list version and selection procedure. “Random” should describe a process that can be explained.
Statistical Concepts in Sample Selection
- FrameDefined list from which sample is drawn (e.g., customer list, business record extract)
- Unit of AnalysisWhat one case represents (e.g., person, transaction, account)
- Coverage GapKnown limitations in the source list (e.g., missing demographic groups)
- RepresentativenessCannot be assumed from matching visible characteristics alone
Keep an inclusion and exclusion log
Set eligibility rules before reviewing individual results. For each rule, keep its reason, date, change history and number of cases affected. Rules may cover falling outside the stated period, failing an eligibility question, being a confirmed duplicate or lacking an answer needed for one calculation. Which rules are appropriate depends on the research question.
A flow record distinguishes:
| Stage | What to record |
|---|---|
| Source | Units in the frame or extract and any known coverage gap. |
| Eligibility | Units known to meet or fail the entry rules; mark unknown eligibility separately. |
| Contact or access | People invited and reached, or records available for analysis, as applicable. |
| Completion | Responses or records meeting the stated completeness rule. |
| Result base | Cases that contributed to each published figure. |
Record overlap between exclusion reasons so cases are not subtracted twice. A participant may be eligible overall but omit one answer; they remain in the study yet are absent from that result’s denominator. For a survey, distinguish invitations, people reached and completed responses where those counts can be established. A record-based study may need different stages.
Sample Selection and Exclusion Process in Research
- SourceUnits in the frame or extract, including known coverage gaps
- EligibilityUnits meeting or failing entry rules; unknown eligibility marked separately
- Contact or AccessPeople invited and reached, or records available for analysis
- CompletionResponses or records meeting completeness criteria
- Result BaseCases contributing to each published figure
Preserve changes and explain their effect
If a rule changes after collection starts, keep the earlier rule, new wording, change date and reason. Recalculate affected figures consistently where possible. Do not quietly drop cases because they weaken a preferred finding. Distinguish a suspected duplicate from one confirmed under a documented check.
Assess coverage and missingness against the target population. A gap in a source list, low participation by an eligible group or many unanswered items may narrow what can be said. Matching the population on a few visible characteristics would not, by itself, establish representativeness. Record what is known and what cannot be measured.
Before release, another reader should be able to trace a published denominator through the selection and exclusion record. If they cannot, repair the record or narrow the claim. The log explains how the result was made; it does not turn a voluntary sample into a probability sample.
