Sampling and experimental design Knowledge-Based Study

Study notes

1. Populations and Samples

   In statistics, a population is the entire group of individuals, objects, or events about which we want to draw conclusions. A sample is a subset of the population selected to gather data and make inferences.

   The Role of Randomness

   To draw valid inferences about a population, the sample must be representative. When selection methods favor certain outcomes or segments of the population over others, the sample suffers from sampling bias.

   • Selection Bias: Occurs when the sampling procedure systematically excludes or underrepresents parts of the population (undercoverage).
   • Voluntary Response Bias: Occurs when individuals choose whether to participate, often overrepresenting extreme viewpoints.
   • Nonresponse Bias: Occurs when selected individuals cannot be contacted or refuse to participate, and their characteristics systematically differ from respondents.

   Common Random Sampling Techniques

   Probability sampling methods ensure that every individual in the population has a known, non-zero probability of selection, reducing systemic bias.

   • Simple Random Sample (SRS): Every possible group of n individuals has an equal chance of being selected. This is typically implemented using random number generators or drawing without replacement.
   • Stratified Random Sample: The population is first divided into homogeneous subgroups called strata based on a shared characteristic (such as age or income level). A simple random sample is then drawn independently from each stratum. This guarantees representation across all subgroups and reduces variability.
   • Cluster Sampling: The population is divided into heterogeneous subgroups called clusters that each resemble the overall population (such as geographic city blocks). A random sample of clusters is chosen, and every individual in the selected clusters is surveyed.
   • Systematic Sampling: Individuals are chosen at regular intervals from an ordered list, such as every k-th individual starting from a randomly determined initial point.

   Distinguishing Studies from Experiments

   The method used to collect data dictates the strength of the conclusions that can be drawn:

   • Observational Study: Researchers observe individuals and measure variables of interest without attempting to influence or manipulate the conditions. Observational studies can demonstrate an association or correlation between variables, but they cannot prove causation due to the potential presence of unmeasured confounding variables.
   • Experiment: Researchers deliberately impose one or more treatments on subjects (experimental units) to measure their response. Well-designed experiments allow for causal conclusions because the researcher controls and isolates the variable under study.

   A confounding variable is a variable that is related to both the explanatory variable and the response variable, obscuring whether the explanatory variable is actually causing changes in the response.

   The Core Principles of Experiments

   A valid experiment relies on four fundamental principles to isolate the effect of the treatment:

   1. Control: Controlling external factors keeps conditions as similar as possible across groups, preventing confounding. A control group provides a baseline comparison and often receives a standard treatment or a placebo.
   2. Randomization: Experimental units must be assigned to treatment groups using a chance process. Random assignment balances the effects of uncontrolled or lurking variables across treatment groups.
   3. Replication: Applying each treatment to many experimental units reduces the role of natural variation and increases the reliability of the findings.
   4. Blocking: If units are known to differ in a way that affects the response variable (such as biological sex or age), units are first grouped into homogeneous blocks, and random assignment to treatments is conducted separately within each block.

   Blinding and the Placebo Effect

   The placebo effect occurs when subjects respond favorably simply because they believe they are receiving a treatment. To counter this, studies use single-blinding (where subjects do not know which treatment they receive) or double-blinding (where neither the subjects nor the administrators measuring the outcome know who received which treatment).