Statistics & Data Science

Start with questions, study units, and distributions. Move through probability, sampling, estimation, and tests before modeling, causal reasoning, and prediction; connect specialized methods to the problems they answer.

Questions, Data, and Description

Define the question, observational unit, variables, and distribution.

Data, Questions, and Study Units

• Population and sample — the population is the full set about which an analysis seeks to learn; a sample is the observed subset. A large sample can still misrepresent its target when selection is biased.
  • sample size — Sample size is the number of observations in a dataset, usually written n. It appears in formulas such as the standard error, degrees of freedom, and power analysis, where larger samples generally improve precision. It also enters many introductory statistics results, including the central limit theorem and Bessel’s correction. In hypothesis testing, a smaller sample size can make a type II error more likely.

• Unit of analysis — the entity represented by one observation in the analysis, such as a person, household, school, county, or cell. Standard errors and conclusions are wrong when dependence within higher-level units is ignored.

• Variable — a recorded attribute that varies across units. Quantitative variables encode amounts; categorical variables encode groups or labels; ordinal variables have an order without guaranteed equal spacing.
  • variables
    • independent variable — The independent variable is the factor in an experiment that is changed or manipulated. If the null hypothesis is true, it has no effect on the dependent variable.
  • Data types
    • categorical data — Categorical data consists of observations that fall into a fixed set of discrete values, rather than a continuum. Examples include blood type and country of origin, with binary cases described as dichotomous.

• Outcome, exposure, and covariate — an outcome is the response being explained or predicted; an exposure is a condition or intervention of interest; covariates are other measured features included for design, adjustment, precision, or description.

• Estimand — the precise population quantity a study intends to estimate. Examples include a population mean, a risk difference, an average treatment effect, or predictive error in a specified target population.

• Data-generating process — the mechanisms that produce both the underlying phenomena and the recorded data. Measurement, selection, missingness, and preprocessing are part of this process rather than nuisances outside the analysis.

• Exploratory and confirmatory analysis — exploratory work searches for patterns and helps generate hypotheses; confirmatory work tests prespecified questions under defined error controls. The same dataset can support both only when their roles are reported honestly.

• Parameter and statistic — a parameter describes a population or model; a statistic is computed from observed data. Statistics estimate parameters only relative to a defined target and sampling process.

• Unit of observation — the entity on which a measurement is recorded. It can differ from the unit of analysis, as when repeated visits are observations but the person is the independent unit.

• Data dictionary — documents each field’s meaning, type, units, allowed values, missing-value codes, derivation, and provenance so analysts do not infer semantics from column names.

• Data provenance — records where data originated and how collection, linkage, cleaning, transformation, and versioning changed them. Provenance makes errors traceable and results auditable.

• Censoring and truncation — censoring preserves partial information about an unobserved value, while truncation removes units outside an observation rule entirely. The two mechanisms require different likelihoods and interpretations.

• Long and wide data — long data store repeated measurements in multiple rows; wide data store them in multiple columns. The useful layout depends on the operation and does not change the underlying study design.

Describing Distributions

• Frequency distribution — a count or proportion for each value or interval. A histogram displays quantitative distributions; a bar chart compares discrete categories and should not imply continuity between them.

• Mean — the arithmetic average. It uses every value and is sensitive to extreme observations. It is found by summing values and dividing by the number of observations or terms. Related forms include the geometric mean and harmonic mean, and it is often contrasted with the median and mode. It also appears in thermodynamics as expectation values and time or ensemble averages, especially in statistical mechanics when computing quantities over ensembles or phase space.

• Median — the middle ordered value, or the average of the two middle values for an even sample size. It is often more resistant than the mean to a small number of extreme observations. It also names the triangle segment from a vertex to the midpoint of the opposite side, and appears in algorithms such as median-of-medians and median maintenance.

• Mode — the most frequent value or category. A distribution can have one mode, several modes, or no uniquely most frequent value. The mode of the series 1, 1, 2, 3, 5, 8, 13, 21 is 1. The mode is the value that appears most often in a list of numbers.

• Variance and standard deviation — variance averages squared deviations from the mean under a population or sample convention; standard deviation is its square root and returns to the variable’s original units. Variance is the second central moment, equal to the expected square minus the square of the expectation. Bessel’s correction, homoscedasticity, and mean-squared-error decompositions provide common contexts; some heavy-tailed distributions have undefined variance.
  • Sample variance — Sample variance is the spread measure computed from squared deviations from the sample mean. It differs from population variance by using a denominator one less than the sample size.

• Interquartile range — the difference between the 75th and 25th percentiles. It summarizes the middle half of a distribution and is less sensitive to tails than the full range. In a boxplot, it is shown by the length of the box and is closely related to the median of the lower and upper halves.

• Skewness — asymmetry in a distribution. Right-skewed data have a longer upper tail, often pulling the mean above the median; left-skew reverses that pattern. It is often described by the sign of the third central or standardized moment, indicating whether the distribution has a longer left or right tail. Positive skew means the distribution has a longer right tail, often pulling the mean above the median, while negative skew produces the opposite pattern.
  • coefficient of skewness — The coefficient of skewness measures the asymmetry of a probability distribution. It is defined using the third central moment divided by the cube of the standard deviation, and it is zero for symmetric distributions.

• Outlier — an observation far from most others under a stated rule or model. Outliers can reflect error, rare valid cases, mixtures of populations, or model failure and should be investigated rather than automatically deleted. An outlier is often treated as anomalous in statistics and machine learning. Outliers can distort summary measures such as the mean, are often shown as points beyond the whiskers in a box plot, and are common in heavy-tailed distributions.

• Standardization — subtracting a reference mean and dividing by a reference standard deviation produces a z-score. Standardization changes scale, not the shape or information content of the distribution.

• Quantile and percentile — a quantile is a value below which a specified proportion of observations falls; a percentile states that proportion on a 0-to-100 scale. Sample conventions can yield slightly different endpoints.

• Range — maximum minus minimum. It is easy to interpret but depends entirely on two observations and generally grows as sample size increases. In functions, it is the set of output values, contrasted with the domain.

• Coefficient of variation — standard deviation divided by the mean, giving relative dispersion for positive ratio-scale measurements. It is unstable near a zero mean and inappropriate for arbitrary-origin scales.

• Covariance — measures how two variables vary together in their original units. Its sign indicates direction, while its magnitude changes when either variable is rescaled. It is the mean of the product of two random variables minus the product of their means, and is zero for uncorrelated variables. It appears in statistical tests and models such as the Shapiro-Wilk test and capital asset pricing theory.

• Correlation — standardized covariance bounded between minus one and one for Pearson correlation. It measures linear association, can hide nonlinear structure, and does not establish causation.

  • Pearson correlation coefficient — It measures the strength of a linear relationship between two variables and is defined as the covariance divided by the product of their standard deviations. Its value ranges from -1 to 1, with 1 and -1 indicating a perfect positive or negative linear relationship. In simple linear regression, it is related to the slope through the ratio of standard deviations, and its square is the coefficient of determination, the variance explained. For ranked data, it is often replaced by Spearman’s correlation, and Fisher’s z-transformation maps it to an inverse hyperbolic tangent scale.

  • association measures

    • correlation coefficient — The correlation coefficient measures the strength and direction of association between two variables and ranges from negative one to one. Pearson’s r is the best-known example, and values of plus or minus one indicate perfect linear relationship.
• Box plot — displays a median, quartiles, whiskers, and flagged values under a chosen convention. It compresses a distribution and can conceal multimodality or small sample size.
  • Boxplots
    • 1.5 IQR rule
      • 1.5 — In statistics, 1.5 times the interquartile range is the standard boxplot cutoff used to extend whiskers and flag outliers. The same value also appears as the bond order of ozone and of each carbon-carbon bond in benzene; it is also the factor in ideal-gas average kinetic energy and occurs in rolling-motion problems.
• descriptive statistics

  • kurtosis — Kurtosis is a statistical measure based on the fourth standardized central moment. It describes the peakedness or tail-heaviness of a distribution, and normal distributions are often assigned a kurtosis of 0 or 3 depending on convention.

  • quantiles — Quantiles divide a distribution into equal-probability regions, with familiar examples including percentiles, quartiles, and medians. They are also central to inverse CDFs, quantile regression, and Q-Q plots used to compare distributions.

  • skewness and kurtosis — Skewness and kurtosis are measures of the shape of a probability distribution. Skewness describes asymmetry, while kurtosis describes how peaked or flat-topped a distribution is compared with a normal curve.

  • Central tendency
    • means — Means are measures of central tendency, with the arithmetic mean being the ordinary average. Other standard varieties include the geometric and harmonic means, and the geometric mean is the nth root of a product.
      • sample mean — The sample mean is the average of a set of sample values, found by adding the observations and dividing by their number. It is commonly written with a bar over the variable and is used as an estimator that converges to the true mean under large samples.
  • measures of dispersion
    • standard deviation — Standard deviation is a measure of spread equal to the square root of the variance. It appears in z- and t-scores, Chebyshev’s inequality, and the 68–95–99.7 rule for normal distributions.
• Data analysis
  • Outliers — Outliers are data points that lie far from the rest of a distribution and can distort measures such as correlation and regression. Malcolm Gladwell’s Outliers uses the term for unusually successful people and discusses the 10,000-Hour Rule.
• data visualization — Data visualization is the graphic display of data through charts, graphs, plots, and similar forms. Clues here point to Tufte’s anti-chartjunk advice, Wilkinson’s Grammar of Graphics, and software such as ggplot2, matplotlib, and Vega-Lite.

  • box plots — Box plots, or box-and-whiskers plots, summarize a distribution with the median, quartiles, and whiskers. Their whiskers are often drawn to extend 1.5 times the interquartile range beyond the box.

  • boxplot — A boxplot, or box-and-whisker plot, summarizes a data set with the five-number summary and highlights the interquartile range. Outliers are typically shown as separate points, and the plot is commonly associated with John Tukey.

  • Edward Tufte — Edward Tufte is a political scientist, statistician, and writer known for work on data visualization and information design. His books include The Visual Display of Quantitative Information and Beautiful Evidence. He emphasized data density and the data-to-ink ratio, coined terms such as chartjunk and data-ink ratio, and popularized sparklines for small inline graphs.
    • The Visual Display of Quantitative Information — The Visual Display of Quantitative Information is Edward Tufte’s classic book on data graphics and chart design. It criticizes chartjunk and praises clear statistical displays such as Minard’s famous visualization of Napoleon’s march into Russia.

  • histogram — A histogram is a bar graph that groups data into frequency bins using adjacent rectangles. In image processing, histograms describe pixel-intensity distributions and are used in equalization and matching.

  • histograms — A histogram displays data in bins, with bars showing the frequency or relative frequency in each interval. Common rules such as Freedman-Diaconis and Sturges help choose binning when constructing these graphs.

  • information graphics
    • Charles Joseph Minard — Charles Joseph Minard was a French engineer and mapmaker famous for his infographic of Napoleon’s disastrous 1812 Russian campaign. His visualizations are celebrated for combining geography, quantity, and motion into a single statistical graphic.
  • scatter plot — A scatter plot is a graph that displays paired data as individual points on x and y axes. It is often used to visualize correlation and to help judge whether a line of best fit or smoother curve is appropriate.

Probability and Distributions

Build from probability rules to random variables and common distributions.

Probability Foundations

• Events, Independence, and Conditioning

  • Sample space and event — the sample space contains possible outcomes; an event is a subset of those outcomes. A probability measure assigns values between zero and one while obeying normalization and additivity.

  • Complement rule — the probability that event A does not occur is one minus the probability that A occurs. This often simplifies calculations involving at least one event.

  • Union and intersection — a union contains outcomes in either event; an intersection contains outcomes shared by both. Addition requires subtracting the overlap unless the events are mutually exclusive.

  • Conditional probability — P(A given B) restricts attention to cases where B occurs. Dependence means P(A given B) differs from P(A); it does not by itself identify a causal relationship. It is central to Bayes’ theorem and posterior probabilities, which update beliefs using information about another event. In notation, it is often written with a vertical bar, as in P(A

    B).

  • Independence — events A and B are independent when P(A and B) equals P(A)P(B). Pairwise independence does not always imply mutual independence among three or more events. For independent events, the probability of their intersection equals the product of their probabilities; learning that one occurred does not change the probability of the other.
    • independence assumptions
      • independent AND identically distributed — Independent and identically distributed variables are observations whose values do not depend on one another and come from the same distribution. The abbreviation i.i.d. is standard in probability and statistics, though time series and other dependent data violate it.

  • Conditional independence — two variables may become independent after conditioning on a third. Conditioning can also create dependence, so the relevant causal or probabilistic structure must be stated.

  • Law of total probability — calculates an event’s probability by averaging its conditional probabilities over a mutually exclusive and exhaustive partition.

  • Bayes’ theorem — updates a probability using evidence: posterior odds equal prior odds multiplied by a likelihood ratio. The result depends on both the diagnostic or evidentiary signal and the prior prevalence. It is widely used in Bayesian inference, spam filtering, medical testing for false-positive rates, and classic problems such as Monty Hall.

  • probability

    • 3/8 — The probability in each problem is 3/8, or 0.375. If four fair coins are tossed, the probability of exactly two tails is 3/8.

    • mutually exclusive events

      • Mutually exclusive — Mutually exclusive events are events that cannot happen at the same time; equivalently, disjoint sets or events share no elements and have empty intersection. In probability, their intersection has probability zero, and the probability of their union is the sum of their individual probabilities.
  • probability theory

    • Andrey Andreyevich Markov — Russian mathematician known for work in probability theory and for the Markov chain, a stochastic process with the memoryless property in which the next state depends only on the current state. He taught at St. Petersburg University and is also associated with results involving the central limit theorem. His name appears in Markov’s inequality, Markov networks, hidden Markov models, the Viterbi algorithm, the Gauss–Markov theorem, and Monte Carlo methods built around Markov processes.

    • Central Limit Theorem or CLT — The Central Limit Theorem says that suitably normalized sums of independent, identically distributed random variables converge in distribution to a normal distribution. It underlies many standard statistical approximations, including the frequentist justification for confidence intervals.

    • moment generating function — The moment generating function of a random variable is the expectation of e to the tX. It is used to recover moments such as the mean and variance and appears in actuarial science and large deviation theory.

    • union bound — The union bound states that the probability of at least one event occurring is at most the sum of the probabilities of the individual events. It is also called Boole’s inequality and follows from the subadditivity of probability measures.

    • Andrei Nikolayevich Kolmogorov — Andrei Nikolayevich Kolmogorov was a foundational Russian mathematician in probability, dynamical systems, and information theory. His name appears in the Kolmogorov axioms, the Chapman-Kolmogorov equation, the Kolmogorov-Smirnov test, and the Kolmogorov-Arnold-Moser theorem.

    • Bernoulli trials — Bernoulli trials are repeated two-outcome experiments with a fixed probability of success on each trial. They underlie the binomial distribution and the geometric distribution.

    • certainty
      • 1 or 100% — This is the probability value of an event that is certain to happen. It is also the number that, when added to the probability of an event’s negation, gives a total of one.
    • cognitive biases
      • Law of Small Numbers — The law of small numbers is the mistaken belief that a small sample should closely match expected averages or random chance. It is often illustrated with the idea that short runs of data can look misleadingly representative, even when they are not.

    • moments — Moments are quantities formed by combining a physical quantity with distance, such as torque, bending moment, or moment of inertia. In mechanics they include torque and bending moments, and beam diagrams often plot bending moments alongside shear. In statistics and probability, moments describe distribution shape; the second moment is variance and the third moment is skewness.

    • probability distribution — A probability distribution assigns probabilities to the possible values of a random variable. Examples include the Poisson, gamma, and hypergeometric distributions.

    • probability paradoxes
      • Saint Petersburg paradox — The Saint Petersburg paradox is a probability puzzle in which a coin-tossing game has an infinite expected payoff. Its setup pays increasing rewards for later heads, making any finite entry fee seem favorable in expectation.
    • probability puzzles
      • The Monty Hall Problem — The Monty Hall problem is the probability puzzle in which a contestant should switch after one losing door is revealed. It is named for Monty Hall of Let’s Make a Deal and is famous for the counterintuitive 2/3 switching strategy.

    • random — Random describes a process or outcome that lacks a predictable pattern and may be modeled by probability distributions. In mathematics and computer science, random behavior appears in Monte Carlo methods, random walks, and pseudorandom number generation. Random sampling gives each member of a population an equal chance of selection.

    • random selection
      • Lottery — A lottery is a random selection system or game of chance in which a marked ticket or paper determines the winner; it is used for things like prizes, draft order, and land distribution. Shirley Jackson’s story “The Lottery” centers on Tessie Hutchinson being stoned after drawing the marked slip from the black box, and Borges’s “The Lottery in Babylon” uses a lottery as a governing device in a mythical Babylon.
    • random variables — Random variables assign numerical values to outcomes in a sample space and are described by a probability distribution. Their moments determine quantities such as expectation and variance, and they model uncertainty in events like coin flips and dice rolls.
      • discrete — Discrete describes objects or random variables that take countable, separated values rather than a continuum. It is the adjective for probability mass functions, discrete distributions, and topics such as combinatorics, graph theory, and the discrete Fourier transform. Discreteness is the property of being countable and separated rather than continuous.

    • randomness — Randomness is the unpredictable property central to random walks, Monte Carlo methods, and stochastic models of Brownian motion. It also underlies randomized algorithms and statistical tests for pseudo-random number generators.

    • sample spaces
      • sample space — A sample space is the set of all possible outcomes in a probability experiment. Events are subsets of that space, which is often denoted by the capital omega (Ω).
  • Probability interpretations
    • frequentism — Frequentism is the interpretation of probability that defines chance in terms of long-run relative frequency across repeated trials. It is often contrasted with Bayesian probability and associated with von Mises’s idea of collectives.
• Random Variables and Large-Sample Behavior

  • Random variable — a numerical function of a random outcome. Discrete variables take countable values; continuous variables are modeled over intervals and assign probability through areas under a density.

  • Expected value — a probability-weighted average over possible values. It describes a long-run or model-based center and need not be a value the variable can actually take. It is often written E(X); for a geometric distribution with parameter p, E(X)=1/p. For a continuous distribution it is computed as ∫ x f(x) dx, and it appears in formulas for variance, covariance, and Markov-type bounds.

  • Law of large numbers — under appropriate assumptions, a sample average converges toward its expectation as the number of observations grows. It does not say short sequences must resemble the theoretical distribution.
    • laws of large numbers

      • The Law of Large Numbers — The law of large numbers says that averages from many independent random trials tend to approach the expected value. It is closely associated with Bernoulli and is often contrasted with the mistaken idea that short-term outcomes change after a streak.

      • The Strong Law of Large Numbers — The strong law of large numbers states that the sample average of independent random variables converges almost surely to the expected value. It gives a rigorous guarantee that empirical means stabilize as the number of observations grows.

      • weak law of large numbers — The weak law of large numbers says that a sample mean tends to approach the population mean as the sample size increases. Chebyshev’s inequality can be used to prove this convergence statement.

  • Central limit theorem — under common regularity conditions, a standardized sum or mean approaches a normal distribution as sample size increases. The approximation can be poor for small samples, extreme skew, heavy tails, or dependence.

    • Lyapunov condition — The Lyapunov condition is a stronger version of the central limit theorem for independent random variables. It requires a bounded growth rate for the two-plus-delta moment relative to total variance, ensuring the usual normal-limit conclusion.

    • limit theorems

      • the central limit theorem — The central limit theorem says that sums of many independent random variables tend toward a normal distribution as the number of terms grows. Its modern form relaxes early identical-distribution assumptions, and results such as Berry-Esseen quantify the rate of convergence.
  • Likelihood — treats observed data as fixed and compares how strongly different parameter values or models would support them. It is not a probability distribution over parameters unless combined with additional assumptions.
    • likelihood functions — Likelihood functions describe the probability of observing data given model parameters, and their logarithms are often used in estimation. Their maxima define maximum likelihood estimation, and likelihood ratios underlie common hypothesis tests.

  • Exchangeability — a joint distribution is unchanged by specified permutations of unit labels. It can justify treating observations symmetrically but is stronger than having identical marginal distributions.

  • probability inequalities

    • Chebyshev’s theorem — Chebyshev’s theorem gives a lower bound on the fraction of data within a given number of standard deviations from the mean. In probability form, it implies that the chance of deviating by at least k standard deviations is at most one over k squared.

    • Markov’s inequality — Markov’s inequality gives an upper bound on the probability that a nonnegative random variable exceeds a given threshold. It is a basic tool in probability theory and underlies several other expectation-based tail bounds.

• Puzzles, Games, and Simulation
  • gambling
    • roulette — Roulette is the casino game played on a wheel with red and black numbered spaces and a green zero. Clues about betting systems, the Eudaemons, and the American and European versions point to this classic gambling game.
  • betting systems
    • Kelly criterion — The Kelly criterion is a betting formula for choosing the fraction of a bankroll to wager for maximum long-term growth. It is named for the Bell Labs engineer who developed it and is used in gambling and investing.
  • Geometric probability
    • Buffon’s Needle — Buffon’s needle is the classic probability experiment of dropping a needle onto parallel lines to estimate pi. It belongs to the family of geometric probability problems associated with the French naturalist Buffon.
  • probability experiments
    • Bernoulli trial — A Bernoulli trial is a single experiment with exactly two outcomes, usually called success and failure. Repeated Bernoulli trials underlie the binomial theorem and the binomial distribution.
  • Probability puzzles
    • Monty Hall problem — The Monty Hall problem is the famous conditional-probability puzzle in which switching doors on a game show improves the odds of winning. It is named for the host of Let’s Make a Deal and is a standard application of Bayes’ theorem.
  • random variate generation
    • Sampling from a distribution — Sampling from a distribution is the task of generating values according to a target probability law, often by methods such as Gibbs sampling or Metropolis-Hastings. It is central to MCMC and related Bayesian methods, including importance sampling and Hamiltonian-based approaches.
  • randomness and simulation

    • pseudo-random numbers — Pseudo-random numbers are values generated by an algorithm to mimic randomness for tasks like simulation and cryptography. Methods such as the Mersenne Twister and middle-square method produce sequences that are statistically distributed but ultimately deterministic from a seed.

    • pseudorandomness — Pseudorandomness is the appearance of randomness produced by a deterministic process. It underlies seeded generators such as the Mersenne Twister and many Monte Carlo simulations.

      • random numbers — Random numbers are values chosen by chance and used in statistics, simulation, and computing. Their pseudorandom versions are generated from a seed so they can imitate unpredictability while remaining reproducible.
  • simulation methods
    • Monte Carlo methods — Monte Carlo methods are randomized numerical algorithms that use repeated sampling or random numbers to approximate quantities, often with deterministic runtime but random output. They are used especially in numerical integration, simulation, optimization, variance reduction, probabilistic search, and statistical physics, and can handle high-dimensional problems that are difficult for deterministic techniques. Important variants and examples include Markov chain Monte Carlo, Metropolis-Hastings, simulated annealing, and Karger’s algorithm. The name comes from the casino in Monaco. They are contrasted with Las Vegas algorithms.
      • Monte Carlo simulation — Monte Carlo simulation is a computational method that uses random sampling to estimate mathematical or physical outcomes. It is especially useful for probability problems and is named for the gambling center in Monaco.

Common Probability Distributions

• probability distributions — A probability distribution assigns probabilities to outcomes or values of a random variable, whether discrete or continuous. Examples include the Bernoulli, Poisson, binomial, and Gaussian distributions, and they underlie tools such as Q-Q plots and the normal distribution.

  • Benford’s law — Benford’s law says that the leading digits in many real-world data sets are highly nonuniform, with 1 appearing far more often than 9. It is often used as a check for forged or manipulated numerical data.

  • Bimodal — A bimodal distribution has two distinct peaks, so the data cluster around two separate values rather than one central mode. It can appear when a population splits into two groups with different underlying causes or tendencies.

  • Cauchy distribution — The Cauchy distribution is the heavy-tailed probability distribution whose standard form is proportional to 1 over 1 plus x squared. It is equivalent to a t-distribution with one degree of freedom, is also called the Witch of Agnesi, and has undefined mean and variance.

  • chi-squared distribution — The chi-squared distribution is the probability distribution of a sum of squared standard normal variables and a special case of the gamma distribution. It underlies Pearson’s tests for goodness of fit and independence in categorical data, appears in the asymptotic theory behind likelihood-ratio tests, and defines the F-distribution through a ratio of independent variables.

    • central chi-squared distribution — The central chi-squared distribution describes the sum of squares of standard normal variables and is determined by degrees of freedom. It appears in goodness-of-fit tests, contingency-table work, and Wilks’s theorem for likelihood-ratio tests.

    • Chi-squared — The chi-squared distribution is used in statistics to model the sampling distribution of variance and to test hypotheses. It is also the limiting distribution that appears in many likelihood-ratio tests.

  • continuous distributions
    • exponential distribution — The exponential distribution is the memoryless continuous distribution with mean equal to the reciprocal of its rate parameter. It models waiting times such as telephone calls, machine repairs, or the time until the first event in a Poisson process.
  • density functions
    • probability density function — The probability density function describes how probability is distributed across values of a continuous random variable, and its integral over an interval gives the probability of landing in that range. It is normalized to integrate to 1 and can be estimated with methods such as kernel density estimation or histogram-based smoothing.

  • Dirichlet distribution — The Dirichlet distribution is a probability distribution over a simplex and the conjugate prior of the multinomial distribution. It underlies Bayesian models such as latent Dirichlet allocation and connects to the Chinese restaurant process and Dirichlet processes.

  • discrete distributions
    • Bernoulli random variable — A Bernoulli random variable is a two-outcome random variable that takes values 0 and 1. Its distribution is determined by a success probability p and failure probability 1-p.
  • distribution functions
    • cumulative distribution function — The cumulative distribution function gives the probability that a random variable is less than or equal to a given value. It is usually written as a capital F and is obtained by integrating a density or mass function.

  • Erlang Distribution — The Erlang distribution is a probability distribution used in queueing theory and telephony, with shape and rate parameters. It is also the distribution of the waiting time for the hth event in a Poisson process, and the gamma distribution is a special case of it.

  • F distribution — The F distribution is a continuous probability distribution formed from the ratio of two chi-square variables normalized by their degrees of freedom. It is the null distribution for the ANOVA test statistic that compares between-group and within-group variance.

  • Laplace distribution — The Laplace distribution is a sharp-peaked probability distribution with a double-exponential shape. It generalizes the Gaussian and is often used to model data with a narrow mode.

  • memorylessness — Memorylessness is the property of a distribution whose future probabilities depend only on the present state, not on the elapsed past. The exponential and geometric distributions have this property, which is also the probabilistic Markov property in continuous form.

  • Pareto distribution — The Pareto distribution is a heavy-tailed probability distribution associated with the idea that a small fraction of causes can account for a large fraction of outcomes. It is named for Vilfredo Pareto and appears in applications ranging from wealth to web traffic.

  • species abundance distributions
    • Fisher’s logarithm series — Fisher’s logarithmic series is a species-abundance distribution used in ecology. It has the form that the number of species with abundance n is proportional to alpha times x to the n over n.
  • Weibull distribution — The Weibull distribution is a flexible family of continuous distributions used heavily in reliability analysis and survival modeling. It can specialize to exponential or Rayleigh forms, and it has also been used to derive Pythagorean expectation in baseball analytics.
• Discrete Distributions

  • Bernoulli distribution — models one binary trial with success probability p. Its expectation is p and its variance is p(1-p).

  • Binomial distribution — counts successes across a fixed number of independent Bernoulli trials with a common success probability. Overdispersion signals that these assumptions may not hold. Its coefficients come from combinations of n choose k, and for large n it approaches a normal distribution while rare-event limits lead toward the Poisson distribution.
    • Binomial Probability Distribution — The binomial probability distribution models repeated trials with two outcomes, where each trial has the same success probability p. Its form is used for counting successes across identical trials and is one of the standard discrete probability distributions.
  • Poisson distribution — models event counts under a constant-rate independent-event process, with equal mean and variance. Negative-binomial models can accommodate extra variation. It has one parameter, λ, and is a discrete probability distribution. It is a limiting case of the binomial distribution when the number of trials is large and the success probability is small. Common applications include arrivals, radioactive decay counts, calls to a call center, and other event-counting processes.
    • Poisson — The Poisson distribution models independent events that occur at a constant average rate over time or space. Its variance equals its mean, which is the signature feature used to identify it.

  • Geometric distribution — counts trials until the first success under independent Bernoulli trials with constant success probability. Like the exponential distribution, it has a memoryless property. It gives the probability that the first success occurs on a given Bernoulli trial; its mean is 1/p. It is used to model waiting time until success or the duration of a Markov chain.

  • Negative-binomial distribution — models counts with variance exceeding the mean or, under another parameterization, trials until a fixed number of successes. The chosen parameterization must be reported.

  • Hypergeometric distribution — counts successes when sampling without replacement from a finite population, making draws dependent rather than independent. It appears in card-drawing problems, acceptance sampling, and ecological mark-recapture, and has noncentral variants named for Fisher and Wallenius.
• Continuous Distributions and Reference Distributions

  • Uniform distribution — assigns equal density across a continuous interval, or equal probability across a finite set. Equal density does not mean every exact continuous value has positive probability. It is the maximum-entropy choice for bounded support. The standard continuous form is on [0,1], with mean 1/2 and variance 1/12. It appears in contexts such as p-values, the German tank problem, and order statistics.

  • Normal distribution — a symmetric bell-shaped family determined by mean and variance. Normality is a model assumption or approximation, not a universal property of observed data.

    • Empirical rule — The empirical rule describes the spread of data in a normal distribution: about 68 percent lies within one standard deviation of the mean, 95 percent within two, and 99 percent within three. It is a convenient approximation used for bell-shaped data sets.

    • Gaussian — A Gaussian distribution is the familiar bell-shaped normal distribution centered on a mean. In physics and chemistry, Gaussian functions appear in wavefunctions and basis sets, and the name also appears in Gaussian random matrices.

    • Standard normal distribution — The standard normal distribution is the normal distribution with mean zero and variance one. It is the centered, unit-variance special case of the Gaussian family.

  • t distribution — a symmetric, heavier-tailed family used when a normal mean is standardized using an estimated standard deviation. Its degrees of freedom determine tail thickness. It is commonly used for small samples when the population standard deviation is unknown, especially in t-tests and Welch’s corrections. The name is tied to William Sealy Gosset’s pseudonym, Student.

  • Chi-square distribution — the distribution of a sum of squared independent standard normal variables. It appears in variance inference, contingency-table tests, and likelihood approximations. It is a gamma-family distribution determined by degrees of freedom. It is commonly used in goodness-of-fit tests and contingency-table tests, and it underlies several standard hypothesis-testing procedures.

  • Exponential and survival distributions — the exponential distribution models a constant event hazard and is memoryless. Real survival processes often require hazards that change with time or covariates.

  • Beta distribution — a flexible family on zero to one, often used for probabilities and proportions or as a prior for a Bernoulli probability.

  • Gamma distribution — a positive continuous family used for waiting times, rates, and skewed quantities. The exponential distribution is a special case under common parameterizations. Its name comes from the gamma function, the integral that generalizes the factorial to complex numbers.

  • normality — Normality is the statistical condition of following a normal, or Gaussian, distribution, often pictured as a bell curve. In chemistry, normality is also a concentration unit based on reactive equivalents per liter, distinct from molarity but related to it by an equivalence factor.

  • continuous probability distributions
    • continuous uniform distribution — The continuous uniform distribution has constant density on a finite interval and linear cumulative distribution function. Its variance is one-twelfth of the square of its support length, and it is the maximum-entropy choice when only the support is fixed. It appears in the German tank problem, the Box-Muller transform, and the study of random matrices and Legendre polynomials.

Sampling and Study Design

Connect sampling and design decisions to what a study can establish.

Sampling and Measurement

• Selecting a Sample
  • sampling — Sampling is the statistical process of selecting a subset of individuals from a population, whether with or without replacement. Related methods include simple random, stratified, clustered, and importance sampling, along with bootstrapping and Monte Carlo techniques that repeatedly sample to estimate distributions. The same term also names a music-production technique that takes fragments of earlier recordings and reuses them in new works. In pop-culture clues, this appears in hip-hop, plunderphonics, and albums by artists such as DJ Shadow and the Avalanches.

    • Aliasing — Aliasing is a sampling artifact in which a signal is sampled too slowly or below the Nyquist rate, causing different frequencies to become indistinguishable and producing false or distorted signals during reconstruction. It is especially familiar in digital audio and other sampled systems. It can produce artifacts such as moiré patterns and the wagon-wheel effect, and is often reduced with low-pass filtering or other anti-aliasing methods.

    • Sample — A sample is a subset of a population used to represent the whole in statistics, polling, and demographic study. Related clues may mention sampling methods, sample size, and sampling distributions or estimators.

  • Simple random sample — every sample of a given size has the same selection probability under the classical design. Random sampling supports population generalization; random treatment assignment supports causal comparison.

  • Stratified sampling — samples within prespecified subgroups, often improving precision or ensuring representation. Survey weights must reflect selection probabilities and any later adjustments.

  • Cluster sampling — samples naturally grouped units such as schools or neighborhoods. Within-cluster similarity reduces effective information relative to the same number of independent observations.

  • Convenience sample — recruits units because they are accessible. Precision within the observed sample does not repair unknown selection differences from the target population.

  • Sampling frame — the operational list or mechanism from which units can be selected. A perfect random draw from a flawed frame still misses uncovered members of the target population.

  • Coverage error — mismatch between the sampling frame and target population through undercoverage, overcoverage, duplication, or ineligible units.

  • Unit nonresponse — a sampled unit provides no usable response; item nonresponse leaves particular variables missing. Weighting can reduce bias only when response mechanisms are modeled adequately.

  • Survey weight — represents how many target-population units an observation stands for after accounting for selection probabilities and possible nonresponse or calibration adjustments.

  • Finite population correction — reduces sampling variance when a substantial fraction of a finite population is sampled without replacement. It is irrelevant for many superpopulation or process-based targets.

  • census — A census is the periodic count of a population, most famously carried out in the United States every ten years. Countries use it to measure residents and households, and it is used to apportion representation and has been associated with early statistical work and population administration.

  • Public opinion polling
    • Gallup — Gallup is the opinion-polling company and family name associated with the city in New Mexico. The polling firm became famous for correctly predicting Franklin Roosevelt’s 1936 landslide and for conducting exit polls.
  • Surveys
    • Exit poll — An exit poll surveys voters as they leave polling places, usually to give media early estimates of election results. They are typically held until after polls close to avoid influencing voters.
• Measurement and Missing Observations

  • Measurement validity — the degree to which evidence supports the intended interpretation and use of a measure. It is not a permanent property of an instrument detached from population and context.

  • Reliability — consistency of measurement across repetitions, raters, items, or occasions. Reliability is necessary for many uses but does not guarantee that the intended construct is measured.

  • Measurement error — the difference between a recorded value and the target quantity. Nondifferential error can bias estimates toward or away from a null depending on the model; it does not universally attenuate effects.

  • Missing data — values may be missing completely at random, conditionally at random given observed information, or through mechanisms involving unobserved values. Complete-case analysis is valid only under assumptions that must be defended.

  • Response bias — systematic difference between recorded answers and target values caused by wording, recall, social desirability, mode, interviewer behavior, or other measurement conditions.

  • measurement and accuracy
    • accuracy — Accuracy is the property of a measurement or instrument being close to the true or correct value.
  • psychometrics
    • item response theory — Item response theory is a psychometric framework that models an examinee’s latent ability through the behavior of individual test items. It is used for adaptive testing and item calibration, often through logistic models with parameters for difficulty, discrimination, and guessing.
  • Measurement theory
    • precision — Precision is the degree to which repeated measurements agree with one another. In science it contrasts with accuracy, which concerns closeness to the true value.
  • research methods

    • citizen science — Citizen science is research in which amateurs help gather or analyze data for scientific projects. Examples include the Christmas Bird Count, Skywarn, CoCoRaHS, and online platforms such as Zooniverse.

    • quantitative research — Quantitative research emphasizes numerical data, empirical observation, and statistical analysis. It is often contrasted with qualitative research and is commonly used in social science studies and data visualization workflows.

Survey Sampling and Polling

• Media measurement
  • Nielsen Media Research — Nielsen Media Research is the company best known for measuring television audiences in the United States. Its ratings systems, including the people meter, became a standard way to track TV viewership and popularity.
• Nathaniel Read Silver
  • Nathaniel Read “Nate” Silver — Nathaniel Read “Nate” Silver is a statistician and analyst best known for founding FiveThirtyEight and for electoral forecasting, especially his accurate prediction of the 2012 U.S. presidential election. He became widely famous for election prediction and later wrote The Signal and the Noise, on uncertainty, probability, and forecasting.
• Polling bias
  • Bradley effect — The Bradley effect is the supposed polling gap in which voters give pollsters socially desirable answers, especially about racial prejudice. It was named after California gubernatorial candidate Tom Bradley and is sometimes linked to the phrase Wilder effect.
• Polling methods
  • tracking poll — A tracking poll is a campaign poll updated repeatedly with small samples over time. It is used to follow shifts in voter preference near the end of an election.
• Pollsters
  • George Horace Gallup — American statistician and pollster who popularized scientific public-opinion polling in the United States. He founded the American Institute of Public Opinion and is best known for accurately predicting Franklin Roosevelt’s 1936 victory over Alf Landon using sampling methods more reliable than the Literary Digest’s.
• survey research organizations
  • Pew Research Center — The Pew Research Center is a Washington, D.C.-based “fact tank” that publishes surveys, public-opinion polling, and demographic reports. Its projects include the Global Attitudes Project and the Forum on Religion and Public Life, and its work often tracks American partisanship and generational attitudes.

Study Design and Causal Inference

• Experiments and Observational Designs

  • Randomized experiment — assigns interventions by a random mechanism, balancing measured and unmeasured baseline factors in expectation. Nonadherence, attrition, interference, and missing outcomes can still complicate interpretation.

  • Observational study — observes exposures without investigator assignment. Causal interpretation requires assumptions about exchangeability, positivity, consistency, measurement, and the modeled data structure.

  • Experimental design

    • control group — A control group is the part of an experiment that does not receive the treatment being tested. It provides the baseline for comparison in clinical trials, randomized experiments, and methods such as difference-in-differences or placebo-controlled studies.

    • control groups — Control groups are the comparison groups in experiments and clinical trials, often receiving no treatment or a placebo. They provide the baseline against which the treatment group is measured, helping scientists isolate the effect of the independent variable.

    • double-blind — A double-blind experiment is one in which neither the participants nor the administrators know who is in the control or experimental group. This design is used to reduce bias in research studies.

    • double-blind experiment — A double-blind experiment is one in which neither the researcher nor the participant knows who is in the control or treatment group. The design is used to reduce bias in testing procedures.

    • experiments — Experiments are controlled procedures used in the social sciences to test hypotheses and estimate causal effects. They can be double-blind, randomized, and blocked or stratified, and they are the setting of classic studies by Milgram and Zimbardo.

    • randomization — Randomization is the process of assigning subjects by chance, especially in randomized controlled trials. It is central to causal inference methods in economics and medicine and is often paired with blinding to reduce bias.

  • Research design
    • Quasi-experimental design — A quasi-experimental design is a research model that compares groups or time periods without full random assignment. Common forms include time-series designs, multiple time-series designs, and nonequivalent control-group designs.
  • study designs
    • longitudinal studies — Longitudinal studies follow the same subjects or cohort over time rather than comparing different groups at one moment. They are used in panel surveys, developmental research, and other studies that track change across years.
• Causal Targets, Assumptions, and Bias
  • Causal inference

    • confounding variables — Confounding variables are hidden factors that blur the relationship between an independent variable and a response variable. They are a classic obstacle to causal inference because they can create spurious associations.

    • regression discontinuity design — Regression discontinuity design is a quasi-experimental method that estimates causal effects by comparing observations just above and below a cutoff. It can be sharp or fuzzy, and it was developed to handle cases where randomization is impossible.

    • difference-in-differences — Difference-in-differences is a causal inference method that compares outcome changes in a treated group with changes in an untreated control group. Its interpretation relies on a parallel-trends assumption, and it is commonly used in observational studies such as Card and Krueger’s minimum-wage analysis.

    • Local average treatment effect — The local average treatment effect is the treatment effect for compliers, the people whose treatment status changes only when their assignment changes. It is often abbreviated LATE and appears in instrumental-variables estimation.

  • Potential outcomes — define causal effects by comparing outcomes a unit would have under different interventions. Because only one is observed for each unit at a time, identification requires design and assumptions.

  • Average treatment effect — the population mean difference between potential outcomes under two specified treatment strategies. The population, treatment versions, outcome, and time horizon are part of the estimand.

  • Confounding — mixing of an exposure-outcome contrast with effects of other causes of the outcome that differ across exposure groups. A variable can be a confounder, mediator, collider, or none depending on the causal question.

  • Directed acyclic graph — represents assumed causal relationships and helps identify adjustment sets. It makes assumptions visible but cannot prove that the graph is correct.

  • Selection bias — conditioning on study entry, retention, measurement, or analysis can create associations that differ from the target population relationship.

  • Conditional exchangeability — after adjustment for a sufficient set of pre-exposure variables, treatment groups have comparable potential-outcome distributions. In observational studies this is an untestable causal assumption.

  • Positivity — every covariate pattern relevant to the target comparison has a nonzero probability of receiving each treatment strategy. Sparse practical overlap can make estimates unstable before exact violations occur.

  • Causal consistency — the observed outcome under the treatment actually received equals the corresponding potential outcome, requiring interventions and their relevant versions to be defined well enough for the comparison.

  • Intention-to-treat effect — compares groups according to randomized assignment regardless of subsequent adherence. It preserves the randomized contrast and estimates the effect of assignment, not necessarily treatment received.

  • paradoxes
    • Simpson paradox — The Simpson paradox is the reversal in aggregated trends that can occur when data are partitioned into subgroups. It is often illustrated by comparing items whose quality changes the apparent cost-benefit relationship, such as more expensive groceries that reduce spoilage.
• Design-Based Strategies and Emulated Trials

  • Difference in differences — compares outcome changes across exposed and comparison groups. Causal interpretation typically relies on a parallel-trends assumption and absence of differential concurrent shocks.

  • Instrumental variable — uses a variable that shifts exposure, affects the outcome only through that exposure, and is independent of relevant unmeasured causes under the stated design. These assumptions are strong and partly untestable. An instrumental variable is a predictor used in econometrics to identify causal effects when ordinary regression would be biased by endogeneity. It is often estimated with two-stage least squares.

  • Regression discontinuity — exploits treatment assignment around a threshold. It estimates a local effect near the cutoff when units cannot precisely manipulate assignment and potential outcomes vary smoothly there.

  • Target trial emulation — specifies the eligibility criteria, treatment strategies, assignment procedure, follow-up, outcomes, causal contrast, and analysis that an observational study seeks to emulate.

Estimation and Hypothesis Testing

Distinguish estimation, uncertainty, and hypothesis testing.

Estimation and Uncertainty

• Estimators and Their Properties

  • Point estimator — a rule that maps sample data to one estimate, such as a sample mean or maximum-likelihood estimate. Estimators are assessed by bias, variance, consistency, robustness, and the loss relevant to the task.

  • Sampling distribution — the distribution an estimator would have across repeated samples from the assumed design or model. Standard errors summarize its spread.

  • Estimator bias — the difference between an estimator’s expected value and its target under a specified model or design. Low bias alone does not ensure low total error.

  • Consistency — an estimator converges toward its target as information grows under stated assumptions. A consistent estimator can still perform poorly in realistic finite samples.

  • Efficiency — compares estimator variability for the same target under a model or design. Greater model-based efficiency can come at the cost of fragility when assumptions fail.

  • Robust estimator — limits sensitivity to departures such as outliers or distributional misspecification. Robustness is always relative to particular deviations and may trade efficiency under the idealized model.

  • Statistical estimation
    • estimators — Estimators are statistics used to approximate an unknown population parameter. Common examples include maximum-likelihood estimators and unbiased estimators, whose precision is often compared using the Cramér–Rao bound.
      • unbiased — An unbiased estimator has expectation equal to the true parameter value. The sample variance uses n minus one in its denominator to preserve this property, and the Gauss-Markov theorem identifies BLUE estimators with it in the linear case.
    • EM — EM, or expectation-maximization, is an iterative method for fitting models with latent variables by alternating between an expectation step and a maximization step. It is widely used for Gaussian mixtures, hidden Markov models, and other maximum-likelihood estimation problems.
  • estimation
    • maximum likelihood estimation — Maximum likelihood estimation is a statistical method for choosing parameter values that make the observed data most probable. It uses the likelihood function, often in logarithmic form, and is closely associated with Fisher information and asymptotic efficiency. It is widely used in frequentist inference and in phylogenetic reconstruction from DNA sequences.
  • Bias — Bias is a systematic deviation, especially the difference between an estimator’s expected value and the true parameter. In statistics it appears in topics like unbiased estimation, selection bias, and the bias-variance tradeoff; in general usage it also means prejudice or slant.
    • survivorship bias — Survivorship bias is the error of focusing on the group that made it through a selection process while ignoring those filtered out. It can distort studies by comparing the general population to only the people who remained, stayed successful, or completed a process.
  • model error
    • bias AND variance — In statistics, bias and variance are two major sources of model error and are often discussed as a trade-off. Bias measures systematic deviation from the true value, while variance measures spread around the estimator’s expectation.
• Intervals, Resampling, and Likelihood
  • Confidence interval — a procedure that captures the fixed target parameter at its nominal rate across repeated compatible samples. After one interval is observed, the classical parameter is not assigned a probability by that procedure.
    • interval estimation
      • confidence intervals — A confidence interval is a range of values built from sample data that is intended to contain the true parameter with a stated probability such as 95%. Frequentist confidence intervals contrast with Bayesian credible intervals, though both express uncertainty around an estimate.

  • Margin of error — a critical value multiplied by an estimated standard error in many standard intervals. It accounts for sampling variation under the design, not all forms of bias or measurement error.

  • Bootstrap — approximates an estimator’s sampling behavior by repeatedly resampling observed units and recomputing it. The resampling unit must respect clustering, repeated measures, time dependence, and the original design.
    • resampling methods
      • bootstrapping — Bootstrapping is a resampling technique that estimates sampling uncertainty by repeatedly sampling with replacement from a dataset. It was developed by Bradley Efron and has variants such as the wild bootstrap and block bootstrap, with applications in statistics, phylogenetics, and computer systems.
  • Maximum likelihood — chooses parameter values that make the observed data most probable under a specified model. Likelihood measures relative support within that model and is not the probability that a parameter is true.
    • Likelihood-based inference
      • likelihood function — A likelihood function gives the probability of observed data as a function of model parameters. It is central to maximum likelihood estimation and to likelihood-ratio tests such as those in Wilks’ theorem.
  • Statistical inference

    • degrees of freedom — Degrees of freedom are the independent parameters a system can vary, whether in mechanics, thermodynamics, or statistics. They determine phase-rule counts, appear in chi-squared calculations, and underlie ideas like normal modes and gimbal lock.

    • information measures

      • Fisher information — Fisher information measures how much an observation tells you about an unknown parameter. It equals the variance of the score function and appears in the Cramér-Rao lower bound.
• Bayesian Updating and Prediction
  • Bayesian statistics — Bayesian statistics is an approach to inference that combines prior beliefs with observed data to produce a posterior distribution. It is based on Bayes’ theorem and is often contrasted with frequentist methods.

    • prior distribution — A prior distribution in Bayesian statistics represents beliefs before the data are observed. It combines with the likelihood to produce the posterior, and special cases include conjugate priors, spike-and-slab priors, and empirical Bayes estimates.

    • Thomas Bayes — Thomas Bayes was the 18th-century English statistician and minister whose theorem updates probabilities using new evidence. His name is attached to Bayesian statistics, Bayesian networks, and the contrast with frequentist probability, as well as the Bayes factor and related subjectivist interpretations of probability. Quizbowl clues often invoke Monty Hall, priors, posteriors, and conditional probability.

    • Bayesian inference

      • Posterior probability — Posterior probability is the updated probability of a hypothesis after evidence has been taken into account. In Bayesian inference, it is proportional to the prior times the likelihood.

      • prior probability — A prior probability is the initial probability assigned to a variable before observing new data. In Bayesian inference, it combines with the likelihood to produce the posterior probability. The term is also used in Bayesian persuasion and in base-rate reasoning.

        • prior probability distribution — A prior probability distribution encodes assumptions about an unknown quantity before observing data in Bayesian inference. It combines with the likelihood to produce the posterior, and special cases include objective and Jeffreys priors.

  • Bayesian posterior — combines a prior distribution with a likelihood to obtain a probability distribution over parameters or predictions. Posterior validity depends on the likelihood, prior, computation, and whether the model represents the problem adequately.

  • Credible interval — a Bayesian interval containing a stated posterior probability for a parameter. Its interpretation depends on the prior, likelihood, and computation used to form the posterior.

  • Posterior predictive distribution — averages predictions over posterior parameter uncertainty to describe future or replicated data under the model. It supports prediction and model checking, not automatic causal interpretation.

Hypothesis Tests and Error Rates

• Tests, Decisions, and Power

  • Null and alternative hypotheses — formal claims used to define a test statistic and its reference distribution. A test should be tied to a scientific question and effect scale rather than treated as an automatic ritual.

  • hypothesis — A hypothesis is a proposed explanation or statement that can be tested by observation or experiment. In scientific and statistical contexts, it is the statement being evaluated against evidence, often contrasted with a null hypothesis; in statistics and experimental design, hypotheses are framed as null and alternative statements and evaluated by significance testing.

  • Test statistic — a data summary chosen so unusually large, small, or extreme values count against a null model. Its reference distribution must match the design and assumptions.

  • p-value — under the null model and its assumptions, the probability of a test statistic at least as extreme as the observed one. It is not the probability that the null hypothesis is true. It is the statistic targeted by significance testing and multiple-comparisons corrections, and is often compared to the familiar 0.05 threshold.

  • Significance threshold — a prespecified rule for rejecting a null model, often expressed as alpha. It controls a long-run testing error under assumptions rather than separating important from unimportant effects.

  • One-sided and two-sided tests — a one-sided alternative specifies direction before seeing results; a two-sided alternative permits either direction. Choosing direction afterward invalidates the advertised error rate.

  • Type I error — rejection of a true null under the testing framework. The significance level controls its probability for a prespecified test when the model and procedure hold.

  • Type II error and power — a Type II error fails to reject a specified false null; power is the probability of rejection under a specified alternative. Power varies with effect size, noise, sample size, design, and analysis.

  • Effect size — a quantitative measure of magnitude, such as a mean difference, standardized difference, risk ratio, odds ratio, or correlation. Practical meaning depends on the scale, baseline risk, uncertainty, and context.

  • Equivalence and noninferiority — specialized designs that test whether effects lie within prespecified margins. Failure to find a conventional difference does not establish equivalence.

  • Optional stopping — repeatedly testing accumulated data and stopping when a threshold is crossed inflates ordinary fixed-sample error rates unless a sequential design accounts for the monitoring rule.

  • statistical hypothesis testing
    • hypothesis testing — Hypothesis testing is the statistical procedure of comparing a null hypothesis against an alternative using a test statistic and p-value. Common examples include z-tests, t-tests, ANOVA, and multiple-comparison corrections such as Bonferroni or Benjamini-Hochberg control.

      • chi-squared test — The chi-squared test compares observed and expected frequencies to assess goodness of fit or independence. Karl Pearson introduced it, and it is named for the chi distribution family.

      • errors — Errors are incorrect values or outcomes that occur in data transmission or statistical inference. In communications they are handled by error-detecting or error-correcting codes, and in hypothesis testing they include type I and type II mistakes.

      • F-statistic — The F statistic is the ratio used in ANOVA to compare variance between groups with variance within groups. It is also used in regression and instrumental-variables testing as a measure of overall model significance or instrument strength.

      • F-test — The F-test is a statistical test based on the F distribution, commonly used to compare variances or assess model fit. It underlies ANOVA and related procedures such as the Chow test.

      • Kolmogorov-Smirnov test — The Kolmogorov-Smirnov test is a nonparametric goodness-of-fit test that compares an empirical distribution function to a reference distribution. Its statistic is the maximum distance between the two cumulative distribution functions.

      • null hypothesis — The null hypothesis is the statistical claim that there is no relationship or no difference in the population being tested. It is the statement researchers fail to reject when a p-value is not small enough, and it is the reference point for type I error in significance testing.

      • p-values — A p-value measures how surprising a result would be if the null hypothesis were true, and it is commonly compared to a significance threshold such as 0.05. Clues often connect p-values to Fisher’s method, Bonferroni correction, and the practice of p-hacking or data dredging.

      • statistical power — Statistical power is the probability that a test rejects the null hypothesis when the null is false. It equals one minus the type II error rate and increases with effect size, sample size, and significance level.

      • statistical significance — Statistical significance means a result is unlikely under the null hypothesis, usually judged by comparing a p-value to a preset alpha level. The idea is central to hypothesis testing, and recent debates have urged lowering the conventional threshold for significance.

      • t-test — A statistical hypothesis test based on the t distribution, used to compare means or test whether a sample mean differs from a population mean, especially when the population standard deviation is unknown and sample sizes are small. It was first published by William Sealy Gosset under the name Student while working at Guinness. It has paired and unpaired forms; Welch’s version relaxes the equal-variance assumption. Nonparametric alternatives include the Mann-Whitney U test and the Wilcoxon signed-rank test.

      • Type II error — A type II error occurs when a false null hypothesis is not rejected in statistical testing. Its complement in hypothesis-testing language is power, and it is the error avoided by an ideal test with high sensitivity.

    • hypothesis tests — Hypothesis tests compare null and alternative models to decide whether data are statistically significant. Common details include p-values, alpha levels, Type II error, and corrections such as Bonferroni and Holm procedures.
  • statistical hypothesis tests
    • goodness-of-fit tests
      • Pearson’s chi-squared test — Pearson’s chi-squared test is a statistical goodness-of-fit test for categorical data. It compares observed and expected frequencies, and it is also used to test independence and Hardy-Weinberg equilibrium.

  • error — Error is the difference between an expected value and an actual value, and numerical methods aim to reduce it through more accurate approximation. In statistics, type I and type II errors are false positives and false negatives.

  • statistical inference and error
    • statistical error — Statistical error is the discrepancy between observed and expected values in a model or experiment. It includes concepts such as residual variation, standard error, and the type I error rate controlled by procedures like the Bonferroni correction.
• Multiple Testing and Common Procedures
  • Multiple testing — examining many hypotheses increases opportunities for false positive findings. Family-wise error and false-discovery-rate procedures control different targets and require a defined family of tests.
    • Multiple comparisons correction
      • Bonferroni correction — The Bonferroni correction adjusts for multiple comparisons by dividing the significance threshold by the number of tests. It is used to reduce false positives when several hypotheses are tested at once.

  • Family-wise error rate — probability of at least one false rejection within a defined family of tests. Bonferroni control is simple and valid under broad dependence but can be conservative.

  • False discovery rate — expected proportion of false rejections among all rejections under the procedure’s conditions. It differs from the probability that any one reported finding is false.

  • Permutation test — builds a reference distribution by rearranging labels or outcomes under an exchangeability condition supplied by the design or null hypothesis.

  • analysis of variance — Analysis of variance, or ANOVA, is a statistical method used to compare the means of multiple samples by using the F statistic to separate between-group from within-group variation. It is a generalization of the t-test for comparing more than two group means; for two samples, it reduces to the usual Student t-test. Common variants include one-way, two-way, repeated-measures, and multivariate forms, and it is often followed by post-hoc tests such as Tukey’s HSD.

  • Chi-square test
    • chi-square — The chi-square statistic compares observed and expected counts to test goodness of fit or independence. It is named for the Greek letter chi and is commonly used with contingency tables.

Regression, Multivariate Methods, and Prediction

Connect models to their assumptions, interpretation, and predictive evaluation.

Regression and Statistical Models

• Regression Components and Model Families
  • regression analysis — Regression analysis is a statistical technique for modeling relationships between variables, often with a best-fit line in linear form. Clues include least squares, the Gauss-Markov theorem, ridge regression, logistic regression, and the coefficient of determination.

    • endogeneity — Endogeneity is the problem that arises when an explanatory variable is correlated with the error term in a regression model. It is often caused by omitted variables, measurement error, or reverse causation.

    • least squares — Least squares is a regression method that chooses parameters by minimizing residual error, and it is the standard fitting technique in linear models. It is used throughout statistics and numerical analysis, including curve fitting and weighted regression.

    • LEAST-SQUARES_ Method — The least-squares method fits a curve or line to measurements by minimizing the sum of squared deviations from the data. Gauss developed it and used it in work that helped recover the orbit of Ceres.

    • linear regression analysis — Linear regression analysis models the relationship between one response variable and one or more input variables. Its most common form is ordinary least squares, which fits a line of best fit to data.

    • Method of least squares — The method of least squares is a regression procedure that chooses parameters to minimize the sum of squared residuals. It is associated with Gauss and Legendre and underlies ordinary linear regression and its generalized variants.

    • regression — Regression is a statistical modeling technique used to fit relationships between variables, often by finding a line of best fit. Common forms include linear regression, logistic regression, and least-squares regression, with fit often assessed by R-squared.

      • Coefficient of determination
        • r squared — R squared, or the coefficient of determination, measures how well a regression explains variation in the response variable. A value closer to one indicates a better fit, with zero residual variance left unexplained by the model.

      • ordinary least squares regression — Least squares regression is a statistical method for fitting a line or curve by minimizing the sum of squared residuals. Ordinary least squares (OLS) is the standard linear form, used to choose coefficients by minimizing the residual sum of squares; under the Gauss-Markov conditions it yields best linear unbiased estimators (BLUE), and it is often contrasted with regularized least-squares methods such as Tikhonov regularization. Its ideal properties can be undermined by problems such as heteroskedasticity or perfect multicollinearity.

      • regression to the mean — Regression to the mean is the statistical tendency for extreme observations to move closer to average on later measurement. It is often discussed in relation to Galton’s heredity work and Kahneman’s analysis of pilot performance.
  • Linear regression — models a conditional mean as a linear combination of predictors. Coefficients are conditional associations under the model and become causal effects only with a valid identification strategy.
    • Gauss-Markov theorem — The Gauss-Markov theorem states that under homoscedastic, uncorrelated errors, ordinary least squares gives the best linear unbiased estimator. It is a cornerstone result for linear regression, especially in econometrics and statistics.
  • Logistic regression — models log-odds of a binary outcome. Exponentiated coefficients are odds ratios, which can differ substantially from risk ratios when outcomes are common.
    • logistic — Logistic regression models the log-odds of an event as a linear function of predictors. It is built around the logistic, or sigmoid, function 1 over 1 plus e to the negative x.

  • Generalized linear model — combines a random-outcome family with a link function and linear predictor. Poisson, logistic, and ordinary linear regression are members under different choices.

  • Intercept — the model’s expected outcome when all encoded predictors equal zero. Its scientific meaning depends on coding and whether zero lies within a relevant range.

  • Regression coefficient — the modeled change in a conditional outcome summary for a one-unit predictor change while other included predictors are held fixed. The outcome scale and link function determine interpretation.

  • Design matrix — numerical representation of predictors, transformations, indicators, and interactions supplied to a regression model. Coding choices define the comparisons its coefficients estimate.

  • Residual — observed minus fitted outcome. Residual plots can reveal nonlinearity, unequal variance, dependence, and influential observations that undermine a model’s intended use.

  • Interaction — occurs when an association or effect differs across levels of another variable on a specified scale. Absence of multiplicative interaction does not imply absence on an additive scale.

  • Nonlinearity — relationships can be represented by transformations, splines, polynomials, trees, kernels, or other flexible functions. Flexibility raises the need for regularization and out-of-sample assessment.

  • Multilevel model — represents variation at nested or crossed levels, such as students within schools or repeated observations within people. Partial pooling can stabilize group estimates while retaining between-group differences.
    • Mixed models
      • random effects — Random effects are variance components in mixed models, often written as the vector u in the linear mixed-model equation. They are estimated differently from fixed effects, with methods such as BLUP used in place of BLUE for the random part.
  • Time-series model — accounts for ordered dependence, trend, seasonality, and shocks. Randomly shuffling time points usually destroys the structure relevant to forecasting.
    • Time series analysis

      • seasonality — Seasonality is the repeating variation that makes data rise and fall at expected times of year. It matters in time-series analysis, where methods like ARIMA differencing and subseries plots are used to account for it.

      • time series data — Time series data are sequences recorded at successive moments in time and analyzed with tools such as ARMA and ARIMA models, moving averages, and differencing. They are used in econometrics, finance, and many forms of signal analysis, including interrupted and stationary variants.

      • autocorrelation function — The autocorrelation function measures how a signal or random process correlates with a time-shifted copy of itself. It is the Fourier-transform partner of the power spectrum in the Wiener-Khinchin theorem and appears in fluorescence correlation spectroscopy and dynamic light scattering.

      • stationary — Stationarity is a property of a stochastic process whose distribution does not change over time. In time-series analysis, it is often sought through differencing or detrending, and it is tested with tools such as the Dickey-Fuller and KPSS tests.

      • time series — A time series is data observed over time, such as daily stock prices or other sequential measurements. It is commonly decomposed into trend, seasonal, and residual components, and ARIMA models are often used to forecast it.

    • state-space models
      • Kalman filter — The Kalman filter is a recursive estimator for linear systems with Gaussian noise. It alternates between prediction and update steps and underlies many tracking and control applications.
    • autocorrelation — Autocorrelation is the correlation of a signal or time series with a lagged copy of itself. It appears in statistics, signal processing, and spatial analysis, where measures such as Moran’s I and the autocorrelation function detect repeated structure or dependence over time or space.
  • quantitative structure-activity relationships — Quantitative structure-activity relationships are regression-based models used in drug discovery to predict biological activity from chemical and physical descriptors. They are also called quantitative structure-property relationships when the target is a property rather than a biological effect.
• Diagnostics and Model Misspecification
  • regression diagnostics

    • leverage — In statistics, leverage measures how influential a data point is in a linear model, often appearing on diagnostic plots alongside residuals and Cook’s distance. In finance, leverage also refers to using debt to magnify returns, as in leveraged buyouts.

    • residuals — Residuals are the differences between observed values and the values predicted by a regression model. They are central to diagnostics like studentized residuals, the Durbin-Watson statistic, and the Breusch-Pagan test.

  • Omitted-variable bias — coefficient distortion that can arise when an excluded cause of the outcome is associated with an included predictor. Adding variables indiscriminately can also create bias. The missing factor can distort the estimated coefficient and lead to misleading causal conclusions.

  • Multicollinearity — strong linear dependence among predictors, which can inflate coefficient uncertainty and make individual effects unstable without necessarily harming prediction.

  • Heteroskedasticity — outcome or residual variance changes across predictor values. It can invalidate ordinary standard errors even when the conditional-mean specification is useful. It is the statistical condition in which variance changes across levels of a variable or across subgroups of residuals. It violates the Gauss-Markov assumptions and is often addressed with weighted or generalized least squares; robust standard errors are also used to adjust inference.

Multivariate Analysis

• factor analysis — Factor analysis is a statistical method that explains observed variables in terms of a smaller set of latent factors. It is used in both exploratory and confirmatory forms, often to study survey data and traits such as the Big Five.

Prediction and Machine Learning

• Learning and Validation
  • machine learning
    • support vector machines — Support vector machines are supervised machine-learning methods that classify data by finding a maximum-margin hyperplane. They use kernels for nonlinear classification and can also be adapted to regression with epsilon-insensitive loss.
  • Data science
    • Data mining — Data mining is the automated search for patterns in large datasets, especially for prediction and marketing. It is commonly applied to customer databases and data warehouses.

  • Training, validation, and test data — training fits model parameters, validation informs model choice, and an untouched test set estimates final performance. Repeatedly consulting the test set turns it into another validation set.

  • Overfitting — learning noise or sample-specific structure that fails to generalize. It is controlled through design, regularization, limited model complexity, adequate sample size, and honest validation. It is the tendency of a model to fit training data too closely and fail to generalize to new data. It is often reduced by regularization methods such as LASSO, early stopping, and dropout.

  • Cross-validation — rotates held-out subsets to estimate predictive performance. Splits must respect patients, families, sites, batches, or time so related observations do not leak across folds.

  • Regularization — penalizes model complexity. Ridge regression shrinks coefficients; lasso can set some to zero; elastic net combines both penalties.
    • LASSO — LASSO is a regression method that adds an L1 penalty to shrink some coefficients exactly to zero. It is often paired with ridge regression in the elastic net and is credited by name to Robert Tibshirani.
  • Loss function — assigns a cost to prediction errors and defines what model fitting or evaluation optimizes. Squared, absolute, logarithmic, and decision-specific losses value errors differently.
    • calculus
      • integration
        • area under the curve — The area under the curve is the quantity found by integrating a graph’s values across an interval. It is used in pharmacokinetics to measure total drug exposure from the concentration-time curve, and in statistics to summarize diagnostic test performance.

  • External validation — evaluates a locked model in data meaningfully separate by place, time, or population. Random internal splits do not test many real deployment shifts.

  • Dataset shift — the joint distribution of predictors and outcomes can change across environments or time. Performance monitoring must distinguish sampling noise from clinically or operationally important drift.

  • Algorithmic fairness — fairness criteria formalize different ethical goals and can be mutually incompatible. Group metrics require contextual judgment about harms, access, measurement, and decision thresholds.
• Classification Performance and Decisions
  • Classification metrics — sensitivity, specificity, predictive values, likelihood ratios, calibration, and decision consequences answer different questions. Accuracy alone can be misleading under class imbalance.
    • diagnostic test evaluation
      • sensitivity — Sensitivity is the true positive rate of a diagnostic test. In Boolean-function complexity, sensitivity counts how many input bits can flip the output when changed one at a time.

  • Confusion matrix — cross-tabulates predicted and observed classes into true positives, false positives, true negatives, and false negatives at a specified threshold.

  • Discrimination — ability to rank or separate units with different outcomes. It does not show whether predicted probabilities are accurate or decisions improve outcomes.

  • Calibration — agreement between predicted probabilities and observed outcome frequencies in a target population. A model can discriminate well but be poorly calibrated.

  • Decision threshold — converts a continuous score or probability into an action. The appropriate threshold depends on consequences, available interventions, capacity, and the target population.

  • Area under the ROC curve — probability that a randomly chosen positive case receives a higher score than a randomly chosen negative case under the evaluated population. It averages across thresholds that may be irrelevant.

  • Brier score — mean squared difference between predicted probabilities and binary outcomes. It reflects both calibration and resolution and must be compared within a defined prediction setting.
• Exploration and Representation

  • clustering — Clustering is an unsupervised data-analysis task that groups similar points into namesake clusters, often using k-means, hierarchical methods, DBSCAN, or OPTICS. Typical clues mention Lloyd’s algorithm, centroids, dendrograms, silhouette analysis, and related bioinformatics or single-cell sequencing tools.

  • Decision trees
    • Gini impurity — Gini impurity is a decision-tree splitting criterion that measures how often a randomly chosen sample would be misclassified by class proportions. For a binary classification problem, it equals one minus the sum of the squared class probabilities.

  • dimensionality — The number of dimensions is the quantity that counts independent coordinates in a space or the coordinates, variables, or features needed to specify points in a dataset. It underlies the curse of dimensionality, which makes data sparse and complicates methods such as k-nearest neighbors, principal component analysis, PCA, and t-SNE; increasing dimensionality can also lead to overfitting.

  • dimensionality reduction
    • principal component analysis — Principal component analysis is a linear dimensionality-reduction method that projects data onto directions of maximum variance, rotating correlated variables into a set of uncorrelated principal components. It finds the leading eigenvectors of the covariance matrix, often via singular value decomposition, and is used for visualization, preprocessing, stratification correction, and explaining variance in high-dimensional data.

Stochastic Models and Specialized Applications

Study stochastic and specialized methods after the basic inference framework.

Stochastic Processes and Markov Models

• Markov chain Monte Carlo — Markov chain Monte Carlo is a family of sampling algorithms used to draw from complex stationary distributions. Examples include Gibbs sampling and the Metropolis-Hastings algorithm.

  • Gibbs sampling — Gibbs sampling is a Markov chain Monte Carlo method for drawing from a joint distribution by repeatedly sampling conditional distributions. It is a special case of Metropolis-Hastings and is named after the same Gibbs associated with free energy.

  • Metropolis-Hastings algorithm — The Metropolis-Hastings algorithm is a Markov chain Monte Carlo method for sampling from a target distribution. It uses an acceptance ratio based on detailed balance and reduces to Gibbs sampling when proposals are always accepted.

• probabilistic sequence models
  • Hidden Markov Model — A hidden Markov model is a probabilistic sequence model with unobservable states and observable emissions, with transitions between states and output probabilities attached to each state. It is used in speech recognition, gene finding, and multiple-sequence alignment, and it is commonly trained with the Baum-Welch algorithm and decoded with Viterbi.
• stochastic processes — Stochastic processes are random processes whose evolution may be studied with tools such as Monte Carlo methods. Brownian motion and Markov chains are standard examples, with the latter described by successive powers of a transition matrix.
  • Markov chain — A memoryless stochastic process built from states and fixed transition probabilities, where the next state depends only on the current one. Standard examples include discrete-time chains, random walks, and Markov chain Monte Carlo methods such as Metropolis-Hastings and Gibbs sampling. Common properties and tests include irreducibility, absorbing states, the Chapman–Kolmogorov identity, and the Markov property.

    • Andrei Markov — Andrei Markov was a Russian mathematician whose name is attached to Markov chains and Markov blanket language in probabilistic models. His work describes systems whose next state depends on the present state rather than the full past.

    • Chapman-Kolmogorov equation — The Chapman-Kolmogorov equation is a Markov-process identity that relates transition probabilities across intermediate states. It expresses the probability of moving in stages by integrating over all possible intermediate values.

    • stationary distribution — A stationary distribution is the long-run distribution reached by a Markov chain after mixing. It is a normalized left eigenvector of the transition matrix, and the same adjective also describes stochastic processes whose joint distribution does not change under time shifts.

  • Markov process — A Markov process is a stochastic process whose future depends only on the present state and not on the earlier history. It is named after the Russian mathematician Andrey Markov.

  • martingales — Martingales are stochastic processes in which the conditional expectation of the next value equals the present value. They are named for the doubling betting strategy, and they appear in Doob’s optional stopping and convergence theorems.

  • master equations — Master equations describe the time evolution of probabilities in Markovian systems, with each state’s rate set by the other states that can feed into it. They are used in both classical stochastic processes and quantum settings, where the Lindblad equation is a common example.

  • Poisson process — A Poisson process is a counting process with independent increments whose event counts over time follow the Poisson distribution with rate parameter lambda. It models randomly arriving events such as customers entering a queue, and its waiting times between events are exponentially distributed.

  • stochastic — Stochastic describes random or probabilistic processes, as opposed to deterministic ones. It appears in topics like stochastic calculus, Markov chains, martingales, and DSGE macroeconomic models.

  • stochastic process — A stochastic process is a family of random variables indexed by time or space, with compatible finite-dimensional distributions. Familiar examples include Brownian motion, Poisson processes, and stock-market fluctuations.

  • Wiener process — A Wiener process is the continuous-time limit of a random walk and a standard model of Brownian motion. It has stationary independent Gaussian increments, Hurst parameter one-half, and quadratic variation equal to time.

  • Fokker-Planck equation — The Fokker-Planck equation describes the time evolution of a stochastic process’s probability density under drift and diffusion. It is closely related to the Langevin equation, the Smoluchowski equation, and the Kolmogorov forward equation.
• graphical models
  • Bayesian network — A Bayesian network is a graphical model that encodes conditional independencies through a directed network of nodes and missing edges. Its nodes are parameterized by conditional probability distributions, and the name refers to Bayes’s theorem.
• probabilistic graphical models
  • Bayesian networks — Bayesian networks are directed acyclic graphs used to model probabilistic dependencies through conditional probabilities at each node. They support inference methods such as variable elimination, Gibbs sampling, and belief propagation, and are closely related to d-separation and Markov blankets.

Information Theory

• Kullback-Leibler divergence — Kullback-Leibler divergence measures how one probability distribution differs from another, and it is also called relative entropy. It appears throughout statistics and machine learning, including variational autoencoders, where it compares approximate and target distributions.

Actuarial Science and Risk

• actuary — An actuary applies statistics and probability to assess risk, especially in insurance and pensions. The profession is associated with mortality tables, life expectancy, and exams such as the P and FM series.

Quality Control and Process Improvement

• quality control
  • Six Sigma — Six Sigma is a process-improvement and quality-control methodology aimed at reducing defects to fewer than 3.4 per million opportunities and controlling variation through statistical methods. It emphasizes process capability, variation reduction, stable operations, and design for quality, and is associated with DMAIC and DMADV as well as Jack Welch’s General Electric.
• statistical quality control
  • W. Edwards Deming — W. Edwards Deming was an American statistician and management theorist, also described as a business theorist and quality-control expert. He was a major influence on Japanese industry after World War II and is associated with statistical quality control, total quality management, continuous improvement, and the drive to reduce fear in organizations. His ideas include the Deming management method, the 14 Points of Management, the System of Profound Knowledge, and the warning that businesses should do things right the first time; he is also linked to the seven deadly diseases of business. The American Statistical Association later named an award after him.

Network Analysis

• Node centrality — Node centrality is a measure of how structurally important a node is within a network. A common subtype is betweenness centrality, which counts how many shortest paths pass through the node.

Quantitative Linguistics

• Zipf’s law — Zipf’s law is an empirical rank-frequency pattern in which the frequency of a word or other item is inversely related to its rank. It is often cited in linguistics and city-size distributions, and can be treated as a special case of the discrete Pareto distribution.

Bibliometrics and Research Metrics

• Bibliometrics

  • citations — Citations are the references listed in bibliographies and indexed by services like Google Scholar. They also underlie metrics such as the impact factor and h-index, and the term citogenesis refers to errors spreading through citation chains.

  • journal impact factor — The journal impact factor is a citation-based metric used to estimate a journal’s average influence. It is calculated from citations to articles published in the previous two years and is often criticized for encouraging citation manipulation and for being a poor proxy for article quality. Clarivate’s Web of Science is the main source used for the measure.

  • h-index — The h-index is a citation metric that counts how many papers a scholar has published that have each been cited at least that many times. It is used to summarize research productivity and impact with a single number.

Demography and Population Measurement

Apply sampling and rate concepts to population counts, fertility, and growth; keep historical population models with their demographic setting.

• Demography

  • dependency ratio — The dependency ratio compares the nonworking-age population to the working-age population. It is often used to describe the pressure that children and the elderly place on productive workers and social support systems.

  • Demographics — Demographics is the study of the size, composition, growth, and distribution of human populations. It uses statistical methods to describe social and population characteristics.

  • fertility rates
    • total fertility rate — The total fertility rate is a demographic measure estimating how many children a woman would have over her lifetime under current age-specific birth rates. It is often discussed with replacement-level fertility around 2.1 births per woman and with historical patterns such as postwar baby booms or later declines.
  • pioneers
    • John Graunt — John Graunt was a London shopkeeper whose Natural and Political Observations on the Bills of Mortality helped found demography. He is often called the Father of Vital Statistics.
  • population ecology
    • fecundity — Fecundity is a measure of reproductive potential, especially the number of viable offspring an organism can produce. In population models and Leslie matrices, it is distinguished from fertility, which refers to realized reproduction.
  • population growth rates
    • rate of natural increase — The rate of natural increase is the crude birth rate minus the crude death rate. It measures population growth without migration and is commonly used in demographic transition models.

Interpretation, Reproducibility, and History

Evaluate interpretation and reproducibility in the context of how evidence was produced.

Reproducibility and Interpretation

• Reproducible analysis — provides data provenance, code, software versions, parameters, and deterministic or seeded steps sufficient to regenerate results where sharing rules permit.

• Researcher degrees of freedom — flexible choices in outcomes, exclusions, transformations, subgroups, and models can produce unstable findings. Preregistration and multiverse or sensitivity analyses make this flexibility visible.

• Data leakage — information unavailable at the intended prediction time enters training or evaluation. Leakage can arise through preprocessing before splitting, future variables, duplicated units, or target-derived features.

• Sensitivity analysis — tests how conclusions change under plausible alternative assumptions, specifications, missing-data mechanisms, or unmeasured bias.

• Model checking — compares model implications with observed patterns and the intended use. Good fit to training data is insufficient when assumptions, calibration, or transportability fail.

• Association and causation — a precise association can still be noncausal, and a causal effect can be estimated imprecisely. Claims should match the design and assumptions rather than the sophistication of the model.

• Version control — records changes to code and text with recoverable history, authorship, and branches. It supports collaboration but does not by itself capture data or software environments.

• Environment capture — records operating system, package versions, dependencies, and configuration through lockfiles, containers, or equivalent manifests so computations can be reconstructed.

• Automated test — executes a defined check on code or data behavior. Useful tests target scientific invariants, transformations, edge cases, and prior failures rather than merely repeating implementation details.

• Executable report — generates narrative, tables, and figures directly from analysis code and controlled inputs, reducing manual transcription and keeping reported results synchronized.

• Scientific reproducibility
  • replication crisis — The replication crisis is the widespread failure of published studies to reproduce reliably, especially in social science and psychology. It has been linked to p-hacking, publication bias, and the pressure to publish positive results.
• Statistical paradoxes
  • Simpson’s paradox — Simpson’s paradox is the statistical phenomenon where a trend reverses or disappears when data are combined rather than separated into groups. It is often illustrated with examples from medical statistics or batting averages such as Derek Jeter and David Justice.

Figures in the History of Statistics

• statistics — Statistics is the branch of mathematics concerned with collecting, analyzing, and interpreting data. It is used in sampling, epidemiology, and other fields, and is closely linked to probability.

  • Francis Galton — Francis Galton was a British polymath and statistician best known for coining “eugenics” and for his work on heredity, fingerprints, and weather mapping. He also popularized the idea of regression, built the bean machine to model normal approximation, and wrote Hereditary Genius.

  • Ronald Fisher — Ronald Fisher was a foundational statistician and geneticist who helped develop ANOVA, the F-test, and modern hypothesis testing. His name also appears in Fisher information, the Fisher exact test, and the fundamental theorem of natural selection.

• Statisticians

  • Emil Gumbel — Emil Gumbel was a Heidelberg probabilist whose name is attached to the Type I extreme value distribution. He was also one of the signers of the Urgent Call for Unity against Nazi electoral success.

  • Francis Anscombe — Francis Anscombe was a statistician noted for Anscombe’s quartet, four data sets with nearly identical summary statistics but very different plots. He also gave his name to a transform used to stabilize the variance of Poisson distributions.

  • George Box — George Box was a statistician known for the remark that all models are wrong, but some are useful. His name appears in the Box-Behnken design, the Box-Jenkins method for fitting ARIMA models, the Ljung-Box test for autocorrelation, and the Box-Cox transformation used to normalize data.

  • Nate Silver — The Signal and the Noise is Nate Silver’s book about prediction, uncertainty, and the limits of forecasts. Silver became famous for election forecasting through FiveThirtyEight and for applying statistical models to politics and sports.

  • Ronald Aylmer Fisher — British statistician and evolutionary biologist associated with ANOVA, maximum likelihood, sufficiency, Fisher’s exact test, the Fisher information matrix, and the fundamental theorem of natural selection. He coined the term null hypothesis and is also associated with the Fisher-Wright model, the Fisher-Yates shuffle, and the principle of sex ratio equilibrium.

  • Sir Francis Galton — Sir Francis Galton was a Victorian statistician and polymath best known for promoting eugenics. He also made important early contributions to correlation, regression, and meteorology, and he was Charles Darwin’s cousin.

  • William Sealy Gosset — William Sealy Gosset was the Guinness brewer and statistician who published under the pseudonym Student. He developed the t-test for small-sample statistics and the t distribution used for small-sample inference and confidence intervals; he is often remembered in mathematics and statistics simply as Student.

  • Corrado Gini — Corrado Gini was an Italian economist and statistician best known for the Gini coefficient, a measure of income inequality. He also wrote on fascism and eugenics and was associated with Mussolini-era institutions in Italy.

  • John Tukey — John Tukey was a major statistician known for exploratory data analysis, the box-and-whisker plot, the Tukey HSD test, and the Cooley-Tukey fast Fourier transform. He also contributed the jackknife method and helped shape modern data analysis practice.

  • Karl Pearson — Karl Pearson was a pioneering statistician who introduced the histogram and developed Pearson’s correlation coefficient r. His work at University College London also inspired the later chi-squared test associated with his name.

Sources and Further Study

• OpenStax, Introductory Statistics 2e

• NIST/SEMATECH e-Handbook of Statistical Methods

• Berkeley Stat 210A notes on hypothesis testing

• The Introduction to Statistical Learning resources

• Hernán and Robins, Causal Inference: What If