Menu

Sampling and Hypothesis Testing

Data scientists often analyse samples instead of entire populations. Statistical inference helps estimate population characteristics and evaluate claims using these samples.

Statistical inference uses sample observations to estimate unknown population parameters and quantify uncertainty. Sampling, confidence intervals, and hypothesis testing are its core techniques.

What Is the Difference Between a Population and a Sample?

A population contains every observation relevant to a research question.

A sample is a selected subset.

Feature Population Sample
Meaning Complete target group Selected observations
Numerical Summary Parameter Statistic
Mean \(\mu\) \(\bar{x}\)
Standard Deviation \(\sigma\) \(s\)
Example All active customers 1,000 selected customers

Sample statistics help estimate population parameters.

Their reliability depends on sampling design, sample size, and variability.

What Are the Main Sampling Methods?

Simple Random Sampling

Every possible sample of a specified size has an equal probability of selection.

import numpy as nprng = np.random.default_rng(42)customers = np.arange(1, 10001)sample = rng.choice(    customers,    size=500,    replace=False)print(len(sample))

Output: 500

This selects 500 unique customer IDs.

Stratified Sampling

The population is divided into groups, and observations are sampled from each.

For example, a survey may include new, regular, and premium customers.

Stratification can improve representation and statistical precision when appropriately designed.

Systematic Sampling

Observations are selected at regular intervals after a random starting point.

For example, every 20th customer record may be selected.

Patterns in record ordering can introduce bias.

Cluster Sampling

Groups such as districts, schools, or stores are sampled.

Cluster sampling can lower collection costs but may increase uncertainty because observations within clusters are often related.

Method Selection Strategy Example
Simple Random Random individuals Customer survey
Stratified Sample from predefined groups Customer segments
Systematic Regular intervals Administrative records
Cluster Randomly selected groups Regional surveys

What Is Sampling Bias?

Sampling bias occurs when the selection process systematically overrepresents or excludes certain observations.

For example, surveying only frequent customers may misrepresent satisfaction among all customers.

A larger biased sample can still produce misleading estimates.

Representativeness depends on the sampling process, not merely sample size.

What Is a Confidence Interval?

A confidence interval estimates a range of plausible values for an unknown population parameter.

For a population mean with known standard deviation, an approximate interval is:

\[ \bar{x}\pm z_{\alpha/2}\frac{\sigma}{\sqrt{n}} \]

For a 95% confidence interval, the normal critical value is approximately 1.96.

Worked Example

Suppose:

\[ \bar{x}=2000,\quad \sigma=500,\quad n=100 \]

The standard error is:

\[ SE=\frac{500}{\sqrt{100}}=50 \]

The interval is:

\[ 2000\pm1.96(50) \]

\[ CI=[1902,2098] \]

The approximate 95% confidence interval ranges from ₹1,902 to ₹2,098.

A 95% confidence level means the procedure would capture the true population parameter in approximately 95% of repeated samples under its assumptions. It does not assign a 95% probability to the fixed parameter lying within this particular interval.

What Is the Student's t-Distribution?

The t-distribution is used for many statistical procedures when the population standard deviation is unknown and is estimated from sample data.

A t-based confidence interval for the mean is:

\[ \bar{x}\pm t_{\alpha/2,n-1}\frac{s}{\sqrt{n}} \]

It has heavier tails than the normal distribution, especially with fewer degrees of freedom.

Python Example

import numpy as npfrom scipy import statsdata = np.array([    1800, 1900, 2000, 2100, 2200,    1850, 1950, 2050, 2150, 2250])mean = data.mean()se = stats.sem(data)ci = stats.t.interval(    confidence=0.95,    df=len(data) - 1,    loc=mean,    scale=se)print("Mean:", mean)print("95% CI:", ci)

The interval estimates the population mean under the assumptions of the t procedure.

What Affects Confidence Interval Width?

  • Sample Size: Larger samples generally produce narrower intervals.
  • Variability: Greater variability increases uncertainty.
  • Confidence Level: Higher confidence requires wider intervals.
  • Sampling Design: Clustering or dependence can increase uncertainty.

What Is Hypothesis Testing?

Hypothesis testing evaluates a statistical claim using sample evidence.

Two hypotheses are defined:

Null Hypothesis (\(H_0\)) represents the baseline claim, often no difference.

Alternative Hypothesis (\(H_1\)) represents a competing claim.

For two checkout designs:

\[ H_0:p_A=p_B \]

\[ H_1:p_A\neq p_B \]

The test evaluates whether observed data provide sufficient evidence against equal conversion probabilities.

Main Steps

  1. Define the hypotheses.
  2. Select an appropriate test.
  3. Choose the significance level.
  4. Calculate the test statistic and p-value.
  5. Interpret the evidence and effect size.

What Is a p-Value?

A p-value measures how unusual the observed result, or one more extreme, would be under the null hypothesis and specified statistical model.

A common significance threshold is:

\[ \alpha=0.05 \]

A result with \(p<0.05\) is conventionally statistically significant at the 5% level.

However, a p-value does not represent the probability that the null hypothesis is true or measure the importance of an observed effect.

What Are Type I and Type II Errors?

Actual Condition Reject \(H_0\) Do Not Reject \(H_0\)
\(H_0\) True Type I error Correct non-rejection
\(H_0\) False Correct rejection Type II error

Type I Error: Rejecting a true null hypothesis.

Its probability is controlled by \(\alpha\) under the test assumptions.

Type II Error: Failing to reject a false null hypothesis.

Its probability is represented by \(\beta\).

Statistical power is:

\[ Power=1-\beta \]

Power is the probability of detecting an effect under a specified alternative.

Where Is Statistical Inference Used?

Application Statistical Objective
Customer Surveys Estimate population satisfaction
Marketing Compare conversion outcomes
Healthcare Evaluate treatment-related differences
Manufacturing Assess process measurements
Product Analytics Test changes in engagement
Data Science Research Quantify uncertainty

Common mistakes: Confusing statistical significance with business importance, misinterpreting confidence intervals, ignoring sampling bias, and treating a p-value as proof that a hypothesis is true or false.


Data Science Tutorial for Beginners

VA

Vaishali Ardhana



Learn via Video Course

undefined Logo

0 hrs



Get in Touch For Details