Probability Distributions in Data Science
Probability distributions describe how likely different values of a random variable are. They help data scientists model event counts, waiting times, numerical measurements, and uncertainty.
A probability distribution assigns probabilities to the possible outcomes of a random variable. The correct distribution depends on whether the variable is discrete or continuous and whether the model assumptions are appropriate.
What Are Discrete and Continuous Random Variables?
| Type | Description | Example |
|---|---|---|
| Discrete | Takes countable values | Number of purchases |
| Continuous | Takes values over numerical intervals | Delivery duration |
A discrete distribution uses a Probability Mass Function (PMF).
\[ P(X=x) \]
A continuous distribution uses a Probability Density Function (PDF). Probabilities are calculated over intervals:
\[ P(a\leq X\leq b)=\int_a^b f(x)\,dx \]
For continuous distributions, density is not the same as probability at an individual point.
What Is a Bernoulli Distribution?
A Bernoulli distribution models one experiment with two outcomes.
\[ X\in\{0,1\} \]
Examples include purchase versus no purchase or churn versus retention.
Its probability mass function is:
\[ P(X=x)=p^x(1-p)^{1-x} \]
The mean and variance are:
\[ E[X]=p \]
\[ Var(X)=p(1-p) \]
If purchase probability is 0.30:
\[ Var(X)=0.30(0.70)=0.21 \]
Bernoulli distributions are fundamental to binary classification.
What Is a Binomial Distribution?
A binomial distribution models the number of successes across a fixed number of independent Bernoulli trials with the same success probability.
\[ P(X=k)=\binom{n}{k}p^k(1-p)^{n-k} \]
Here, \(n\) is the number of trials, \(k\) is the number of successes, and \(p\) is the success probability.
Example
A website has a 20% conversion probability. The probability of exactly three purchases from ten independent visits is:
\[ P(X=3)=\binom{10}{3}(0.2)^3(0.8)^7 \]
\[ P(X=3)\approx0.2013 \]
from scipy.stats import binomprobability = binom.pmf(k=3, n=10, p=0.2)print(round(probability, 4))Output: 0.2013
Real-world customers may have different conversion probabilities, so the binomial assumptions should be checked.
What Is a Poisson Distribution?
A Poisson distribution models event counts over a fixed interval under a constant-rate Poisson process.
Examples include support requests per hour or incoming service calls.
\[ P(X=k)=\frac{e^{-\lambda}\lambda^k}{k!} \]
Here, \(\lambda\) represents the expected event count.
Suppose a support centre receives four requests per hour on average.
The probability of exactly three requests is:
\[ P(X=3)=\frac{e^{-4}4^3}{3!}\approx0.1954 \]
from scipy.stats import poissonprint(round(poisson.pmf(k=3, mu=4), 4))Output: 0.1954
For a Poisson distribution:
\[ E[X]=Var(X)=\lambda \]
Data exhibiting substantial overdispersion may require an alternative count distribution.
What Is a Normal Distribution?
A normal distribution is a continuous, symmetric, bell-shaped distribution defined by its mean and standard deviation.
\[ f(x)= \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{(x-\mu)^2}{2\sigma^2}} \]
Its main properties include:
- Mean, median, and mode are equal.
- Approximately 68% of values lie within one standard deviation.
- Approximately 95% lie within two standard deviations.
- Approximately 99.7% lie within three standard deviations.
Z-Score
A Z-score measures how many standard deviations an observation lies from the mean.
\[ z=\frac{x-\mu}{\sigma} \]
Suppose:
\[ x=4,\quad \mu=0,\quad \sigma=2 \]
Then:
\[ z=2 \]
The observation is two standard deviations above the mean.
from scipy.stats import normprobability = norm.cdf(4, loc=0, scale=2)print(round(probability, 4))Output: 0.9772
Approximately 97.72% of values fall at or below 4 under this normal distribution.
What Is an Exponential Distribution?
An exponential distribution models waiting times between events in a homogeneous Poisson process.
\[ f(x)=\lambda e^{-\lambda x}, \quad x\geq0 \]
The expected waiting time is:
\[ E[X]=\frac{1}{\lambda} \]
It has the memoryless property, meaning the remaining waiting-time distribution does not depend on elapsed waiting time.
This model is useful for certain arrival processes but may be unsuitable when event rates change over time.
Probability Distributions Compared
| Distribution | Type | Application |
|---|---|---|
| Bernoulli | Discrete | Binary outcomes |
| Binomial | Discrete | Successes across fixed trials |
| Poisson | Discrete | Event counts |
| Normal | Continuous | Symmetric numerical variation |
| Exponential | Continuous | Waiting times |
| Uniform | Discrete or continuous | Equal likelihood or density across supported outcomes |
What Is Expected Value?
Expected value represents the probability-weighted average outcome.
For a discrete variable:
\[ E[X]=\sum_x xP(X=x) \]
Suppose the probabilities of making zero, one, or two purchases are 0.50, 0.30, and 0.20.
\[ E[X]=0(0.50)+1(0.30)+2(0.20)=0.70 \]
The expected purchase count is 0.70, representing an average across repeated observations rather than a possible individual count.
What Is the Central Limit Theorem?
The Central Limit Theorem (CLT) explains why sample means often approach a normal sampling distribution as sample size increases.
Under classical conditions, including independent and identically distributed observations with finite variance:
\[ \frac{\bar{X}-\mu}{\sigma/\sqrt{n}} \xrightarrow{d}N(0,1) \]
This concerns the distribution of sample means, not the original observations.
For example, individual customer spending values may be skewed, but the means of repeated sufficiently large random samples may be approximately normally distributed.
What Is Standard Error?
Standard error measures the variability of an estimator across repeated samples.
For a sample mean:
\[ SE(\bar{X})=\frac{\sigma}{\sqrt{n}} \]
Using an estimated standard deviation:
\[ \widehat{SE}=\frac{s}{\sqrt{n}} \]
Larger samples generally reduce standard error under the required assumptions.
Python Simulation
import numpy as npimport matplotlib.pyplot as pltrng = np.random.default_rng(42)sample_means = [ rng.exponential(scale=10, size=50).mean() for _ in range(1000)]plt.hist(sample_means, bins=30)plt.title("Sampling Distribution of the Mean")plt.xlabel("Sample Mean")plt.ylabel("Frequency")plt.show()The distribution of the simulated sample means is typically more symmetric than the underlying exponential distribution.
The CLT supports confidence intervals and statistical inference, but no single sample size guarantees a good approximation for every dataset.
Common mistakes: Assuming every variable is normally distributed, confusing density with probability, ignoring distribution assumptions, and assuming the CLT makes individual observations normal.
Data Science Tutorial for Beginners
Vaishali Ardhana








