Menu

Probability Distributions in Data Science

Probability distributions describe how likely different values of a random variable are. They help data scientists model event counts, waiting times, numerical measurements, and uncertainty.

A probability distribution assigns probabilities to the possible outcomes of a random variable. The correct distribution depends on whether the variable is discrete or continuous and whether the model assumptions are appropriate.

What Are Discrete and Continuous Random Variables?

Type Description Example
Discrete Takes countable values Number of purchases
Continuous Takes values over numerical intervals Delivery duration

A discrete distribution uses a Probability Mass Function (PMF).

\[ P(X=x) \]

A continuous distribution uses a Probability Density Function (PDF). Probabilities are calculated over intervals:

\[ P(a\leq X\leq b)=\int_a^b f(x)\,dx \]

For continuous distributions, density is not the same as probability at an individual point.

What Is a Bernoulli Distribution?

A Bernoulli distribution models one experiment with two outcomes.

\[ X\in\{0,1\} \]

Examples include purchase versus no purchase or churn versus retention.

Its probability mass function is:

\[ P(X=x)=p^x(1-p)^{1-x} \]

The mean and variance are:

\[ E[X]=p \]

\[ Var(X)=p(1-p) \]

If purchase probability is 0.30:

\[ Var(X)=0.30(0.70)=0.21 \]

Bernoulli distributions are fundamental to binary classification.

What Is a Binomial Distribution?

A binomial distribution models the number of successes across a fixed number of independent Bernoulli trials with the same success probability.

\[ P(X=k)=\binom{n}{k}p^k(1-p)^{n-k} \]

Here, \(n\) is the number of trials, \(k\) is the number of successes, and \(p\) is the success probability.

Example

A website has a 20% conversion probability. The probability of exactly three purchases from ten independent visits is:

\[ P(X=3)=\binom{10}{3}(0.2)^3(0.8)^7 \]

\[ P(X=3)\approx0.2013 \]

from scipy.stats import binomprobability = binom.pmf(k=3, n=10, p=0.2)print(round(probability, 4))

Output: 0.2013

Real-world customers may have different conversion probabilities, so the binomial assumptions should be checked.

What Is a Poisson Distribution?

A Poisson distribution models event counts over a fixed interval under a constant-rate Poisson process.

Examples include support requests per hour or incoming service calls.

\[ P(X=k)=\frac{e^{-\lambda}\lambda^k}{k!} \]

Here, \(\lambda\) represents the expected event count.

Suppose a support centre receives four requests per hour on average.

The probability of exactly three requests is:

\[ P(X=3)=\frac{e^{-4}4^3}{3!}\approx0.1954 \]

from scipy.stats import poissonprint(round(poisson.pmf(k=3, mu=4), 4))

Output: 0.1954

For a Poisson distribution:

\[ E[X]=Var(X)=\lambda \]

Data exhibiting substantial overdispersion may require an alternative count distribution.

What Is a Normal Distribution?

A normal distribution is a continuous, symmetric, bell-shaped distribution defined by its mean and standard deviation.

\[ f(x)= \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{(x-\mu)^2}{2\sigma^2}} \]

Its main properties include:

  • Mean, median, and mode are equal.
  • Approximately 68% of values lie within one standard deviation.
  • Approximately 95% lie within two standard deviations.
  • Approximately 99.7% lie within three standard deviations.

Z-Score

A Z-score measures how many standard deviations an observation lies from the mean.

\[ z=\frac{x-\mu}{\sigma} \]

Suppose:

\[ x=4,\quad \mu=0,\quad \sigma=2 \]

Then:

\[ z=2 \]

The observation is two standard deviations above the mean.

from scipy.stats import normprobability = norm.cdf(4, loc=0, scale=2)print(round(probability, 4))

Output: 0.9772

Approximately 97.72% of values fall at or below 4 under this normal distribution.

What Is an Exponential Distribution?

An exponential distribution models waiting times between events in a homogeneous Poisson process.

\[ f(x)=\lambda e^{-\lambda x}, \quad x\geq0 \]

The expected waiting time is:

\[ E[X]=\frac{1}{\lambda} \]

It has the memoryless property, meaning the remaining waiting-time distribution does not depend on elapsed waiting time.

This model is useful for certain arrival processes but may be unsuitable when event rates change over time.

Probability Distributions Compared

Distribution Type Application
Bernoulli Discrete Binary outcomes
Binomial Discrete Successes across fixed trials
Poisson Discrete Event counts
Normal Continuous Symmetric numerical variation
Exponential Continuous Waiting times
Uniform Discrete or continuous Equal likelihood or density across supported outcomes

What Is Expected Value?

Expected value represents the probability-weighted average outcome.

For a discrete variable:

\[ E[X]=\sum_x xP(X=x) \]

Suppose the probabilities of making zero, one, or two purchases are 0.50, 0.30, and 0.20.

\[ E[X]=0(0.50)+1(0.30)+2(0.20)=0.70 \]

The expected purchase count is 0.70, representing an average across repeated observations rather than a possible individual count.

What Is the Central Limit Theorem?

The Central Limit Theorem (CLT) explains why sample means often approach a normal sampling distribution as sample size increases.

Under classical conditions, including independent and identically distributed observations with finite variance:

\[ \frac{\bar{X}-\mu}{\sigma/\sqrt{n}} \xrightarrow{d}N(0,1) \]

This concerns the distribution of sample means, not the original observations.

For example, individual customer spending values may be skewed, but the means of repeated sufficiently large random samples may be approximately normally distributed.

What Is Standard Error?

Standard error measures the variability of an estimator across repeated samples.

For a sample mean:

\[ SE(\bar{X})=\frac{\sigma}{\sqrt{n}} \]

Using an estimated standard deviation:

\[ \widehat{SE}=\frac{s}{\sqrt{n}} \]

Larger samples generally reduce standard error under the required assumptions.

Python Simulation

import numpy as npimport matplotlib.pyplot as pltrng = np.random.default_rng(42)sample_means = [    rng.exponential(scale=10, size=50).mean()    for _ in range(1000)]plt.hist(sample_means, bins=30)plt.title("Sampling Distribution of the Mean")plt.xlabel("Sample Mean")plt.ylabel("Frequency")plt.show()

The distribution of the simulated sample means is typically more symmetric than the underlying exponential distribution.

The CLT supports confidence intervals and statistical inference, but no single sample size guarantees a good approximation for every dataset.

Common mistakes: Assuming every variable is normally distributed, confusing density with probability, ignoring distribution assumptions, and assuming the CLT makes individual observations normal.


Data Science Tutorial for Beginners

VA

Vaishali Ardhana



Learn via Video Course

undefined Logo

0 hrs



Get in Touch For Details