Covariance and Correlation Between Variables

Covariance and correlation describe how two variables move together. Both appear whenever you look at relationships between columns in a dataset.

Covariance

Cov(X, Y) = Σ (xᵢ − x̄)(yᵢ − ȳ) / (n − 1)

• Cov(X, Y): the covariance. A positive value means the variables tend to rise together, and a negative value means one tends to rise as the other falls.
• xᵢ − x̄ and yᵢ − ȳ: how far each pair of values sits from its own mean.
• n − 1: the sample divisor, as used for sample variance.

Covariance depends on the units, so it is hard to judge on its own. A covariance of 500 could be strong or weak, depending on the data scale.

Correlation

r = Cov(X, Y) / (sₓ × sᵧ)

• r: the correlation coefficient, always between −1 and +1.
• sₓ and sᵧ: the standard deviations of X and Y. Dividing by them removes the units.

An r near +1 shows a strong positive linear relationship, near −1 a strong negative one, and near 0 little linear relationship. Correlation measures only straight-line patterns, so a clear curve can still give r close to 0.

Example: Hours studied (X) and exam score (Y) for four students: X = 1, 2, 3, 4 and Y = 50, 60, 70, 80. Find r.

1. x̄ = 2.5 and ȳ = 65
2. The products (xᵢ − x̄)(yᵢ − ȳ) are (−1.5)(−15) = 22.5, (−0.5)(−5) = 2.5, (0.5)(5) = 2.5 and (1.5)(15) = 22.5, which add up to 50
3. Cov(X, Y) = 50 / 3 ≈ 16.67
4. sₓ = √(5 / 3) ≈ 1.29 and sᵧ = √(500 / 3) ≈ 12.91
5. r = 16.67 / (1.29 × 12.91) ≈ 1.00

The points lie exactly on a straight line, so r = 1.

import pandas as pd

df = pd.DataFrame({"hours": [1, 2, 3, 4], "score": [50, 60, 70, 80]})
print(df.corr())

Code Explanation:

• The two columns are placed in a DataFrame.
• df.corr() calculates the correlation between every pair of columns.
• The output is a matrix with 1.0 on the diagonal (since each column correlates perfectly with itself) and 1.0 between hours and score for this data.