Calculus for Data Science
Calculus provides mathematical techniques for understanding how functions change as their inputs vary. In data science, this is particularly important for measuring prediction sensitivity and optimising machine learning models.
For example, a regression model may estimate sales revenue using advertising expenditure. Calculus can describe how rapidly the predicted revenue changes as the advertising budget increases.
Calculus is the branch of mathematics that studies change and accumulation. Differential calculus, which focuses on rates of change, is especially important for understanding gradients and optimisation in machine learning.
This lesson covers limits, derivatives, partial derivatives, and their applications in predictive modelling.
What Is a Limit in Calculus?
A limit describes the value a function approaches as its input approaches a particular point.
It is represented as:
\[ \lim_{x\to a}f(x)=L \]
Here:
- \(x\) represents the input variable.
- \(a\) represents the value being approached.
- \(L\) represents the limiting value.
Consider:
\[ f(x)=x^2 \]
As \(x\) approaches 3:
\[ \lim_{x\to3}x^2=9 \]
Limits form the mathematical foundation of derivatives and continuity.
Why Are Limits Important?
Limits help describe the behaviour of functions near particular points.
They are used to define instantaneous rates of change and examine whether functions approach finite values.
In machine learning, limits also appear in mathematical analysis of optimisation procedures and convergence.
What Is a Derivative?
A derivative measures the instantaneous rate of change of a function with respect to its input.
The derivative of \(f(x)\) is defined as:
\[ f'(x)=\lim_{h\to0}\frac{f(x+h)-f(x)}{h} \]
The expression measures how the function changes over a small change in its input.
Example of a Derivative
Consider:
\[ f(x)=x^2 \]
Its derivative is:
\[ f'(x)=2x \]
At \(x=3\):
\[ f'(3)=6 \]
The derivative indicates that the function has an instantaneous rate of change of 6 at \(x=3\).
Geometrically, the derivative represents the slope of the tangent line at that point.
What Are the Basic Rules of Differentiation?
Differentiation follows several standard mathematical rules.
| Rule | Function | Derivative |
|---|---|---|
| Constant Rule | \(f(x)=c\) | \(0\) |
| Power Rule | \(f(x)=x^n\) | \(nx^{n-1}\) |
| Sum Rule | \(f(x)=g(x)+h(x)\) | \(g'(x)+h'(x)\) |
| Product Rule | \(f(x)=g(x)h(x)\) | \(g'h+gh'\) |
| Chain Rule | \(f(x)=g(h(x))\) | \(g'(h(x))h'(x)\) |
These rules allow derivatives of increasingly complex mathematical functions to be calculated.
Example Using the Power Rule
Consider:
\[ f(x)=3x^3+2x^2+5 \]
Differentiating each term gives:
\[ f'(x)=9x^2+4x \]
The constant term disappears because its derivative is zero.
How Are Derivatives Used in Regression?
Regression models estimate numerical outputs using mathematical functions.
Consider a simple prediction model:
\[ \hat{y}=wx+b \]
Here, \(w\) represents the model coefficient and \(b\) represents the intercept.
The model's prediction error can be represented using a loss function.
For one observation:
\[ L(w)=(wx+b-y)^2 \]
Differentiating with respect to \(w\):
\[ \frac{dL}{dw}=2(wx+b-y)x \]
The derivative indicates how the loss changes as the coefficient changes.
This information helps optimisation algorithms determine how model parameters should be updated.
What Are Partial Derivatives?
A partial derivative measures the rate of change of a function with respect to one variable while holding the others constant.
Consider:
\[ f(x,y)=x^2+3xy+y^2 \]
The partial derivative with respect to \(x\) is:
\[ \frac{\partial f}{\partial x}=2x+3y \]
The partial derivative with respect to \(y\) is:
\[ \frac{\partial f}{\partial y}=3x+2y \]
Each derivative describes how the function changes along a particular input direction.
Partial derivatives are essential when a model contains multiple parameters.
How Do Partial Derivatives Apply to Machine Learning?
Most machine learning models contain more than one parameter.
A multiple linear regression model may be expressed as:
\[ \hat{y}=w_1x_1+w_2x_2+b \]
The prediction depends on coefficients \(w_1\) and \(w_2\), along with the intercept \(b\).
The loss function is therefore a function of multiple parameters:
\[ L=L(w_1,w_2,b) \]
Partial derivatives measure the influence of each parameter:
\[ \frac{\partial L}{\partial w_1} \]
\[ \frac{\partial L}{\partial w_2} \]
\[ \frac{\partial L}{\partial b} \]
These values help determine how individual parameters contribute to prediction error.
What Is the Difference Between Derivatives and Partial Derivatives?
| Aspect | Derivative | Partial Derivative |
|---|---|---|
| Function Type | Usually a single independent variable | Multiple independent variables |
| Main Purpose | Measure change with respect to one input | Measure change with respect to one input while others remain fixed |
| Notation | \(df/dx\) | \(\partial f/\partial x\) |
| Example | \(f(x)=x^2\) | \(f(x,y)=x^2+y^2\) |
| Machine Learning Use | Simple optimisation problems | Multi-parameter optimisation |
How Are Derivatives Calculated in Python?
SymPy is a Python library for symbolic mathematics.
It can calculate derivatives analytically.
import sympy as spx = sp.Symbol("x")f = 3*x**3 + 2*x**2 + 5derivative = sp.diff(f, x)print(derivative)Output:
9*x**2 + 4*xThe diff() function calculates the symbolic derivative.
Calculating Partial Derivatives
import sympy as spx, y = sp.symbols("x y")f = x**2 + 3*x*y + y**2dx = sp.diff(f, x)dy = sp.diff(f, y)print("df/dx:", dx)print("df/dy:", dy)Output:
df/dx: 2*x + 3*y
df/dy: 3*x + 2*ySymbolic differentiation is useful for understanding mathematical expressions.
Deep learning frameworks typically rely on automatic differentiation for large computational graphs.
Why Is Calculus Important in Neural Networks?
Neural networks contain many parameters that influence model predictions.
Training involves adjusting these parameters to reduce a loss function.
Calculus provides the mathematical basis for calculating how the loss changes with respect to individual weights and biases.
A neural network may contain thousands or millions of trainable parameters.
Calculating every derivative manually would be impractical.
Automatic differentiation and backpropagation allow gradients to be computed systematically.
These gradients are then used by optimisation algorithms to update model parameters.
What Are the Practical Applications of Calculus?
Calculus supports several data science operations.
- Model Optimisation: Determines how parameters should change to reduce prediction errors.
- Regression Analysis: Measures the sensitivity of predictions to input changes.
- Neural Network Training: Supports gradient computation through backpropagation.
- Numerical Optimisation: Helps identify stationary points and local minima.
- Sensitivity Analysis: Examines how outputs respond to small changes in inputs.
Common Mistakes in Calculus for Data Science
- Confusing a function's value with its derivative.
- Treating partial derivatives as ordinary derivatives without considering other variables.
- Forgetting the chain rule when differentiating composite functions.
- Assuming a derivative of zero always identifies a minimum.
- Ignoring the domain and differentiability conditions of a function.
Data Science Tutorial for Beginners
Vaishali Ardhana








