Bayes' Theorem and Probabilistic Reasoning

What Is Bayes' Theorem?

Bayes' theorem is a mathematical rule for updating the probability of a hypothesis when new evidence becomes available.

It is especially useful when AI systems must reason under uncertainty.

The formula is:

[
P(A \mid B) = \frac{P(B \mid A)P(A)}{P(B)}
]

Where:

• (P(A \mid B)) is the probability of A given evidence B.
• (P(B \mid A)) is the probability of observing B if A is true.
• (P(A)) is the prior probability of A.
• (P(B)) is the overall probability of observing B.

The prior represents what is believed before considering the new evidence. The posterior represents the updated probability after considering that evidence.

A Practical Example: Spam Detection

Suppose:

• 20% of emails are spam.
• 80% of spam emails contain a suspicious keyword.
• 10% of legitimate emails also contain that keyword.

We want to calculate the probability that an email is spam when the keyword appears.

Let:

• (S) = the email is spam.
• (K) = the keyword appears.

The known probabilities are:

[
P(S)=0.20
]

[
P(K \mid S)=0.80
]

[
P(K \mid \neg S)=0.10
]

First, calculate the probability that the keyword appears:

[
P(K) = P(K \mid S)P(S) + P(K \mid \neg S)P(\neg S)
]

[
P(K) = (0.80 \times 0.20) + (0.10 \times 0.80)
]

[
P(K) = 0.24
]

Now apply Bayes' theorem:

[
P(S \mid K) = \frac{0.80 \times 0.20}{0.24}
]

[
P(S \mid K) \approx 0.667
]

The probability that an email is spam given the keyword is approximately 66.7%.

Notice that the keyword appears in some legitimate emails as well. This is why evidence must be considered alongside the initial probability of spam.

Implementing Bayes' Theorem in Python

prior_spam = 0.20
keyword_given_spam = 0.80
keyword_given_not_spam = 0.10
prior_not_spam = 1 - prior_spam
keyword_probability = (
keyword_given_spam * prior_spam
keyword_given_not_spam * prior_not_spam
)
posterior_spam = (
keyword_given_spam * prior_spam
/ keyword_probability
)
print("Probability of spam:", round(posterior_spam, 3))
print("Percentage:", round(posterior_spam * 100, 1))

Output:

Probability of spam: 0.667

Percentage: 66.7

Where Is Bayes' Theorem Used?

Bayesian reasoning is used in:

• Naive Bayes classification.
• Spam filtering.
• Document categorization.
• Medical decision support.
• Fault diagnosis.
• Updating predictions when new evidence becomes available.

Naive Bayes classifiers apply Bayes' theorem with a simplifying assumption that features are conditionally independent given the class. Although this assumption is often unrealistic, the approach can still work well for certain classification tasks.

Practice Exercise

A website receives 1,000 visits. Assume 5% of visits are fraudulent. A fraud detector flags 90% of fraudulent visits, but it also incorrectly flags 10% of legitimate visits.

Using Bayes' theorem, calculate the probability that a visit is fraudulent given that the detector flags it.

Hint: Calculate the overall probability of being flagged before calculating the posterior probability.