Loading Data from Web APIs

An API (Application Programming Interface) allows applications to exchange information. In data science, APIs are commonly used to retrieve weather observations, financial information, public statistics, and other data published by external services.

Many APIs return data in JSON format.

Sending an API Request

The Python requests library makes HTTP requests to web services.

Install it if necessary:

pip install requests pandas

The following example uses a public sample API to retrieve records.

import requests
url = "https://jsonplaceholder.typicode.com/posts"
response = requests.get(url, timeout=15)
response.raise_for_status()
records = response.json()
print(records[:2])

The example performs a GET request and converts the JSON response into Python objects. The raise_for_status() method raises an exception if the server returns an unsuccessful HTTP status code.

Converting API Data into a DataFrame

After retrieving a JSON response containing a list of records, you can convert it into a DataFrame.

import requests
import pandas as pd
url = "https://jsonplaceholder.typicode.com/posts"
response = requests.get(url, timeout=15)
response.raise_for_status()
records = response.json()
df = pd.DataFrame(records)
print(df.head())
print(df.columns)

The resulting DataFrame contains fields such as userId, id, title, and body.

You can then filter the records or select relevant columns.

selected_data = df[["userId", "title"]]

print(selected_data.head())

Handling API Errors

API requests can fail because of network problems, invalid URLs, rate limits, or server errors. Use exception handling to manage these situations.

import requests
url = "https://jsonplaceholder.typicode.com/posts"
try:
response = requests.get(url, timeout=15)
response.raise_for_status()
records = response.json()
print("Records retrieved:", len(records))
except requests.exceptions.Timeout:
print("The request timed out.")
except requests.exceptions.RequestException as error:
print("Unable to retrieve data:", error)

For production projects, also consider API authentication, pagination, rate limits, response validation, and secure storage of API keys. Never place private credentials directly in publicly shared source code.