5. Understanding Kafka Topics

A topic is where Kafka stores a related stream of events, and in most cases, it needs to be created before producers can write to it. Think of it as the named place where one business flow lives, such as all bookstore order events going into an orders topic. In production, automatic topic creation is usually disabled so topics do not appear by accident.

When creating a topic, you normally choose the topic name, partition count, and replication factor. Partitions control how much parallelism Kafka can support, while replication controls how many copies of the data are kept for safety. In a single-node development setup, replication factor 1 is fine, but real clusters usually use higher replication.

bash

kafka-topics.sh --create \

--topic orders \

--partitions 3 \

--replication-factor 1 \

--bootstrap-server localhost:9092

Topics can be tuned with settings that control how long data stays available and how older records are handled. Two of the most common settings are retention.ms and cleanup.policy. These shape how Kafka manages storage over time.

Retention.ms tells Kafka how long to keep records before they can be deleted. cleanup.policy decides whether Kafka removes old messages normally or keeps only the latest record for each key through compaction. For a topic full of order history, retention is usually the main setting.

If the bookstore wants to keep completed orders for seven days, it could set retention to roughly 604800000 milliseconds. That gives consumers enough time to read and replay the data if needed. It also prevents the topic from growing forever and filling disk space.

Good topic names make a Kafka system much easier to understand. A clear naming style helps developers, analysts, and operators know what each stream is for without digging through extra documentation. That becomes especially useful when the number of topics starts to grow.

A useful naming pattern is to keep names lowercase and use dots or hyphens to show the domain and event type. A name like bookstore.orders.created is much easier to interpret than something vague like data1. It tells you the system, the entity, and the event in one glance.

Once a team picks a naming style, the important thing is consistency. That makes permissions, monitoring, and troubleshooting much easier across the whole Kafka cluster. A steady naming convention also helps new team members understand the platform faster.