8. Your First Apache Kafka Application

A first Kafka producer for the bookstore is usually small and practical. Its job is to capture order details, serialize them, and send them to the orders topic. That makes it the entry point for the whole event flow.

The producer can also attach the order ID as the key. That helps Kafka send related messages to the same partition, which is useful when ordering matters for one order or one customer. It also makes partition behavior more predictable.

The basic pattern is simple: build the payload, convert it to JSON, call produce(), and flush before the script exits. That is enough to get a working event into Kafka. After that, you can improve it with better error handling or richer logging.

A consumer is the part of the application that listens to the orders topic and reacts to each incoming event. In the bookstore example, that could represent shipping, billing, or inventory logic. Its job is to poll Kafka, read messages, and process them safely.

The consumer usually runs in a loop. It calls poll(), checks whether a record arrived, converts the JSON back into a Python object, and performs the business action. Once processing succeeds, it commits the offset so Kafka knows the message is handled.

network hiccup or bad record. A reliable consumer keeps running, logs problems clearly, and only advances progress after the work is actually done.

The easiest way to test a Kafka pipeline is to start the consumer first. That way it is already waiting when the producer sends a test order. Then you run the producer and watch the event move through the system.

If everything is working, the consumer should print the order almost immediately after the producer sends it. That confirms the full path is correct: producer, broker, topic, consumer. It is a simple test, but it proves the pipeline works end to end.

After that first test, the next steps are usually adding more consumers, trying different offsets, and sending more realistic order data. That helps you see how Kafka behaves in a slightly more realistic setup and prepares you for advanced topics later.