Many Python web applications slow down or time out when handling tasks that take more than a second to complete, such as sending emails, resizing images, or generating PDFs. Celery Redis Python solves this by moving those tasks to background workers that run independently from your main application. Mastering async task queues is an essential skill for any Python backend developer building production-grade applications in 2026. <\/p>\n\n\n\n
Want to build production-grade Python backends with async task queues, REST APIs, and cloud deployment? Explore HCL GUVI’s <\/strong>Python Programming Course<\/strong><\/a>, designed for developers who want to go beyond tutorials and build real scalable systems.<\/p>\n\n\n\n Celery is an open-source distributed task queue<\/a> for Python<\/a>. It lets you define functions as tasks and execute them asynchronously<\/a> in the background using worker processes.<\/p>\n\n\n\n Redis is an in-memory data store used here as a message broker. When your app creates a task, Celery sends a message to Redis. A Celery worker picks it up and executes the task independently of your web process.<\/p>\n\n\n\n The flow looks like this:<\/p>\n\n\n\n If you are using Django, also install the Django Celery integration:<\/p>\n\n\n\n The simplest way to run Redis locally is with Docker:<\/p>\n\n\n\n Redis will now be available at redis:\/\/localhost:6379.<\/p>\n\n\n\n Create a celery.py file in your project root:<\/p>\n\n\n\n The broker is where Celery sends task messages. The backend is where results are stored after execution.<\/p>\n\n\n\n The @app.task decorator registers the function as a Celery task that can be called asynchronously.<\/p>\n\n\n\n Using .delay() sends the task to Redis immediately and returns an AsyncResult object. Your web server responds to the client without waiting for the task to finish.<\/p>\n\n\n\n The worker connects to Redis, listens for incoming task messages, and executes them as they arrive. You can run multiple workers to process tasks in parallel.<\/p>\n\n\n\n Every Celery task goes through defined states:<\/p>\n\n\n\n Celery supports multiple queues, allowing you to separate high-priority and low-priority tasks.<\/em><\/p>\n\n\n\nWhat Is Celery and What Is Redis?<\/strong><\/h2>\n\n\n\n
\n
Setting Up Celery with Redis in Python<\/strong><\/h2>\n\n\n\n
Step 1: Install Dependencies<\/strong><\/h3>\n\n\n\n
pip install celery redis<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n pip install django-celery-results<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n Step 2: Start a Redis Instance<\/strong><\/h3>\n\n\n\n
docker run -d -p 6379:6379 redis<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n Step 3: Configure Celery<\/strong><\/h3>\n\n\n\n
from celery import Celery
app = Celery(
“myproject”,
broker=”redis:\/\/localhost:6379\/0″,
backend=”redis:\/\/localhost:6379\/0″,
include<\/strong>=[“myproject.tasks”]
)
app.conf.update(
task_serializer=”json”,
result_serializer=”json”,
accept_content=[“json”],
timezone=”Asia\/Kolkata”,
enable_utc=True<\/strong>,
)<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\nStep 4: Define Tasks<\/strong><\/h3>\n\n\n\n
# tasks.py
from myproject.celery import app
import time
@app.task
def send_welcome_email(user_email):
time.sleep(2) # Simulate email sending
print<\/strong>(f”Email sent to {user_email}”)
return<\/strong> f”Done: {user_email}”
@app.task
def generate_report(report_id):
time.sleep(5) # Simulate report generation
return<\/strong> f”Report {report_id} ready”<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\nStep 5: Call Tasks Asynchronously<\/strong><\/h3>\n\n\n\n
# In your view or API handler
from myproject.tasks import send_welcome_email, generate_report
# Async call – returns immediately
result = send_welcome_email.delay(“user@example.com”)
print<\/strong>(result.id) # Task ID for tracking
# Check result later
print<\/strong>(result.status) # PENDING, SUCCESS, FAILURE
print<\/strong>(result.get()) # Blocks until result is ready<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\nStep 6: Start a Celery Worker<\/strong><\/h3>\n\n\n\n
celery -A myproject worker –loglevel=info<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n Celery Task States<\/strong><\/h2>\n\n\n\n
State<\/strong><\/td> Meaning<\/strong><\/td><\/tr> PENDING<\/td> Task is queued, not yet picked up<\/td><\/tr> STARTED<\/td> Worker has begun executing the task<\/td><\/tr> SUCCESS<\/td> Task completed successfully<\/td><\/tr> FAILURE<\/td> Task raised an exception<\/td><\/tr> RETRY<\/td> Task failed and is scheduled to retry<\/td><\/tr> REVOKED<\/td> Task was manually cancelled<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
\n\n Celery<\/strong> is one of the most widely used Python libraries for background task processing, with millions of downloads each month on PyPI<\/strong>. It has been adopted by organizations such as Instagram<\/strong>, Mozilla<\/strong>, and Robinhood<\/strong> to handle asynchronous workloads. Among the supported message brokers, Redis<\/strong> is the most popular choice because of its low latency, straightforward setup, and excellent performance for many common task queue workloads.\n\n<\/div>\n\n\n\nUsing Task Queues for Priority<\/strong><\/h2>\n\n\n\n