TQDM Python: Guide to Progress Bars!

When working with long-running tasks or loops in Python, it’s often useful to provide feedback to the user about the progress of the task. This is where the TQDM Python module comes in.

TQDM is a fast, extensible progress bar library in Python that helps you track the progress of loops, iterations, and processes in real time. And it make even your console apps look way cooler.

So, whether you’re processing data, downloading files, or running complex algorithms, TQDM can help enhance user experience by showing progress in a visually appealing way.

By the end of this guide, you’ll have a comprehensive understanding of the TQDM Python module, how it works, and how to integrate it effectively into your code to track progress.

What is TQDM in Python?

TQDM (which means “progress” in Arabic) is a Python library that makes it easy to create progress bars for loops and other iterable processes. It’s designed to be lightweight and provides real-time progress updates without slowing down your code. TQDM can be used in both terminal/command-line interfaces and Jupyter notebooks.

Key Features of TQDM:

Real-time progress bars that show the percentage of completion, time taken, and estimated time remaining.
Works with for loops, functions, and iterators.
Compatible with command-line environments and Jupyter notebooks.
Customizable display options and support for nested progress bars.

Installing TQDM in Python

To use TQDM in your Python environment, you first need to install it. You can install TQDM using pip, Python’s package manager.

Installation:

pip install tqdm

Once installed, you can import it and start using it in your Python scripts.

How to Use TQDM in Python

TQDM can be used in a variety of ways depending on your needs. The most common use case is to display a progress bar for a for loop or any iterable.

1. Basic Usage of TQDM in a For Loop

To use TQDM, you simply wrap your iterable (such as a range(), list, or any other iterable object) with tqdm() in your loop. This will automatically create a progress bar that updates as the loop iterates.

Example: Basic For Loop with TQDM

from tqdm import tqdm
import time

for i in tqdm(range(10)):
    time.sleep(0.5)  # Simulate a task that takes time

Output:

100%|████████████████████████| 10/10 [00:05&lt;00:00,  2.00it/s]

In this example, tqdm(range(10)) tracks the progress of the loop. The progress bar updates in real-time, showing the percentage complete, the number of iterations, and the estimated time remaining.

2. TQDM with Lists and Iterators

TQDM can be used with any iterable, such as lists, tuples, or custom iterators. Simply wrap the iterable with tqdm() to track its progress.

Example: TQDM with a List

from tqdm import tqdm

my_list = [1, 2, 3, 4, 5]

for item in tqdm(my_list):
    # Simulate a task
    time.sleep(0.5)

This will create a progress bar for iterating over the list my_list.

3. Using TQDM in Functions

You can also use TQDM to track progress within functions. This is useful when you’re processing a large dataset or performing multiple tasks inside a function.

Example: TQDM in a Function

from tqdm import tqdm
import time

def process_data(data):
    for item in tqdm(data):
        # Simulate processing each item
        time.sleep(0.3)

data = range(20)
process_data(data)

This example demonstrates using TQDM to track progress inside the process_data() function as it processes each item in the dataset.

4. Using TQDM with File Downloads

TQDM can also be used to track the progress of file downloads or data processing tasks where the progress is not necessarily tied to a loop, but to chunks of data being processed.

Example: Tracking a File Download with TQDM

import requests
from tqdm import tqdm

url = "https://example.com/largefile.zip"
response = requests.get(url, stream=True)
total_size = int(response.headers.get('content-length', 0))

with open("largefile.zip", "wb") as f:
    for data in tqdm(response.iter_content(1024), total=total_size // 1024, unit='KB'):
        f.write(data)

This example demonstrates how to use TQDM to show progress while downloading a file.

Customizing the TQDM Progress Bar

TQDM offers several customization options, allowing you to change the appearance and behavior of the progress bar to suit your needs.

1. Changing the Progress Bar Description

You can add a custom description to the progress bar using the desc parameter. This is useful for giving context to what the progress bar is tracking.

Example:

from tqdm import tqdm
import time

for i in tqdm(range(10), desc="Processing"):
    time.sleep(0.5)

Output:

Processing: 100%|███████████████████| 10/10 [00:05&lt;00:00,  2.00it/s]

2. Adjusting the Update Interval

The progress bar updates automatically, but you can control how frequently it updates using the mininterval parameter. This is helpful for long-running tasks where frequent updates aren’t necessary.

Example:

from tqdm import tqdm
import time

for i in tqdm(range(10), mininterval=1):
    time.sleep(2)

In this case, the progress bar only updates once every second.

3. Customizing Progress Bar Format

You can customize the format of the progress bar using the bar_format parameter. This allows you to adjust what information is displayed (such as percentage, elapsed time, or iteration count).

Example:

from tqdm import tqdm
import time

for i in tqdm(range(10), bar_format="{l_bar}{bar} | {n_fmt}/{total_fmt} tasks"):
    time.sleep(0.5)

Output:

100%|████████████████████████| 10/10 tasks

This example shows a custom progress bar format with a task count.

TQDM in Jupyter Notebooks

If you are working in a Jupyter notebook or a similar interactive environment, TQDM has special support for displaying progress bars directly within the notebook interface. To use TQDM in Jupyter, you need to import the tqdm.notebook module.

Example: Using TQDM in a Jupyter Notebook

from tqdm.notebook import tqdm
import time

for i in tqdm(range(10)):
    time.sleep(0.5)

This will display a progress bar within the notebook’s output cell, updating in real-time as the loop progresses.

Best Practices for Using TQDM

1. Use TQDM for Long-Running Tasks

TQDM is most useful when tracking long-running tasks or loops with many iterations. Adding a progress bar gives users immediate feedback on how much of the task is completed and how long it will take to finish.

Example:

for i in tqdm(range(1000000)):
    pass  # Long-running task

2. Combine TQDM with Logging

For tasks where you want to track progress while also logging key information, consider combining TQDM with Python’s logging module. This allows you to show progress while still printing or logging status messages.

Example:

import logging
from tqdm import tqdm

logging.basicConfig(level=logging.INFO)

for i in tqdm(range(10)):
    logging.info(f"Processing item {i}")

3. Use TQDM in Parallel Processing

TQDM can be combined with parallel processing libraries like multiprocessing or concurrent.futures to show progress bars even when performing tasks concurrently.

Example: Using TQDM with `concurrent.futures`

from concurrent.futures import ThreadPoolExecutor
from tqdm import tqdm
import time

def process_task(n):
    time.sleep(0.5)
    return n

with ThreadPoolExecutor(max_workers=5) as executor:
    results = list(tqdm(executor.map(process_task, range(10)), total=10))

This example demonstrates how TQDM can track progress when running multiple tasks concurrently.

Summary of Key Concepts

TQDM is a Python library that provides real-time progress bars for loops and iterable tasks.
You can use TQDM by simply wrapping any iterable, such as a for loop, list, or iterator, with the tqdm() function.
TQDM is fully customizable, allowing you to change the description, format, and update frequency of the progress bar.
You can use TQDM in command-line environments as well as Jupyter notebooks.
TQDM is useful for tracking long-running tasks, file downloads, and parallel processing.

Exercises

Basic Progress Bar: Write a Python program that uses TQDM to display a progress bar while simulating a task that takes 10 seconds to complete.
File Download Progress Bar: Use TQDM to track the progress of downloading a large file from the internet.
Custom Progress Bar: Write a Python script that uses TQDM with a custom progress bar format to track the processing of a list of items.

Lightning bolt and Python code snippet with "LEARN PYTHON PROGRAMMING MASTERCLASS" in blocky caps

Check out our FREE Learn Python Programming Masterclass to hone your skills or learn from scratch.

The course covers everything from first principles to Graphical User Interfaces and Machine Learning

You can read more about dqtm on GitHub.

FAQ

Q1: Can I use TQDM with functions that don’t use loops?

A1: Yes, you can use TQDM even if your code doesn’t use explicit loops, as long as the function is working with an iterable (such as a generator or a list). For functions that process large datasets or multiple tasks in sequence, you can wrap the tasks in an iterable and use TQDM to track their progress.

Example:

from tqdm import tqdm

tasks = [task1, task2, task3]  # List of tasks
for task in tqdm(tasks):
    task()  # Call each task function

Q2: How can I stop TQDM from printing a new line after completion?

A2: By default, TQDM prints a new line after the progress bar finishes. If you want to prevent this behavior (for example, in cases where you want subsequent output to appear on the same line), set the leave parameter to False.

Example:

from tqdm import tqdm

for i in tqdm(range(10), leave=False):
    pass  # Task

Q3: How can I use TQDM with asynchronous tasks or async functions?

A3: TQDM doesn’t natively support asynchronous tasks (like those written with async and await). However, you can track progress using TQDM within an async loop by manually updating the progress bar using the update() method.

Example:

import asyncio
from tqdm import tqdm

async def async_task():
    await asyncio.sleep(1)

async def main():
    loop = tqdm(total=10)
    for _ in range(10):
        await async_task()
        loop.update(1)
    loop.close()

asyncio.run(main())

Q4: How do I update the TQDM progress bar manually?

A4: If you need more control over the progress bar (for example, if you are processing items in a function that doesn’t use a loop), you can manually update the progress bar using the update() method. This is useful for custom progress tracking where you can’t easily wrap the iterable with tqdm().

Example:

from tqdm import tqdm

pbar = tqdm(total=100)
for i in range(10):
    # Manually update the progress bar by increments of 10
    pbar.update(10)
pbar.close()

Q5: Can I use TQDM with pandas when processing dataframes?

A5: Yes, TQDM provides a handy extension for pandas that allows you to apply progress bars directly to pandas operations such as apply(), groupby(), and more. To use TQDM with pandas, first install the tqdm pandas extension and use the tqdm.pandas() method to enable progress bars.

Example:

import pandas as pd
from tqdm import tqdm

# Enable TQDM progress bars for pandas
tqdm.pandas()

df = pd.DataFrame({'A': range(1000000)})
# Apply a function with a progress bar
df['A'].progress_apply(lambda x: x + 1)

Q6: Can I change the appearance of the progress bar, such as the bar character?

A6: Yes, you can customize the appearance of the progress bar by changing the bar_format or modifying the characters used in the progress bar. For instance, you can replace the default character (█) with another symbol.

Example:

from tqdm import tqdm

for i in tqdm(range(100), bar_format="{l_bar}{bar}| {n_fmt}/{total_fmt}", ascii=".>"):
    pass

This will use . and > for the progress bar instead of the default block character.

Q7: How can I use TQDM to track progress across multiple processes?

A7: You can use TQDM with Python’s multiprocessing module, but you need to carefully manage shared state. TQDM provides a solution called tqdm.contrib.concurrent for this, which works with concurrent.futures for parallel execution.

Example:

from concurrent.futures import ThreadPoolExecutor
from tqdm.contrib.concurrent import thread_map
import time

def process_item(item):
    time.sleep(1)
    return item

# Use thread_map with tqdm for parallel execution and progress tracking
items = range(10)
results = thread_map(process_item, items, max_workers=4)

This method allows you to track progress when using multiple threads or processes.

Q8: Can I nest TQDM progress bars (i.e., show progress for tasks inside other tasks)?

A8: Yes, TQDM supports nested progress bars. This is useful when you have tasks within tasks and want to track the progress of both the inner and outer tasks.

Example: Nested Progress Bars

from tqdm import tqdm
import time

outer = tqdm(total=3, desc='Outer Loop')
for _ in range(3):
    inner = tqdm(total=5, desc='Inner Loop', leave=False)
    for _ in range(5):
        time.sleep(0.1)
        inner.update(1)
    inner.close()
    outer.update(1)
outer.close()

This displays an inner progress bar for each task within the outer progress bar.

Q9: Can I display the progress bar as a percentage only, without additional information?

A9: Yes, you can customize the progress bar to display only specific information, such as the percentage of completion. To do this, use the bar_format parameter to control what is displayed.

Example: Showing Only Percentage

from tqdm import tqdm

for i in tqdm(range(100), bar_format="{percentage:.0f}%"):
    pass

This will display only the percentage, without other details like time or iteration count.

Q10: Can I log messages to the console while using TQDM without breaking the progress bar?

A10: Yes, you can log messages while using TQDM by calling tqdm.write() instead of print(). This ensures that the message is printed without interfering with the progress bar.

Example:

from tqdm import tqdm

for i in tqdm(range(10)):
    if i == 5:
        tqdm.write("Reached halfway point")

The progress bar will continue without being disrupted by the log message.

Table of Contents

What is TQDM in Python?

Key Features of TQDM:

Installing TQDM in Python

Installation:

How to Use TQDM in Python

1. Basic Usage of TQDM in a For Loop

Example: Basic For Loop with TQDM

Output:

2. TQDM with Lists and Iterators

Example: TQDM with a List

3. Using TQDM in Functions

Example: TQDM in a Function

4. Using TQDM with File Downloads

Example: Tracking a File Download with TQDM

Customizing the TQDM Progress Bar

1. Changing the Progress Bar Description

Example:

Output:

2. Adjusting the Update Interval

Example:

3. Customizing Progress Bar Format

Example:

Output:

TQDM in Jupyter Notebooks

Example: Using TQDM in a Jupyter Notebook

Best Practices for Using TQDM

1. Use TQDM for Long-Running Tasks

Example:

2. Combine TQDM with Logging

Example:

3. Use TQDM in Parallel Processing

Example: Using TQDM with concurrent.futures

Summary of Key Concepts

Exercises

FAQ

Q1: Can I use TQDM with functions that don’t use loops?

Example:

Q2: How can I stop TQDM from printing a new line after completion?

Example:

Q3: How can I use TQDM with asynchronous tasks or async functions?

Example:

Q4: How do I update the TQDM progress bar manually?

Example:

Q5: Can I use TQDM with pandas when processing dataframes?

Example:

Q6: Can I change the appearance of the progress bar, such as the bar character?

Example:

Q7: How can I use TQDM to track progress across multiple processes?

Example:

Q8: Can I nest TQDM progress bars (i.e., show progress for tasks inside other tasks)?

Example: Nested Progress Bars

Q9: Can I display the progress bar as a percentage only, without additional information?

Example: Showing Only Percentage

Q10: Can I log messages to the console while using TQDM without breaking the progress bar?

Example:

Similar Posts

Example: Using TQDM with `concurrent.futures`