Python Environment Variables: A Comprehensive Guide

In Python environment variables are dynamic values that the operating system and other programs use to influence the behavior of the software, including your Python applications.

Environment variables can store important configuration details such as file paths, database credentials, API keys, and more. Understanding how to use environment variables in Python is crucial for developing secure, portable, and configurable applications.

By the end of this guide, you will have a solid understanding of how to work with Python environment variables, enabling you to build more flexible and secure applications.

What Are Environment Variables?

Environment variables are key-value pairs that can be accessed by the operating system and applications running on the system. These variables provide essential information that can control the behavior of programs without requiring changes to the code itself. Common environment variables include paths to system directories, database connection strings, and API credentials.

Key Characteristics:

• Key-Value Format: Environment variables are stored as key-value pairs (e.g., DATABASE_URL=postgres://user:password@localhost/dbname).
• Operating System Agnostic: Environment variables are supported by all major operating systems, including Windows, Linux, and macOS.
• Dynamic: They can be changed at runtime, allowing programs to adjust their behavior based on the system environment.

Why Use Environment Variables in Python?

• Security: Environment variables are often used to store sensitive information such as API keys, database credentials, and passwords, avoiding hardcoding them in the source code.
• Portability: By using environment variables, you can configure your application to work in different environments (development, staging, production) without modifying the code.
• Configurability: Environment variables allow you to change application behavior dynamically based on the environment the application is running in.

How to Access Environment Variables in Python

Python provides a simple way to interact with environment variables through the os module. The os.environ dictionary contains all the environment variables available to the current process.

Accessing Environment Variables with os.environ

You can access an environment variable by referencing its key in the os.environ dictionary.

Example:

import os

# Accessing an environment variable
db_url = os.environ.get("DATABASE_URL")
print(f"Database URL: {db_url}")

In this example, os.environ.get("DATABASE_URL") retrieves the value of the DATABASE_URL environment variable. If the variable doesn’t exist, None is returned.

Setting Environment Variables in Python

You can also set or modify environment variables in Python using os.environ.

Example: Setting an Environment Variable

import os

# Set an environment variable
os.environ["MY_ENV_VAR"] = "my_value"

# Access the newly set environment variable
print(os.environ["MY_ENV_VAR"])  # Output: my_value

In this example, os.environ["MY_ENV_VAR"] = "my_value" sets a new environment variable named MY_ENV_VAR with the value my_value. You can then access it like any other environment variable.

Removing Environment Variables

To remove an environment variable, you can use del with os.environ.

Example: Removing an Environment Variable

import os

# Remove an environment variable
if "MY_ENV_VAR" in os.environ:
    del os.environ["MY_ENV_VAR"]
    print("MY_ENV_VAR removed.")
else:
    print("MY_ENV_VAR does not exist.")

In this example, the environment variable MY_ENV_VAR is deleted from the environment.

Best Practices for Using Environment Variables in Python

1. Use Environment Variables for Sensitive Information

One of the most important uses of environment variables is to store sensitive information such as API keys, database passwords, and credentials. Never hardcode sensitive data in your Python code. Instead, use environment variables to keep your sensitive data secure and out of source control.

Example: Accessing an API Key from an Environment Variable

import os

# Accessing an API key stored in an environment variable
api_key = os.environ.get("API_KEY")
if api_key:
    print("API key found!")
else:
    print("API key not found.")

In this example, the API key is stored in an environment variable and accessed securely at runtime.

2. Use Default Values for Missing Environment Variables

When accessing environment variables, always provide a default value in case the variable is not set. This avoids errors if the environment variable is missing.

Example: Providing a Default Value

import os

# Access environment variable with a default value
db_url = os.environ.get("DATABASE_URL", "postgres://localhost/defaultdb")
print(f"Database URL: {db_url}")

In this example, if DATABASE_URL is not set, a default value postgres://localhost/defaultdb is used.

3. Avoid Setting Environment Variables Programmatically for Production

While setting environment variables programmatically in Python (via os.environ) is useful for local development or testing, it’s best to set environment variables at the operating system or container level in production environments. This allows for better security and flexibility.

How to Set Environment Variables in Different Operating Systems

1. Setting Environment Variables in Windows

In Windows, you can set environment variables temporarily for the session using the Command Prompt or Powershell, or permanently through the System Properties interface.

Temporarily Setting an Environment Variable in Command Prompt:

set MY_ENV_VAR=my_value

This sets the MY_ENV_VAR variable for the current session. It will be accessible in Python as long as the session is active.

2. Setting Environment Variables in Linux and macOS

In Linux and macOS, you can set environment variables temporarily in the terminal or permanently in shell configuration files such as .bashrc, .zshrc, or .bash_profile.

Temporarily Setting an Environment Variable in the Terminal:

export MY_ENV_VAR=my_value

This sets MY_ENV_VAR for the current session.

Permanently Setting Environment Variables:

To set an environment variable permanently, add the following line to ~/.bashrc or ~/.zshrc:

export MY_ENV_VAR=my_value

Then, reload the shell configuration:

source ~/.bashrc

Using Environment Variables with dotenv

The dotenv package is a popular tool for managing environment variables in Python projects. It allows you to store environment variables in a .env file, making it easier to manage and load them during development.

Installing dotenv:

You can install python-dotenv using pip:

pip install python-dotenv

Example: Using dotenv to Load Environment Variables from a .env File

1. Create a .env file in the root of your project:

   API_KEY=my_secret_key
   DATABASE_URL=postgres://user:password@localhost/dbname

2. Use dotenv in your Python script to load the environment variables:

   import os
   from dotenv import load_dotenv

   # Load environment variables from the .env file
   load_dotenv()

   # Access the environment variables
   api_key = os.getenv("API_KEY")
   db_url = os.getenv("DATABASE_URL")

   print(f"API Key: {api_key}")
   print(f"Database URL: {db_url}")

In this example, the .env file is used to store environment variables, and dotenv loads them into the Python environment.

Common Pitfalls When Using Environment Variables

1. Forgetting to Set Environment Variables

If you try to access an environment variable that isn’t set, Python will return None (if you use os.environ.get()) or raise a KeyError (if you access it directly through os.environ[]). Always use get() with a default value to avoid unexpected errors.

2. Committing .env Files to Version Control

Never commit your .env files to version control repositories like Git, as these files often contain sensitive information such as API keys and database credentials. Always add .env to your .gitignore file to prevent it from being tracked by Git.

Summary of Key Concepts

• Environment variables are key-value pairs that influence the behavior of programs, and they are widely used for storing configuration details and sensitive information.
• Use os.environ.get() to safely access environment variables in Python.
• Set environment variables using os.environ[] in Python, but avoid doing this in production environments.
• dotenv is a useful tool for loading environment variables from a .env file during development.
• Always provide default values when accessing environment variables and avoid hardcoding sensitive data in your code.

Exercises

1. Access Environment Variables: Write a Python script that accesses and prints the value of a custom environment variable (e.g., MY_API_KEY) using both os.environ and dotenv.
2. Set Environment Variables: Set an environment variable at the operating system level (Windows or Linux/macOS), and access it in Python using os.environ.
3. Check for Missing Environment Variables: Write a Python function that checks if critical environment variables like DATABASE_URL or API_KEY are set and prints an error message if they are missing.

Check out our FREE Learn Python Programming Masterclass to hone your skills or learn from scratch.

The course covers everything from first principles to Graphical User Interfaces and Machine Learning

You can read about the Python environment in the official Python documents.

FAQ

Q1: What happens if I try to access an environment variable that doesn’t exist?

A1: If you use os.environ.get(), and the environment variable does not exist, it will return None by default, avoiding errors. However, if you access the variable directly using os.environ["VAR_NAME"] and it doesn’t exist, Python will raise a KeyError.

Example:

import os

# Safe access (returns None if not found)
value = os.environ.get("NON_EXISTENT_VAR")
print(value)  # Output: None

# Unsafe access (raises KeyError if not found)
value = os.environ["NON_EXISTENT_VAR"]  # Raises KeyError

Q2: How can I check if an environment variable is set?

A2: You can check if an environment variable is set by using os.environ.get(). If it returns None, the variable is not set. You can also use the in operator to check if the environment variable exists in os.environ.

Example:

import os

# Check if the environment variable exists
if "API_KEY" in os.environ:
    print("API_KEY is set.")
else:
    print("API_KEY is not set.")

Q3: Can I store non-string values (e.g., integers, booleans) in environment variables?

A3: Environment variables are stored as strings, so you cannot directly store non-string values like integers or booleans. If you need to store an integer or boolean, convert it to a string before setting the environment variable and then convert it back when retrieving it.

Example: Storing and Accessing an Integer

import os

# Set an integer as an environment variable (convert to string)
os.environ["MAX_RETRIES"] = str(5)

# Access and convert back to integer
max_retries = int(os.environ.get("MAX_RETRIES", "3"))  # Default to 3 if not set
print(max_retries)

Q4: How can I permanently set an environment variable in my operating system?

A4: To set an environment variable permanently, you need to set it in the operating system, not through Python. Here’s how to do it:

• Windows:

1. Open System Properties > Environment Variables.
2. Add a new variable under User or System environment variables.

• Linux/macOS:
  Add the environment variable to your shell configuration file (e.g., .bashrc, .zshrc, or .bash_profile):

  export MY_ENV_VAR="my_value"

Then, run source ~/.bashrc to apply the changes.

Q5: Can I use environment variables with virtual environments in Python?

A5: Yes, environment variables work seamlessly with Python virtual environments. When you activate a virtual environment, the environment variables are inherited from the system unless explicitly overwritten. You can also set environment variables specific to a virtual environment by modifying the environment within that session.

Q6: Is it safe to include .env files in version control (e.g., Git)?

A6: No, it’s not safe to include .env files in version control if they contain sensitive information like API keys, passwords, or database credentials. Always add .env to your .gitignore file to prevent it from being committed to Git. This ensures that sensitive information stays out of your source code repository.

Q7: What is the best way to handle missing environment variables?

A7: The best way to handle missing environment variables is to:

1. Use os.environ.get() and provide a default value in case the variable is missing.
2. Raise a custom error or log a message if critical environment variables like API keys or database URLs are missing.

Example:

import os

# Check if a critical environment variable is missing
db_url = os.environ.get("DATABASE_URL")
if not db_url:
    raise RuntimeError("DATABASE_URL is not set!")

Q8: Can I update environment variables during runtime in a Python program?

A8: Yes, you can update environment variables during runtime using os.environ[], but these changes only persist for the duration of the script. Once the script ends, the environment variables are reset to their original values. Changes made in the script do not affect the system’s global environment variables.

Q9: How do I unset or delete an environment variable in Python?

A9: To remove an environment variable during the execution of your Python program, use the del keyword with os.environ.

Example:

import os

# Unset (delete) an environment variable
if "MY_ENV_VAR" in os.environ:
    del os.environ["MY_ENV_VAR"]
    print("Environment variable removed.")

Q10: Can I use environment variables in Python for configuring different environments (development, production, etc.)?

A10: Yes, environment variables are commonly used to configure different environments (e.g., development, testing, production) without changing the code. You can set different values for variables like DEBUG, DATABASE_URL, or API_KEY depending on the environment your application is running in. For example, you might have a different DATABASE_URL for your development environment compared to production.