Python Copy File: Comprehensive Guide

Most Python applications have have some interaction with files, and the Python copy file operation is one of the most common.

Copying files from one location to another is often essential in applications such as backups, data migrations, or organizing directories. Python provides several built-in modules and methods that make it easy to copy files efficiently and reliably.

By the end of this guide, you’ll have a thorough understanding of how to copy files using Python and when it’s best to use each method.

Methods for Copying Files in Python

Python offers multiple ways to copy files, but the most commonly used approach is through the shutil module. The shutil module provides high-level file operations, including several methods for copying files. Let’s dive into each of these methods and their use cases.

1. Copying Files Using shutil.copy()

The shutil.copy() function is one of the most commonly used methods to copy files in Python. It copies the file contents from a source file to a destination file or directory. This method does not copy the metadata (such as file permissions or timestamps); it only copies the data.

Syntax:

import shutil

shutil.copy(src, dst)

• src: The source file you want to copy.
• dst: The destination file or directory where you want to copy the file.

Example: Basic File Copying

import shutil

# Copying a file from one location to another
shutil.copy('source.txt', 'destination.txt')

In this example, the file source.txt is copied to destination.txt. If destination.txt does not exist, it will be created. If it does exist, it will be overwritten.

2. Copying Files Along with Metadata Using shutil.copy2()

If you need to copy both the file contents and metadata (such as timestamps and permissions), use the shutil.copy2() method. This function is similar to shutil.copy(), but it preserves the original file’s metadata.

Syntax:

import shutil

shutil.copy2(src, dst)

Example: Copying File with Metadata

import shutil

# Copying a file with its metadata
shutil.copy2('source.txt', 'destination_with_metadata.txt')

This method copies both the file’s contents and its metadata to the destination.

3. Copying Files to a Directory Using shutil.copy()

You can copy a file to a directory by specifying the directory path as the destination. The original file name will be retained in the new directory.

Example: Copying a File to a Directory

import shutil

# Copy a file to the 'backup/' directory
shutil.copy('source.txt', 'backup/')

If backup/ is a directory, the file source.txt will be copied into it, retaining the original file name.

4. Copying File Permissions Only Using shutil.copymode()

If you only want to copy the file permissions (such as read/write/execute permissions) without copying the contents or metadata, use the shutil.copymode() function.

Syntax:

import shutil

shutil.copymode(src, dst)

Example: Copying File Permissions

import shutil

# Copy the file permissions from source.txt to destination.txt
shutil.copymode('source.txt', 'destination.txt')

In this example, only the file permissions are copied from source.txt to destination.txt. The file contents and metadata remain unchanged.

5. Copying an Entire Directory Using shutil.copytree()

If you need to copy an entire directory, including all of its subdirectories and files, use the shutil.copytree() method. This method recursively copies all files and directories from the source directory to the destination.

Syntax:

import shutil

shutil.copytree(src, dst)

• src: The source directory.
• dst: The destination directory.

Example: Copying a Directory

import shutil

# Copy the entire 'my_folder' directory to 'backup_folder'
shutil.copytree('my_folder', 'backup_folder')

This will copy all the contents of my_folder (including subdirectories) to backup_folder. If backup_folder does not exist, it will be created.

6. Copying Files Using os Module

While the shutil module is preferred for copying files, the os module can also be used for lower-level file operations. However, the os module does not have a direct method to copy files; instead, you can manually open the source file, read its contents, and write them to the destination file.

Example: Copying a File with os

import os

# Copy file manually using os
with open('source.txt', 'rb') as src_file:
    with open('destination.txt', 'wb') as dst_file:
        dst_file.write(src_file.read())

This method reads the contents of source.txt and writes them to destination.txt. While this approach works, it is less efficient and flexible than using shutil.

Error Handling When Copying Files

It is important to handle errors when copying files, especially when dealing with file permissions, non-existent files, or file system restrictions. Python’s shutil functions raise exceptions, such as FileNotFoundError and PermissionError, when issues arise.

Example: Error Handling with Try-Except

import shutil

try:
    shutil.copy('source.txt', 'destination.txt')
    print("File copied successfully!")
except FileNotFoundError:
    print("Source file not found.")
except PermissionError:
    print("Permission denied. Unable to copy file.")

This example uses a try-except block to handle errors that might occur during the file copy process.

Best Practices for Copying Files in Python

1. Use shutil.copy2() When Metadata Preservation Is Required

When you need to retain file metadata (such as timestamps or permissions), always use shutil.copy2(). This method ensures that the file’s attributes are preserved during the copy.

Example:

shutil.copy2('source.txt', 'destination.txt')

2. Handle Exceptions with Try-Except Blocks

Always use exception handling (try-except) when performing file operations. This will help catch and manage potential errors such as missing files or insufficient permissions.

3. Use os.path.exists() to Check if Files Exist

Before copying a file, check whether the source file exists to avoid unnecessary errors.

Example:

import os
import shutil

if os.path.exists('source.txt'):
    shutil.copy('source.txt', 'destination.txt')
else:
    print("Source file does not exist.")

Common Pitfalls to Avoid

1. Overwriting Files Without Warning

By default, shutil.copy() will overwrite the destination file if it already exists. To avoid accidental overwrites, you can check whether the destination file exists before copying.

Example:

import os
import shutil

if not os.path.exists('destination.txt'):
    shutil.copy('source.txt', 'destination.txt')
else:
    print("Destination file already exists.")

2. Forgetting to Use shutil.copy2() for Metadata

If you need to retain file metadata (such as timestamps), always use shutil.copy2(). Using shutil.copy() alone will not preserve metadata.

3. Copying Open Files

Attempting to copy files that are currently open or in use can result in errors, especially on operating systems like Windows. Ensure the file is not in use before copying it.

Practical Examples

1. Backup Script: Copying Multiple Files to a Backup Directory

import shutil
import os

def backup_files(file_list, backup_dir):
    # Ensure the backup directory exists
    if not os.path.exists(backup_dir):
        os.makedirs(backup_dir)

    for file in file_list:
        if os.path.exists(file):
            shutil.copy(file, backup_dir)
            print(f"Copied {file} to {backup_dir}")
        else:
            print(f"File {file} does not exist.")

# Example usage
files_to_backup = ['file1.txt', 'file2.txt', 'file3.txt']
backup_directory = 'backup/'
backup_files(files_to_backup, backup_directory)

2. Copying a File and Renaming It

import shutil

# Copy the file and rename it at the destination
shutil.copy('source.txt', 'renamed_file.txt')

This example copies source.txt and renames the copied file to renamed_file.txt.

Summary of Key Concepts

• Use shutil.copy() to copy file contents without metadata.
• Use shutil.copy2() to copy both file contents and metadata (such as timestamps and permissions).
• To copy an entire directory, use shutil.copytree().
• Always handle potential errors, such as missing files or permission issues, with try-except blocks.
• Use shutil.copymode() to copy file permissions without copying the contents.

Exercises

1. Basic File Copy: Write a Python function that takes a source file and a destination directory as input and copies the file to the directory.
2. Backup Directory: Create a script that copies all files from one directory to another, preserving file metadata.
3. Error Handling: Modify a file copy script to handle errors such as non-existent source files or permission errors, and log these errors to a file.

Check out our FREE Learn Python Programming Masterclass to hone your skills or learn from scratch.

The course covers everything from first principles to Graphical User Interfaces and Machine Learning

Remember, you can always refer to the official Python documentation. Here is the entry on high-level file operations.

FAQ

Q1: What is the difference between shutil.copy() and shutil.copy2()?

A1: The main difference between shutil.copy() and shutil.copy2() is that shutil.copy() copies only the file’s contents, while shutil.copy2() copies both the file’s contents and its metadata (such as file permissions, modification times, and other attributes). If you need to retain the original file’s metadata, you should use shutil.copy2().

Q2: How can I avoid overwriting files when copying?

A2: To avoid overwriting files when copying, you can check if the destination file already exists using os.path.exists() before copying. If the file exists, you can choose not to copy or rename the destination file.

Example:

import os
import shutil

if not os.path.exists('destination.txt'):
    shutil.copy('source.txt', 'destination.txt')
else:
    print("Destination file already exists.")

Q3: How do I copy only certain types of files (e.g., only .txt files) from a directory?

A3: To copy only files of a specific type, such as .txt files, you can iterate over the files in the directory and check their extensions using os.path.splitext() or str.endswith() before copying.

Example:

import os
import shutil

source_dir = 'source_folder/'
dest_dir = 'destination_folder/'

for file_name in os.listdir(source_dir):
    if file_name.endswith('.txt'):
        shutil.copy(os.path.join(source_dir, file_name), dest_dir)

This code copies only .txt files from source_folder to destination_folder.

Q4: How can I copy files between two remote locations or machines?

A4: To copy files between remote locations or machines, Python’s built-in modules like shutil and os won’t work directly because they only handle local file systems. For remote file copying, you can use libraries like paramiko (for SFTP) or scp (for SSH-based copying).

Example with Paramiko (SFTP):

import paramiko

# Set up SFTP client
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('remote_host', username='user', password='password')

sftp = ssh.open_sftp()
sftp.put('local_file.txt', '/remote/path/remote_file.txt')  # Copy from local to remote
sftp.get('/remote/path/remote_file.txt', 'local_file.txt')  # Copy from remote to local
sftp.close()
ssh.close()

Q5: Can I copy a file and rename it in the destination directory?

A5: Yes, you can copy a file and rename it by specifying the new file name in the destination path.

Example:

import shutil

# Copy file and rename it at the destination
shutil.copy('source.txt', 'destination_folder/renamed_file.txt')

This will copy source.txt to destination_folder and rename it to renamed_file.txt.

Q6: How can I copy hidden files (files starting with a dot, e.g., .config)?

A6: Hidden files in Unix-based systems (like Linux or macOS) are files that start with a dot (.). They are treated like regular files by Python’s shutil functions, so you can copy them like any other file. Just make sure to specify the full file name, including the dot, when copying.

Example:

import shutil

# Copy a hidden file
shutil.copy('.config', 'backup/.config')

Q7: What happens if the source file is currently being used or open when I try to copy it?

A7: If the source file is open or being used by another process, especially on systems like Windows, you might get a PermissionError when trying to copy it. To avoid this, ensure that the file is not being used during the copy operation. If the file must remain open during the copy process, consider handling errors using a try-except block.

Example:

import shutil

try:
    shutil.copy('source.txt', 'destination.txt')
except PermissionError:
    print("The source file is currently in use.")

Q8: Can I copy files with progress feedback (like a progress bar)?

A8: Python’s shutil module does not provide built-in progress tracking for file copying. However, you can implement a progress bar using the tqdm library by reading and copying the file in chunks and updating the progress bar as each chunk is copied.

Example with Progress Bar:

import shutil
import os
from tqdm import tqdm

def copy_with_progress(src, dst):
    total_size = os.path.getsize(src)
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst, tqdm(total=total_size, unit='B', unit_scale=True, desc=dst) as pbar:
        while True:
            chunk = fsrc.read(1024 * 1024)  # Read in 1 MB chunks
            if not chunk:
                break
            fdst.write(chunk)
            pbar.update(len(chunk))

# Example usage
copy_with_progress('source.txt', 'destination.txt')

Q9: How can I copy a file to a destination with different file permissions?

A9: By default, shutil.copy() copies file contents but does not preserve file permissions. If you want to modify the file permissions after copying, you can use the os.chmod() function to set custom file permissions.

Example:

import shutil
import os

shutil.copy('source.txt', 'destination.txt')

# Set read-only permissions for the copied file
os.chmod('destination.txt', 0o444)

Q10: How do I copy only newer files to a backup directory?

A10: To copy only newer files (i.e., files that have been modified more recently than those in the backup directory), you can compare the modification times of the source and destination files using os.path.getmtime().

Example:

import os
import shutil

def copy_if_newer(src, dst):
    if not os.path.exists(dst) or os.path.getmtime(src) > os.path.getmtime(dst):
        shutil.copy(src, dst)
        print(f"Copied {src} to {dst}")
    else:
        print(f"{dst} is up-to-date")

# Example usage
copy_if_newer('source.txt', 'backup/source.txt')