Reading and Writing Files in Python (with examples)

In this blog post, let's dive into how to read and write files in Python. If you're using Python, sooner or later you're definitely going to come across a situation where you need to read or write files. Whether you're handling data for a project, logging information, or just organizing your thoughts, file operations are essential.

I'll walk you through this in simple terms, starting with the basics and then moving on to more complex examples. By the end of this post, you should have a better understanding of how to manage files effectively in Python. And remember, we're going to focus on using the context manager with open, which is a neat and efficient way to handle files. So, let's get started!

Reading a File in Python

Reading a file in Python is straightforward, especially with the with open statement. Let's start with a very simple example. Assume we have a text file named testfile.txt in the same directory as our Python script. This is important because it makes referring to the file in our script much easier. Here’s a simple script to read the content of testfile.txt

with open('testfile.txt', 'r') as file:
    content = file.read()
    print(content)

In this script, with open('testfile.txt', 'r') as file: is the key line. The with open part is known as a context manager, and it takes care of opening and then automatically closing the file after you’re done with it. This is much more efficient and cleaner than having to manually close the file. 'testfile.txt' is, of course, the name of the file we're opening. The 'r' indicates the mode in which we open the file, where 'r' stands for 'read' mode. The as file is simply a way to refer to the file inside our block of code; you could name it anything, but 'file' is a clear and descriptive choice.

Next, the content = file.read() line reads the entire content of the file and stores it in the variable content. This approach is best suited for smaller files, as it loads the entire file content into memory.

Finally, print(content) does exactly what it says: it prints the content of the file that we just read to the console. When this script is run, it will open testfile.txt, read everything in it, and print it out to the console.

Writing to a File in Python

Writing to a file in Python is as straightforward as reading from one. Just like in reading, we use the with open statement, but this time with a different mode. Let's say we want to write some text to a file named output.txt

with open('output.txt', 'w') as file:
    file.write("Hello, Python!")

In this script, the line with open('output.txt', 'w') as file: is where the magic starts. We're opening output.txt in 'write' mode, as indicated by 'w'. If output.txt doesn't exist, Python will create it for us. If it does exist, Python will overwrite it, so be careful with this mode! The as file part assigns the file object to the variable file for easy reference within our code block.

The next line, file.write("Hello, Python!"), writes the string "Hello, Python!" into our file. This method is straightforward and great for when you need to write text to a file in one go.

Remember, the 'write' mode will always overwrite the existing content of the file. If you want to add content to an existing file without overwriting it, you should use the 'append' mode, which we'll cover later. This basic example shows how simple it is to create and write content to files in Python.

Append Mode

Sometimes, you'll want to add content to an existing file without overwriting it. This is where the append mode, indicated by 'a', becomes useful.

with open('example.txt', 'a') as file:
    file.write("\nAdding more content.")

In this example, we open example.txt in append mode. Unlike the write mode 'w', which overwrites the entire file, append mode 'a' adds the new content to the end of the file. If example.txt doesn’t exist, Python will create it, just like in write mode.

The line file.write("\nAdding more content.") adds a new line of text to the end of the file. The \n at the beginning ensures that the new content starts on a new line.

Choosing between write ('w') and append ('a') modes depends on your specific needs. Use write mode when you want to create a new file or overwrite an existing file completely. It's useful when you have a fresh set of data that should replace the old one. On the other hand, append mode is perfect for cases where you need to continually add new data to an existing file, like logging events in a log file, or when you want to preserve existing data and just add to it.

Using 'try-except' for Safe File Operations

To handle potential errors (like a file not existing) gracefully, wrap your file operations in a try-except block. This prevents the script from crashing due to errors during file operations.

try:
    with open('somefile.txt', 'r') as file:
        print(file.read())
except FileNotFoundError:
    print("File not found")

If you don't use try-except block and the file doesn't exist for example, you will get the below error.

➜  read_write python read.py

Traceback (most recent call last):
  File "/Users/suresh/Documents/read_write/read.py", line 1, in <module>
    with open('testfil.txt', 'r') as file:
FileNotFoundError: [Errno 2] No such file or directory: 'testfil.txt'

Python 'readline()' vs 'readlines()'

The readline() method reads a single line from the file each time it's called. If you call it repeatedly, it will read the next line in each call until it reaches the end of the file. I have a test file called lines.txt as shown below. This method is particularly useful when you want to process a file line by line, without loading the entire file into memory. It's ideal for large files where memory efficiency is a concern.

line 1
line 2
line 3
line 4
line 5

with open('lines.txt', 'r') as file:
    line = file.readline()
    while line:
        print(line.strip())  # strip() removes the newline character
        line = file.readline()

On the other hand, readlines() reads all the lines in a file and returns them as a list of strings. Each string in the list represents a line in the file. readlines() is handy when you need to process all lines at once However, it can be memory-intensive for very large files since it loads the entire file into memory.

with open('lines.txt', 'r') as file:
    lines = file.readlines()
    for line in lines:
        print(line.strip())

In summary, use readline() for memory-efficient, sequential reading, especially for large files. Use readlines() when you need to work with all lines simultaneously, and memory isn't a concern.

The strip() method in these scripts serves an important purpose: it removes any leading and trailing whitespace from the string, which includes the newline character \n at the end of each line. When reading lines from a file, each line typically ends with a newline character to denote the end of that line. This character is also read into your string when you use readline() or readlines()