Files are an essential part of working with computers, thus using Python to write to and read from a file are basic skills that you need to master. In this article, I’ll show you how to do the things you came here for, e.g.:
- How to open a file in Python
- Reading a file with Python (both at once or line-by-line)
- Writing to a file with Python
- Why you should specify the encoding
- Working with files the modern way, using pathlib
- Copy, move, rename, and delete files
- Check if a file or directory exists
When working with files, there will come that point where you need to know about file modes and permissions. I included an explanation on this topic as well, in case you need it.
Open a file in Python
In Python, we open a file with the open() function. It’s part of Python’s built-in functions, you don’t need to import anything to use open(). The open() function expects at least one argument: the file name. If the file was successfully opened, it returns a file object that you can use to read from and write to that file.
As soon as you open a file with Python, you are using system resources that you need to free once you’re done. If you don’t, you create a so-called resource leak. If you do this often enough, you might run into problems, because there’s a limit to the number of open files a regular operating system user can have.
This doesn’t have to be a problem though. When your program exits, all resources are freed up automatically. That’s nice and considerate of the Python devs, but a lot of software keeps running indefinitely. With such software, you need to be more careful and close the resources you don’t need anymore. This way, you make sure other software can access the file safely. Also, when you open a lot of files, each open file takes up memory and other resources. For these reasons, it’s a good habit to close a file when you’re done with it.
Closing a file used to be a manual task, that can be forgotten easily. Luckily, Python has the ‘with’ statement to help us out. We’ll discuss the with statement soon. I first want to show you the ‘old-fashioned’ way of doing things.
The old fashioned way to open a file
To illustrate why the with statement is so useful, let’s first open and close a file in the traditional, manual way. For these examples, I created a small file called text.txt with three lines of text, in the same directory as the script:
f = open('text.txt')
print(f.read())
f.close()Code language: Python (python)Output:
This is line 1 This is another line This is the last one, bye!
If you’re feeling adventurous, try and remove the f.close(). It should work the same; Python won’t complain and will close the file resource on exit.
Python read file content at once
In the example above, you can see how we read all contents from the file so we can print it. If you want to read all the file content into a single string, at once, use the read() method on a file object, without arguments. This is no problem for small files, but do realize that you load all the contents into memory at once. This could be a problem with large files, but we’ll get to that soon.
Using with open() in Python
There’s a potential problem in the previous example. If an exception occurs while Python reads the file, the f.close() call will never be reached. And this is a trivial example, but you can imagine that with more complicated software, in which exceptions might get caught so the software can keep running, it can go wrong quickly.
Another problem that may arise, is that you simply forget to write the close() call. It happens, we are all humans. Therefore, with modern Python, it’s recommended to use the with-statement when you can. By using Python’s with open(), the opened file resource will only be available in the indented block of code:
with open('text.txt') as f:
text = f.read()
print(text)Code language: Python (python)Output:
This is line 1 This is another line This is the last one, bye!
In this example, the with-statement makes sure the file is closed, even if an exception occurs. That’s because Python closes the resource automatically as soon as we step out of the scope of this with-statement. Due to this automatic close, you can not use the file (called f in our case) outside of the with-statement. If you want to use it again, you have to open it again.
File modes
So far, we used the open() function with just one argument: the file name. It’s a mandatory argument, but open() also has a number of optional extra arguments, like the file mode. This mode defaults to ‘rt’, which stands for ‘read text’. You’re opening the file for reading, which means you can not write to it, and you’re expecting it to be a text file.
Other modes that you can pass are listed in the following table:
| Character | Meaning |
| r | open file for reading (default) |
| w | open file for writing, truncating the file first |
| x | create a new file and open it for writing |
| a | Open for writing, append to the end of the file if it exists already |
| t | Text mode (the default), can be used in combination with rwxa |
| b | Binary mode (as opposed to text mode), can be used in combination with rwxa |
| + | open a disk file for updating (reading and writing) |
I won’t go into the details of all the modes here but instead, I’ll explain and demonstrate most of them in the appropriate sections below.
Specify the encoding
Text files are stored as bytes, and the encoding decides how those bytes turn into characters and back. Nowadays, almost every text file is UTF-8. Since Python 3.15, that’s also what open() uses by default on every operating system.
Older Python versions use the encoding of your system’s locale instead. On Linux and macOS, that’s usually UTF-8 too, but on Windows it often isn’t. A file written on one computer then reads back as garbage on another, or raises a UnicodeDecodeError. Here’s what a UTF-8 file looks like when it’s read with cp1252, a common Windows encoding:
with open('cafe.txt', 'w', encoding='utf-8') as f:
f.write('Café ☕\n')
with open('cafe.txt', encoding='cp1252') as f:
print(f.read())Code language: Python (python)Output:
Café ☕
So if your code might run on an older Python version, pass the encoding explicitly whenever you open a text file: open('text.txt', encoding='utf-8'). It costs you a few characters and makes your code behave the same everywhere. I left it out of most examples below to keep them short.
Python write file
Now that you know about file modes and how to open a file, we can use Python to write to files as well. We can do so in three steps:
- First, we need to determine the file mode. If you look at the table above, we’ll need to use ‘w’ and ‘t’. Since ‘t’ is the default, we can leave it out.
- Next, we need to open the file for writing
- And finally, we call the write() method on our file object.
Write expects one argument in this case: a Python string. If you have data of another type, like a number, you need to manually convert it to a string. After all, text files don’t have the concept of data types: they simply contain text.
In the example below, we convert integers to strings using the str() function, and write them to the file:
with open('test.txt', 'w') as f:
for i in range(1, 5):
f.write(str(i))
with open('test.txt', 'r') as f:
print(f.read())Code language: Python (python)There is now a file on your file system (in the same directory as your script), with the following content:
1234Newline characters
Did you expect each number to be on a new line?
When writing files, you need to explicitly specify newline characters. To try this on your own computer, adapt the code to make it look like this:
with open('test.txt', 'w') as f:
for i in range(1, 5):
f.write(f'Number {i}\n')
with open('test.txt', 'r') as f:
print(f.read())Code language: Python (python)Output:
Number 1 Number 2 Number 3 Number 4
Note that I used an f-string here. I personally like them a lot, but you can also use something like str(i) + '\n'.
Python append to a file
Now that we have a test file, we can also append some extra data. By default, when we open a file for writing, we overwrite anything that’s already present. So, again, we first look in the file modes table above which mode we can use to append data to our file. We need mode ‘a’, for append.
In the following example, we:
- Write to a file just like in the previous example using Python’s
with open(). - After that, we open the file again, this time with the append flag, and add some extra lines.
- Read back the file contents and print to screen so we can inspect the result
Here you go:
# First create a file, with a couple of lines
with open('test.txt', 'w') as f:
for i in range(1, 5):
f.write(f'Number {i}\n')
# Now add some extra lines using append mode
with open('test.txt', 'a') as f:
for i in range(5, 8):
f.write(f'Append number {i}\n')
with open('test.txt') as f:
print(f.read())Code language: Python (python)Output:
Number 1 Number 2 Number 3 Number 4 Append number 5 Append number 6 Append number 7
Python read file to list
With small files, it can be convenient to read all lines at once into a list. There are two ways to do this:
with open('test.txt') as f:
lines = list(f)
# lines = ['Number 1\n', 'Number 2\n', 'Number 3\n', 'Number 4\n',
# 'Append number 5\n', 'Append number 6\n', 'Append number 7\n']Code language: Python (python)Is equivalent to:
with open('test.txt') as f:
lines = f.readlines()
# lines = ['Number 1\n', 'Number 2\n', 'Number 3\n', 'Number 4\n',
# 'Append number 5\n', 'Append number 6\n', 'Append number 7\n']Code language: Python (python)Read file line-by-line
You don’t always have the luxury of reading files at once. For instance, when you need to work with 140 GB-sized log files, you can’t read it all into memory and get away with it. Your program will likely crash, or at least become extremely slow.
It is much more efficient to parse the file line-by-line in such cases. To read a file line by line, we can treat the opened file as an iterator. After opening the file, we can use a Python for-loop to iterate over the lines:
with open('test.txt', 'r') as f:
for line in f:
print(line, end='')Code language: Python (python)Output:
Number 1 Number 2 Number 3 Number 4 Append number 5 Append number 6 Append number 7
Each line you read still ends with its newline character, \n. And print() adds a newline of its own, so a plain print(line) would put an empty line between every line of the file. That’s why I pass end='', which tells print not to add one. If you want to work with the text itself, remove the newline with line.rstrip() or line.rstrip('\n').