Python's Requests Library (Guide)

Work With Status Codes

The first piece of information that you can gather from Response is the status code. A status code informs you of the status of the request.

For example, a 200 OK status means that your request was successful, while a 404 NOT FOUND status means that the resource you were looking for wasn’t found. There are many other possible status codes as well to give you specific insights into what happened with your request.

By accessing .status_code, you can see the status code that the server returned:

>>> response.status_code
200

.status_code returned 200, which means that your request was successful and the server responded with the data you requested.

Sometimes, you might want to use this information to make decisions in your code:

if response.status_code == 200:
    print("Success!")
elif response.status_code == 404:
    print("Not Found.")

With this logic, if the server returns a 200 status code, then your program will print Success!. If the result is a 404, then your program will print Not Found.

Requests goes one step further in simplifying this process for you. If you use a Response instance in a Boolean context, such as a conditional statement, then it’ll evaluate to True when the status code is less than 400, and False otherwise.

That means you can modify the last example by rewriting the if statement:

if response:
    print("Success!")
else:
    raise Exception(f"Non-success status code: {response.status_code}")

In the code snippet above, you implicitly check whether the .status_code of response is between 200 and 399. If it’s not, then you raise an exception with an error message that includes the non-success status code wrapped in an f-string.

Keep in mind that this method does not verify whether the status code is equal to 200. This is because other status codes within the 200 to 399 range, such as 204 NO CONTENT and 304 NOT MODIFIED, are also considered successful because they provide some workable response.

For example, the status code 204 tells you that the response was successful, but there’s no content to return in the message body.

So, make sure you use this convenient shorthand only if you want to know whether the request was generally successful. Then, if necessary, you’ll need to handle the response appropriately based on the status code.

Suppose you don’t want to check the response’s status code in an if statement. Instead, you want to use Request’s built-in capacities to raise an exception if the request was unsuccessful. You can do this using .raise_for_status():

import requests
from requests.exceptions import HTTPError

URLS = ["https://api.github.com", "https://api.github.com/invalid"]

for url in URLS:
    try:
        response = requests.get(url)
        response.raise_for_status()
    except HTTPError as http_err:
        print(f"HTTP error occurred: {http_err}")
    except Exception as err:
        print(f"Other error occurred: {err}")
    else:
        print("Success!")

If you invoke .raise_for_status(), then Requests will raise an HTTPError for status codes between 400 and 600. If the status code indicates a successful request, then the program will proceed without raising that exception.

Now, you know a lot about how to deal with the status code of the response you receive back from the server. However, when you make a GET request, you rarely only care about the status code of the response. Usually, you want to see more. Next, you’ll learn how to view the actual data that the server sent back in the body of the response.

Access the Response Content

The response of a GET request often has some valuable information, known as a payload, in the message body. Using the attributes and methods of Response, you can view the payload in a variety of formats.

To see the response’s content in bytes, you use .content:

>>> import requests

>>> response = requests.get("https://api.github.com")
>>> response.content
b'{"current_user_url":"https://api.github.com/user", ...}'

>>> type(response.content)
<class 'bytes'>

While .content gives you access to the raw bytes of the response payload, you’ll often want to convert them into a string using a character encoding such as UTF-8. response will do that for you when you access .text:

>>> response.text
'{"current_user_url":"https://api.github.com/user", ...}'

>>> type(response.text)
<class 'str'>

Because the decoding of bytes to a str requires an encoding scheme, Requests will try to guess the encoding based on the response’s headers if you don’t specify one. You can provide an explicit encoding by setting .encoding before accessing .text:

>>> response.encoding = "utf-8"  # Optional: Requests infers this.
>>> response.text
'{"current_user_url":"https://api.github.com/user", ...}'

If you take a look at the response, then you’ll see that it’s actually serialized JSON content. To get a dictionary, you could take the str that you retrieved from .text and deserialize it using json.loads(). However, the direct way to accomplish this task is to use .json():

>>> response.json()
{'current_user_url': 'https://api.github.com/user', ...}

>>> type(response.json())
<class 'dict'>

The type of the return value of .json() is a dictionary, so you can access values in the object by key:

>>> response_dict = response.json()
>>> response_dict["emojis_url"]
'https://api.github.com/emojis'

You can do a lot with status codes and message bodies. But if you need more information, like metadata about the response itself, you’ll need to look at the response’s headers.

View Response Headers

The response headers can give you useful information, such as the content type of the response payload and how long to cache the response. To view these headers, access .headers:

>>> import requests

>>> response = requests.get("https://api.github.com")
>>> response.headers
{'Server': 'github.com',
...
'X-GitHub-Request-Id': 'AE83:3F40:2151C46:438A840:65C38178'}

The .headers attribute returns a dictionary-like object, allowing you to access header values by key. For example, to see the content type of the response payload, you can access "Content-Type":

>>> response.headers["Content-Type"]
'application/json; charset=utf-8'

There’s something special about this dictionary-like headers object. The HTTP specification defines headers as case-insensitive, which means you can access them without worrying about their capitalization:

>>> response.headers["content-type"]
'application/json; charset=utf-8'

Whether you use the key "content-type" or "Content-Type", you’ll get the same value.

Now that you’ve seen the most useful attributes and methods of Response in action, you already have a good overview of Requests’ basic usage. You can get content from the internet and work with the response that you receive.

But there’s more to the internet than plain, straightforward URLs. In the next section, you’ll take a step back and see how your responses change when you customize your GET requests to account for query string parameters.