Python's Requests Library (Guide)

Status Codes

The first bit of information that you can gather from Response is the status code. A status code informs you of the status of the request.

For example, a 200 OK status means that your request was successful, whereas a 404 NOT FOUND status means that the resource you were looking for wasn’t found. There are many other possible status codes as well to give you specific insights into what happened with your request.

By accessing .status_code, you can see the status code that the server returned:

>>> response.status_code
200

.status_code returned 200, which means that your request was successful and the server responded with the data that you were requesting.

Sometimes, you might want to use this information to make decisions in your code:

if response.status_code == 200:
    print("Success!")
elif response.status_code == 404:
    print("Not Found.")

With this logic, if the server returns a 200 status code, then your program will print Success!. If the result is a 404, then your program will print Not Found.

Requests goes one step further in simplifying this process for you. If you use a Response instance in a conditional expression, then it’ll evaluate to True if the status code was smaller than 400, and False otherwise.

Therefore, you can simplify the last example by rewriting the if statement:

if response:
    print("Success!")
else:
    raise Exception(f"Non-success status code: {response.status_code}")

In the code snippet above, you implicitly check whether the .status_code of response is between 200 and 399. If it’s not, then you raise an exception that includes the non-success status code in an f-string.

Keep in mind that this method is not verifying that the status code is equal to 200. The reason for this is that other status codes within the 200 to 399 range, such as 204 NO CONTENT and 304 NOT MODIFIED, are also considered successful in the sense that they provide some workable response.

For example, the status code 204 tells you that the response was successful, but there’s no content to return in the message body.

So, make sure you use this convenient shorthand only if you want to know if the request was generally successful. Then, if necessary, you’ll need to handle the response appropriately based on the status code.

Let’s say you don’t want to check the response’s status code in an if statement. Instead, you want to use Request’s built-in capacities to raise an exception if the request was unsuccessful. You can do this using .raise_for_status():

import requests
from requests.exceptions import HTTPError

URLS = ["https://api.github.com", "https://api.github.com/invalid"]

for url in URLS:
    try:
        response = requests.get(url)
        response.raise_for_status()
    except HTTPError as http_err:
        print(f"HTTP error occurred: {http_err}")
    except Exception as err:
        print(f"Other error occurred: {err}")
    else:
        print("Success!")

If you invoke .raise_for_status(), then Requests will raise an HTTPError for status codes between 400 and 600. If the status code indicates a successful request, then the program will proceed without raising that exception.

Now, you know a lot about how to deal with the status code of the response that you got back from the server. However, when you make a GET request, you rarely only care about the status code of the response. Usually, you want to see more. Next, you’ll see how to view the actual data that the server sent back in the body of the response.

Content

The response of a GET request often has some valuable information, known as a payload, in the message body. Using the attributes and methods of Response, you can view the payload in a variety of different formats.

To see the response’s content in bytes, you use .content:

>>> import requests

>>> response = requests.get("https://api.github.com")
>>> response.content
b'{"current_user_url":"https://api.github.com/user", ...}'

>>> type(response.content)
<class 'bytes'>

While .content gives you access to the raw bytes of the response payload, you’ll often want to convert them into a string using a character encoding such as UTF-8. response will do that for you when you access .text:

>>> response.text
'{"current_user_url":"https://api.github.com/user", ...}'

>>> type(response.text)
<class 'str'>

Because the decoding of bytes to a str requires an encoding scheme, Requests will try to guess the encoding based on the response’s headers if you don’t specify one. You can provide an explicit encoding by setting .encoding before accessing .text:

>>> response.encoding = "utf-8"  # Optional: Requests infers this.
>>> response.text
'{"current_user_url":"https://api.github.com/user", ...}'

If you take a look at the response, then you’ll see that it’s actually serialized JSON content. To get a dictionary, you could take the str that you retrieved from .text and deserialize it using json.loads(). However, a simpler way to accomplish this task is to use .json():

>>> response.json()
{'current_user_url': 'https://api.github.com/user', ...}

>>> type(response.json())
<class 'dict'>

The type of the return value of .json() is a dictionary, so you can access values in the object by key:

>>> response_dict = response.json()
>>> response_dict["emojis_url"]
'https://api.github.com/emojis'

You can do a lot with status codes and message bodies. But, if you need more information, like metadata about the response itself, then you’ll need to look at the response’s headers.

Headers

The response headers can give you useful information, such as the content type of the response payload and a time limit on how long to cache the response. To view these headers, access .headers:

>>> import requests

>>> response = requests.get("https://api.github.com")
>>> response.headers
{'Server': 'GitHub.com',
...
'X-GitHub-Request-Id': 'AE83:3F40:2151C46:438A840:65C38178'}

.headers returns a dictionary-like object, allowing you to access header values by key. For example, to see the content type of the response payload, you can access "Content-Type":

>>> response.headers["Content-Type"]
'application/json; charset=utf-8'

There’s something special about this dictionary-like headers object, though. The HTTP specification defines headers as case-insensitive, which means that you’re able to access these headers without worrying about their capitalization:

>>> response.headers["content-type"]
'application/json; charset=utf-8'

Whether you use the key "content-type" or "Content-Type", you’ll get the same value.

Now that you’ve seen the most useful attributes and methods of Response in action, you already have a good overview of Requests’ basic usage. You can get content from the Internet and work with the response that you receive.

But there’s more to the Internet than plain and straightforward URLs. In the next section, you’ll take a step back and see how your responses change when you customize your GET requests to account for query string parameters.