Python & APIs: A Winning Combo for Reading Public Data

SOAP vs REST vs GraphQL

Even though some of the examples above are geared toward newer platforms or apps, web APIs have been around for quite a long time. In the late 1990s and early 2000s, two different design models became the norm in exposing data publicly:

1. SOAP (Simple Object Access Protocol) is typically associated with the enterprise world, has a stricter contract-based usage, and is mostly designed around actions.
2. REST (Representational State Transfer) is typically used for public APIs and is ideal for fetching data from the Web. It’s much lighter and closer to the HTTP specification than SOAP.

Nowadays, there’s a new kid in town: GraphQL. Created by Facebook, GraphQL is a very flexible query language for APIs, where the clients decide exactly what they want to fetch from the server instead of letting the server decide what to send.

If you want to learn more about the differences between these three design models, then here are a few good resources:

• What is SOAP?
• What is REST?
• API 101: SOAP vs. REST
• Introduction to GraphQL
• Comparing API Architectural Styles: SOAP vs REST vs GraphQL vs RPC

Even though GraphQL is on the rise and is being adopted by bigger and bigger companies, including GitHub and Shopify, the truth is that the majority of public APIs are still REST APIs. Therefore, for the purpose of this tutorial, you’ll learn only about REST APIs and how to consume them using Python.

APIs and requests: A Match Made in Heaven

When consuming APIs with Python, there’s only one library you need: requests. With it, you should be able to do most, if not all, of the actions required to consume any public API.

You can install requests by running the following command in your console:

$ python -m pip install requests

The examples in this tutorial have been tested with Python 3.11.2 and requests 2.31.0. However, any supported version of Python and requests should yield similar results.

Endpoints and Resources

As you saw above, the first piece of information necessary for consuming an API is the API URL, typically called the base URL. The base URL structure is no different from the URLs that you use for browsing Google, YouTube, or Facebook, though it usually contains the word api. This isn’t mandatory, just more of a rule of thumb.

For example, here are the base URLs for a few well-known API players:

• https://api.twitter.com
• https://api.github.com
• https://api.stripe.com

As you can see, all of the above start with https://api and include the remaining official domain, such as .twitter.com or .github.com. There’s no specific standard for how the API base URL should look, but it’s quite common for it to mimic this structure.

If you try opening any of the above links, then you’ll notice that most of them will return an error or ask for credentials. That’s because APIs sometimes require authentication steps before you can use them. You’ll learn more about this a bit later in the tutorial.

Next, using the Cat API, you’ll try to make a basic request to see how it may differ from the Random User Generator API that you tried above:

>>> import requests
>>> response = requests.get("https://api.thecatapi.com/")
>>> response.text
'{"message":"The Cat API","version":"1.2.5"}'

In this case, when calling the base URL, you get this generic message saying "The Cat API". This is because you’re calling the base URL, which is typically used for very basic information about an API, not the real data.

Calling the base URL alone isn’t a lot of fun, but that’s where endpoints come in handy. An endpoint is a part of the URL that specifies what resource you want to fetch. Well-documented APIs usually contain an API reference, which is extremely useful for knowing exactly which endpoints and resources an API has and how to use them.

You can check the official documentation to learn more about how to use the Cat API and what endpoints are available. In there, you’ll find a /breeds endpoint that you can use to fetch all the available breed resources or objects.

Now, give it a try in code locally using the breeds endpoint and some of the API knowledge that you already have:

>>> response = requests.get("https://api.thecatapi.com/v1/breeds")
>>> response.text
'[{"weight":{"imperial":"7  -  10","metric":"3 - 5"},"id":"abys",...}]'

There you go, your first breed listing using the Cat API!

If you’re a dog person, don’t fret. There’s an API for you, too, with the same endpoint but a different base URL:

>>> response = requests.get("https://api.thedogapi.com/v1/breeds")
>>> response.text
'[{...,"id":1,"name":"Affenpinscher",...}]'

I bet you’re already thinking about different ways that you can use these APIs to make some cute side project, and that’s the great thing about APIs. Once you start using them, there’s nothing stopping you from turning a hobby or passion into a fun project.

Before you move forward, one thing you need to know about endpoints is the difference between http:// and https://. In a nutshell, HTTPS is the encrypted version of HTTP, making all traffic between the client and the server much safer. When consuming public APIs, you should definitely stay away from sending any private or sensitive information to http:// endpoints and use only those APIs that provide a secure https:// base URL.

For more information on why it’s important to stick to HTTPS when browsing online, check out Exploring HTTPS With Python.

In the next section, you’ll dig a bit further into the main components of an API call.

Request and Response

As you very briefly read above, all interactions between a client—in this case, your Python console—and an API are split into a request and a response:

• Requests contain relevant data regarding your API request call, such as the base URL, the endpoint, the method used, the headers, and so on.
• Responses contain relevant data returned by the server, including the data or content, the status code, and the headers.

Using the Cat API again, you can drill down a bit more into what exactly is inside the Request and Response objects:

>>> response = requests.get("https://api.thecatapi.com/v1/breeds")
>>> response
<Response [200]>
>>> response.status_code
200
>>> response.text
'[{"weight":{"imperial":"7  -  10","metric":"3 - 5"},
"id":"abys","name":"Abyssinian","origin":"Egypt",
"temperament":"Active, Energetic, Independent, Intelligent, Gentle",...}]'
>>> response.headers
{'content-type': 'application/json; charset=utf-8',
'x-response-time': '1ms',  'Date': 'Mon, 24 Jul 2023 12:37:59 GMT',
'Server': 'Google Frontend', 'Content-Length': '76734', ...}

>>> response.request
<PreparedRequest [GET]>
>>> request = response.request
>>> request.url
'https://api.thecatapi.com/v1/breeds'
>>> request.path_url
'/v1/breeds'
>>> request.method
'GET'
>>> request.headers
{'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate',
'Accept': '*/*', 'Connection': 'keep-alive'}

Here, you’ve accessed a few of the most important attributes that are available for Request and Response objects.

You’ll learn more about some of these attributes in this tutorial, but if you want to dig even further, then you can check Mozilla’s documentation on HTTP messages for a more in-depth explanation of each attribute.

Status Codes

Status codes are one of the most important pieces of information to look for in any API response. They tell you if your request was successful, if it’s missing data, if it’s missing credentials, and so on.

With time, you’ll recognize the different status codes without help. But for now, here’s a list with some of the most common status codes that you’ll find:

You saw 200 OK earlier in the examples that you executed, and you might even recognize 404 Not Found from browsing the Web.

Fun fact: Companies tend to use 404 error pages for private jokes or pure fun, like these examples below:

In the API world, though, developers have limited space in the response for this kind of fun. But they make up for it in other places, like the HTTP headers. You’ll see some examples soon enough!

You can check the status of a response using .status_code and .reason. The requests library also prints the status code in the representation of the Response object:

>>> response = requests.get("https://api.thecatapi.com/v1/breeds")
>>> response
<Response [200]>
>>> response.status_code
200