POST and GET Requests in Python

by Posted on

Whenever you visit a website, fill out a signup form, or request data from an API, you’re most likely making an HTTP POST or GET request. Developers often need to programmatically interact with web servers using both types of requests. This is quite common in data science and data engineering when procuring data from across the web or communicating with services in a data stack.. In this post, I’ll define what POST and GET requests are, and how to use them in Python.

What are POST and GET Requests?

When a client (say a browser or a Python script) needs to send data to or get data from a web server, it makes an HTTP request. There are two types of HTTP requests that we’ll talk about today. POST and GET. Quite simply –

GET is used to request data from a resource on a web server. This is done by sending a GET request to a URL. Examples include viewing a webpage in a browser, or asking a REST API for some data.

POST is used to send data to a web server so it can be processed. Examples include sending the data in a form filled out on a webpage, or sending data to a REST API to create an object or initiate some action.

Requests in Python

Thankfully, Python makes it easy to initiate GET and POST requests. Though there are a few different libraries you can use, the Requests library is my favorite. First, we’ll need to install it. You can use pip or another package manager to do so.

pip install requests

GET Requests

Now, let’s make a simple GET request to the Google homepage:

import requests

response = requests.get('https://www.google.com')
print(response.status_code)
print(response.text)

In this example, we make the GET request and are returned a response which contains a status_code, some headers and the HTML of the Google home page.

In some cases we back a JSON result set instead of HTML. REST APIs typically send back JSON for example. The Requests library makes that easy for us as well.

import requests

response = request.get('https://www.google.com/some-endpoint/')
resp_json = response.json()

You might also want to pass parameters in the URL of your GET request. For example, perhaps you want to pass a “name” parameter to the (fake) URL we made the GET request to above. Such a URL would look like this in a browser:

https://www.google.com/some-endpoint/?name=david

You can do this in Python by creating a dict with each key and value you’d like to send along with your URL in the request.

import requests
param_payload = {'name': 'david'}
response = request.get('https://www.google.com/some-endpoint/', params=param_payload)

POST Requests

Now, let’s take a look at POST requests. As discussed above, a POST request is used to send data to the web server. When such a request is made, you will get a response from the server which often contains some kind of status or other information about the result of your POST.

Unlike GET requests, you’ll always be passing data to go along with the URL. For POST requests we use the “data” parameter to do so. Here is an example of sending a message to a HTTP endpoint so it can be handed by that service.

import requests
import json

message_data = { "message": "Hello, world!}

response = requests.post("https://www.someservice.com/send-message/", data=json.dumps(message_data), headers = {'Content-Type': 'application/json'})

Upon a successful POST, we will receive some JSON back from the server telling us that all is well.

>> print(response.json())
{"status":"success"}

Wrapping Up

Making GET and POST requests in Python is easy thanks to the Requests library. In fact, I find Python to be one of the easiest languages for interacting with REST APIs and other HTTP resources in general.

If you haven’t already, you can sign up for the Data Liftoff mailing list to get more content and to stay up to date on the latest in data science and data engineering.

Published by James Densmore