Working with Amazon S3

Amazon S3 is one of the most popular choices for an object storage service. Many organizations use it for a data lake, a data repository that stores both structured and unstructured data.

In this recipe, we’ll look at how to read from and write to an S3 bucket in Polars.

Getting ready

You need your AWS account for this recipe. Follow the instructions in the link I mentioned under Technical requirements to create your AWS account.

Let’s create an S3 bucket. I introduce how to do that in the following steps, but if you prefer reading the documentation on your own, here’s a useful link: https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html.

Here are the steps for creating an S3 bucket:

1. Navigate to https://aws.amazon.com/ and sign in to your account.
2. You can either type S3 in the search bar, click on Services on the top left, or just choose S3 in Recently visited if you see one.

...