Integrating with DuckDB

DuckDB is an in-process analytical database. The speed is like that of Polars, which is really fast. Like Polars, DuckDB has built-in integrations with other tools. One example is its integration with Arrow. Just like Polars, DuckDB allows for zero-copy to and from the Arrow format. DuckDB has APIs in many languages such as Python, C, Go, R, and Swift. You can check out DuckDB’s documentation to learn more about it at https://duckdb.org/.

DuckDB has a solid SQL API with a rich set of features. It’s superior to the SQL API in Polars as of the time of writing this. You can also run a SQL query on a Polars DataFrame directly without copying data. Even if you convert a DuckDB relation to a Polars DataFrame or vice versa, it allows you to enable zero-copy operations, thanks to Apache Arrow columnar memory format used both in Polars and DuckDB.

Getting ready

You need to have PyArrow installed as Arrow is the fundamental technology through which...