Polars is a DataFrame interface on top of an OLAP Query Engine implemented in Rust using Apache Arrow Columnar Format as the memory model.
The core is written in Rust, but the library is also available in Python.
This is free and open source software.
Key Features
- Lazy | eager execution.
- Multi-threaded.
- SIMD.
- Query optimization.
- Powerful expression API.
- Hybrid Streaming (larger than RAM datasets).
- Fast: Polars is written from the ground up, designed close to the machine and without external dependencies.
- I/O: First class support for all common data storage layers: local, cloud storage & databases.
Easy to use: Write your queries the way they were intended. Polars, internally, will determine the most efficient way to execute using its query optimizer. - Out of Core: Polars supports out of core data transformation with its streaming API. Allowing you to process your results without requiring all your data to be in memory at the same time
- Parallel: Polars fully utilises the power of your machine by dividing the workload among the available CPU cores without any additional configuration.
- Vectorized Query Engine: Polars uses Apache Arrow, a columnar data format, to process your queries in a vectorized manner. It uses SIMD to optimize CPU usage.
- Rust | Python | NodeJS | R | …
Website: github.com/pola-rs/polars
Support: Documentation
Developer: Ritchie Vink
License: MIT License
Polars is written in Rust and Python. Learn Rust with our recommended free books and free tutorials. Learn Python with our recommended free books and free tutorials.
Related Software
| Python Data Analysis | |
|---|---|
| pandas | High-level building block for doing practical, real world data analysis |
| NumPy | Core package for scientific computing with Python |
| SciPy | Ecosystem for mathematics, science, and engineering |
| Polars | DataFrame interface on top of an OLAP Query Engine |
| statsmodels | Statistical modeling and econometrics in Python |
| Dask | Advanced parallelism for analytics |
| Orange | Component-based framework for machine learning and data mining |
| Modin | Drop-in replacement for pandas |
| Vaex | Fast visualization of big data |
| AWS DW | Extends the power of pandas library |
| yt | Multi-code Toolkit for Analyzing and Visualizing Volumetric Data |
| HoloViews | Make Data Analysis and Visualization Seamless |
| datatable | Manipulate 2-dimensional tabular data structures |
| xarray | Work with labelled multi-dimensional arrays and datasets |
| pyjanitor | Extend pandas with readable data-cleaning functions |
| Optimus | Agile Data Preparation Workflows |
Read our verdict in the software roundup.
 Explore our comprehensive directory of recommended free and open source software. Our carefully curated collection spans every major software category.This directory is part of our ongoing series of informative articles for Linux enthusiasts. It features hundreds of detailed reviews, along with open source alternatives to proprietary solutions from major corporations such as Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk. You’ll also find interesting projects to try, hardware coverage, free programming books and tutorials, and much more. Discovered a useful open source Linux program that we haven’t covered yet? Let us know by completing this form. |
Please read our Comment Policy before commenting.