Ibis is a portable DataFrame framework that provides a consistent interface for manipulating data across different database systems and analytical engines. It allows users to construct analytical expressions without tying their application code to a particular execution backend.
Traditional data analysis workflows often require rewriting queries or adapting code when moving between local processing and database systems. Ibis addresses this by separating the description of a computation from the engine responsible for executing it.
This makes the framework useful for analysts and developers working with datasets that may begin on a local machine before moving to larger database infrastructure.
This is free and open source software.
Key Features
- Unified DataFrame API for working with more than 20 supported analytical backends.
- Lazy evaluation allows expressions to be constructed before execution.
- Interactive mode supports iterative exploration of datasets.
- DuckDB provides the default backend for local DataFrame operations.
- Supports analytical engines including DataFusion, Polars, Spark, and Trino.
- Database integrations include PostgreSQL, SQLite, MySQL, ClickHouse, BigQuery, and Snowflake.
- Compiles expressions into SQL for compatible database engines.
- Allows SQL queries and DataFrame operations to be combined within the same workflow.
- Supports filtering, grouping, aggregation, sorting, and other common data transformations.
- Reads common data formats including CSV and Apache Parquet through supported backends.
- Enables applications to move between local and remote execution by changing the backend connection.
- Provides reusable expressions that can be composed into more complex analytical operations.
- Integrates with established analytical tools and data processing frameworks.
Website: github.com/ibis-project/ibis
Support:
Developer: Ibis contributors
License: Apache License 2.0
Ibis is written in Python. Learn Python with our recommended free books and free tutorials.
Related Software
| Python Data Analysis | |
|---|---|
| pandas | High-level building block for doing practical, real world data analysis |
| NumPy | Core package for scientific computing with Python |
| SciPy | Ecosystem for mathematics, science, and engineering |
| Polars | DataFrame interface on top of an OLAP Query Engine |
| statsmodels | Statistical modeling and econometrics in Python |
| Dask | Advanced parallelism for analytics |
| Orange | Component-based framework for machine learning and data mining |
| Modin | Drop-in replacement for pandas |
| Vaex | Fast visualization of big data |
| AWS DW | Extends the power of pandas library |
| yt | Multi-code Toolkit for Analyzing and Visualizing Volumetric Data |
| HoloViews | Make Data Analysis and Visualization Seamless |
| datatable | Manipulate 2-dimensional tabular data structures |
| xarray | Work with labelled multi-dimensional arrays and datasets |
| pyjanitor | Extend pandas with readable data-cleaning functions |
| Optimus | Agile Data Preparation Workflows |
Read our verdict in the software roundup.
LinuxLinks helps you discover the best free and open source software for Linux. Explore our extensive collection of recommended applications, including alternatives to popular proprietary software.We also publish software reviews, Linux tutorials, hardware reviews, free programming books and articles covering the wider open source community. Browse our Linux resources to discover more. Know a useful free and open source Linux application we haven't covered? Suggest a program for inclusion. Has your software been featured on LinuxLinks? Display one of our Featured on LinuxLinks badges on your website or GitHub README. |


Please read our Comment Policy before commenting.