Data Analysis

Ibis – portable DataFrame library for data analysis

Ibis is a portable DataFrame framework that provides a consistent interface for manipulating data across different database systems and analytical engines. It allows users to construct analytical expressions without tying their application code to a particular execution backend.

Traditional data analysis workflows often require rewriting queries or adapting code when moving between local processing and database systems. Ibis addresses this by separating the description of a computation from the engine responsible for executing it.

This makes the framework useful for analysts and developers working with datasets that may begin on a local machine before moving to larger database infrastructure.

This is free and open source software.

Key Features

  • Unified DataFrame API for working with more than 20 supported analytical backends.
  • Lazy evaluation allows expressions to be constructed before execution.
  • Interactive mode supports iterative exploration of datasets.
  • DuckDB provides the default backend for local DataFrame operations.
  • Supports analytical engines including DataFusion, Polars, Spark, and Trino.
  • Database integrations include PostgreSQL, SQLite, MySQL, ClickHouse, BigQuery, and Snowflake.
  • Compiles expressions into SQL for compatible database engines.
  • Allows SQL queries and DataFrame operations to be combined within the same workflow.
  • Supports filtering, grouping, aggregation, sorting, and other common data transformations.
  • Reads common data formats including CSV and Apache Parquet through supported backends.
  • Enables applications to move between local and remote execution by changing the backend connection.
  • Provides reusable expressions that can be composed into more complex analytical operations.
  • Integrates with established analytical tools and data processing frameworks.

Website: github.com/ibis-project/ibis
Support:
Developer: Ibis contributors
License: Apache License 2.0

Ibis is written in Python. Learn Python with our recommended free books and free tutorials.


Related Software

Python Data Analysis
pandasHigh-level building block for doing practical, real world data analysis
NumPyCore package for scientific computing with Python
SciPyEcosystem for mathematics, science, and engineering
PolarsDataFrame interface on top of an OLAP Query Engine
statsmodelsStatistical modeling and econometrics in Python
DaskAdvanced parallelism for analytics
OrangeComponent-based framework for machine learning and data mining
ModinDrop-in replacement for pandas
VaexFast visualization of big data
AWS DWExtends the power of pandas library
ytMulti-code Toolkit for Analyzing and Visualizing Volumetric Data
HoloViewsMake Data Analysis and Visualization Seamless
datatableManipulate 2-dimensional tabular data structures
xarrayWork with labelled multi-dimensional arrays and datasets
pyjanitorExtend pandas with readable data-cleaning functions
OptimusAgile Data Preparation Workflows

Read our verdict in the software roundup.


Best Free and Open Source Software LinuxLinks helps you discover the best free and open source software for Linux. Explore our extensive collection of recommended applications, including alternatives to popular proprietary software.

We also publish software reviews, Linux tutorials, hardware reviews, free programming books and articles covering the wider open source community. Browse our Linux resources to discover more.

Know a useful free and open source Linux application we haven't covered? Suggest a program for inclusion.

Has your software been featured on LinuxLinks? Display one of our Featured on LinuxLinks badges on your website or GitHub README.
Subscribe

Please read our Comment Policy before commenting.

Notify of
guest
0 Comments
Oldest
Newest Most Voted