Daft is a high-performance data engine designed for AI and multimodal workloads. It lets you process structured data alongside images, audio, video, and embeddings using a Python-native interface backed by a Rust execution engine.
Daft can run locally or scale to distributed clusters, while providing integrations with cloud storage, data lake formats, model providers, and machine learning frameworks.
This is free and open source software.
Key Features
- Process structured and multimodal data within a single framework.
- Work with images, audio, video, text, and embeddings.
- Run LLM prompts, generate embeddings, and perform AI inference at scale.
- Python-native interface with a performance-oriented Rust execution engine.
- Run locally or scale workloads across Ray and Kubernetes clusters.
- Vectorized execution engine with out-of-core processing.
- Query optimizer for improving data processing efficiency.
- Read data from Amazon S3, Google Cloud Storage, and other storage systems.
- Integrates with Apache Iceberg, Delta Lake, Hugging Face, and Unity Catalog.
- Provides integrations with OpenAI, Transformers, and custom models.
- Uses Apache Arrow for efficient columnar data representation.
Website: github.com/Eventual-Inc/Daft
Support:
Developer: Eventual, Inc.
License: Apache-2.0
Daft is written in Rust and Python. Learn Rust with our recommended free books and free tutorials. Learn Python with our recommended free books and free tutorials.
Related Software
| Data Analysis Tools | |
|---|---|
| Hadoop | Distributed processing of large data sets across clusters of computers |
| Storm | Distributed and fault-tolerant realtime computation |
| Drill | Distributed system for interactive analysis of large-scale datasets |
| Flink | Framework and distributed processing engine |
| Spark | Unified analytics engine for large-scale data processing |
| Pentaho | Enterprise reporting, analysis, dashboard, data mining, workflow and more |
| HPCC Systems | Designed for the enterprise to resolve Big Data challenges |
| Rapid Miner | Knowledge discovery in databases, machine learning, and data mining |
Read our verdict in the software roundup.
 Explore our comprehensive directory of recommended free and open source software. Our carefully curated collection spans every major software category.This directory is part of our ongoing series of informative articles for Linux enthusiasts. It features hundreds of detailed reviews, along with open source alternatives to proprietary solutions from major corporations such as Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk. You’ll also find interesting projects to try, hardware coverage, free programming books and tutorials, and much more. Discovered a useful open source Linux program that we haven’t covered yet? Let us know by completing this form. |
Please read our Comment Policy before commenting.