Daft is a high-performance data engine designed for AI and multimodal workloads. It lets you process structured data alongside images, audio, video, and embeddings using a Python-native interface backed by a Rust execution engine.
Daft can run locally or scale to distributed clusters, while providing integrations with cloud storage, data lake formats, model providers, and machine learning frameworks.
This is free and open source software.
Key Features
- Process structured and multimodal data within a single framework.
- Work with images, audio, video, text, and embeddings.
- Run LLM prompts, generate embeddings, and perform AI inference at scale.
- Python-native interface with a performance-oriented Rust execution engine.
- Run locally or scale workloads across Ray and Kubernetes clusters.
- Vectorized execution engine with out-of-core processing.
- Query optimizer for improving data processing efficiency.
- Read data from Amazon S3, Google Cloud Storage, and other storage systems.
- Integrates with Apache Iceberg, Delta Lake, Hugging Face, and Unity Catalog.
- Provides integrations with OpenAI, Transformers, and custom models.
- Uses Apache Arrow for efficient columnar data representation.
Website: github.com/Eventual-Inc/Daft
Support:
Developer: Eventual, Inc.
License: Apache-2.0
Daft is written in Rust and Python. Learn Rust with our recommended free books and free tutorials. Learn Python with our recommended free books and free tutorials.
Related Software
| Data Analysis Tools | |
|---|---|
| Hadoop | Distributed processing of large data sets across clusters of computers |
| Storm | Distributed and fault-tolerant realtime computation |
| Drill | Distributed system for interactive analysis of large-scale datasets |
| Flink | Framework and distributed processing engine |
| Spark | Unified analytics engine for large-scale data processing |
| Pentaho | Enterprise reporting, analysis, dashboard, data mining, workflow and more |
| HPCC Systems | Designed for the enterprise to resolve Big Data challenges |
| Daft | High-performance distributed data processing engine |
| Apache Beam | Unified programming model for batch and streaming data processing |
Read our verdict in the software roundup.
 Explore our carefully curated directory of recommended free and open source software, covering every major software category.The directory forms part of our extensive collection of articles for Linux enthusiasts. It includes hundreds of detailed reviews, together with free and open source alternatives to proprietary software from companies such as Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk. LinuxLinks also covers interesting projects worth exploring, Linux-compatible hardware, free programming books and tutorials, and much more. Know a useful free and open source Linux application that we haven’t covered? Tell us about it using our submission form. |
Please read our Comment Policy before commenting.