Apache Chukwa is a data collection system developed for monitoring large distributed computing environments. It was designed to gather operational information from many machines and process the resulting data using the Hadoop ecosystem.
The system separates data collection from storage, processing and visualisation. Agents gather information on individual machines, collectors receive the data, and Hadoop-based processing components prepare it for subsequent analysis.
Chukwa was intended for installations requiring centralised visibility into large numbers of servers. Its architecture reflects the needs of distributed infrastructure, where logs and measurements must be collected continuously from numerous independent systems.
This is free and open source software.
Key Features
- Collect monitoring information and log data from distributed systems.
- Use agents running on monitored machines to gather data.
- Forward collected information to dedicated collector services.
- Store collected records using Hadoop Distributed File System facilities.
- Process accumulated information through Hadoop MapReduce jobs.
- Support configurable data collection and processing pipelines.
- Separate data acquisition, storage, analysis and presentation into distinct components.
- Use adaptors to obtain information from different supported sources.
- Process and organise accumulated event information for later examination.
- Support distributed collection across multiple monitored hosts.
- Provide a web-based interface through the Hadoop Infrastructure Care Center.
- Display selected monitoring results through dashboards and visualisations.
- Support integration with Hadoop storage and processing infrastructure.
- Provide an extensible architecture for additional data collection and analysis components.
Website: chukwa.apache.org
Support:
Developer: The Apache Software Foundation
License: Apache License 2.0
Chukwa is written in Java and JavaScript. Learn Java with our recommended free books and free tutorials. Learn JavaScript with our recommended free books and free tutorials.
Related Software
| Log Analyzers | |
|---|---|
| Kibana | Browser based interface for logstash and ElasticSearch |
| logstash | Log processing, search, and analytics |
| OpenObserve | Cloud-native observability platform |
| GoAccess | Real-time web log analyzer and interactive viewer |
| Fluentd | Data collector for unified logging layer |
| Loki | Horizontally-scalable, highly-available, multi-tenant log aggregation system |
| Graphite | Enterprise scalable realtime graphing |
| SigNoz | Monitor your applications and troubleshoot problems |
| Apache Flume | Delivers data from applications to Apache Hadoop's HDFS |
| OpenTSDB | Scalable, distributed Time Series Database |
| VictoriaLogs | High-performance log database designed to ingest, store, and query log data |
| syslog-ng | Collects, filters, and routes logs across systems |
| CLP | Compresses logs for efficient search and analysis |
| Matomo SLA | Imports web server logs into Matomo for analysis |
Read our verdict in the software roundup.
 LinuxLinks helps you discover the best free and open source software for Linux. Explore our extensive collection of recommended applications, including alternatives to popular proprietary software.We also publish software reviews, Linux tutorials, hardware reviews, free programming books and articles covering the wider open source community. Browse our Linux resources to discover more. Know a useful free and open source Linux application we haven't covered? Suggest a program for inclusion. Has your software been featured on LinuxLinks? Display one of our Featured on LinuxLinks badges on your website or GitHub README. |
Please read our Comment Policy before commenting.