Scribe is a distributed log aggregation system originally developed at Facebook for collecting event data from large numbers of servers. It was designed for environments where applications generate continuous streams of log messages that need to be consolidated for storage and analysis.
The system uses a hierarchy of collection servers. Individual machines can send events to local Scribe servers, which forward the information to central servers or other aggregation layers.
This architecture allows the collection infrastructure to handle temporary network failures and unavailable destinations. Scribe can buffer messages locally and forward them when connectivity is restored, reducing the risk of losing information during interruptions.
This is free and open source software.
Key Features
- Collect log events streamed from multiple clients.
- Aggregate events from large numbers of distributed machines.
- Forward messages through configurable hierarchies of collection servers.
- Use local buffering when destination servers are temporarily unavailable.
- Forward buffered messages after a failed destination becomes available again.
- Write collected events to local or network-mounted filesystems.
- Configure network stores for forwarding events to other Scribe servers.
- Use buffer stores with primary and secondary destinations.
- Distribute messages among configured stores using bucket-based routing.
- Forward individual events to multiple destinations through multi stores.
- Discard selected events through null stores.
- Write event records using Thrift-based file storage.
- Support applications communicating through Apache Thrift interfaces.
- Configure separate storage behaviour for different categories of log messages.
Website: github.com/facebookarchive/scribe
Support:
Developer: Facebook
License: Apache License 2.0
Scribe is written in C++. Learn C++ with our recommended free books and free tutorials.
Related Software
| Log Analyzers | |
|---|---|
| Kibana | Browser based interface for logstash and ElasticSearch |
| logstash | Log processing, search, and analytics |
| OpenObserve | Cloud-native observability platform |
| GoAccess | Real-time web log analyzer and interactive viewer |
| Fluentd | Data collector for unified logging layer |
| Loki | Horizontally-scalable, highly-available, multi-tenant log aggregation system |
| Graphite | Enterprise scalable realtime graphing |
| SigNoz | Monitor your applications and troubleshoot problems |
| Apache Flume | Delivers data from applications to Apache Hadoop's HDFS |
| OpenTSDB | Scalable, distributed Time Series Database |
| VictoriaLogs | High-performance log database designed to ingest, store, and query log data |
| syslog-ng | Collects, filters, and routes logs across systems |
| CLP | Compresses logs for efficient search and analysis |
| Matomo SLA | Imports web server logs into Matomo for analysis |
Read our verdict in the software roundup.
LinuxLinks helps you discover the best free and open source software for Linux. Explore our extensive collection of recommended applications, including alternatives to popular proprietary software.We also publish software reviews, Linux tutorials, hardware reviews, free programming books and articles covering the wider open source community. Browse our Linux resources to discover more. Know a useful free and open source Linux application we haven't covered? Suggest a program for inclusion. Has your software been featured on LinuxLinks? Display one of our Featured on LinuxLinks badges on your website or GitHub README. |


Please read our Comment Policy before commenting.
Scribe was developed at Facebook and released as open source but is currently unmaintained.