Most components of a web application produce operational log files. Some logs are written by each application in a unique format. Other components generate out-of-the-box logs. Monitoring system logs is an essential activity for anyone charged with taking decisions. System administrators need to monitor logs to look out for unusual activity, to troubleshoot applications and websites that are under their control. By scanning logs, extracting and correlating data, system administrators can investigate and resolve problems, carry out capacity planning, help to detect vulnerabilities, ensure the smooth running of services and balancing capacity, and establish who has used services and when.
Information captured in log files is an important strategic resource to carry out analytics and searches. Making sense of logs helps organisations make better customer-focused decisions.
The purpose of this article is to identify the best open source software for collecting, parsing, storing, and making sense of logs. The applications featured in this article are ideal for organisations of any size.
To provide an insight into the quality of software that is available, we have compiled a list of 12 high quality open source log analyzers that offer an excellent way to manage and interrogate logs. Software featured in this article provided finely grained, real-time monitoring, generating scalable realtime graphing.
Here’s our verdict captured in a legendary LinuxLinks-style ratings chart. Only free and open source software is eligible for inclusion.
Click the links in the table below to learn more about each log analyzer.
| Log Analyzers | |
|---|---|
| Kibana | Browser based interface for logstash and ElasticSearch |
| logstash | Log processing, search, and analytics |
| Fluentd | Data collector for unified logging layer |
| Loki | Horizontally-scalable, highly-available, multi-tenant log aggregation system |
| Graylog2 | Log management solution implementation storing logs in ElasticSearch |
| Graphite | Enterprise scalable realtime graphing |
| SigNoz | Monitor your applications and troubleshoot problems |
| Apache Flume | Delivers data from applications to Apache Hadoop's HDFS |
| OpenTSDB | Scalable, distributed Time Series Database |
| LogoRRR | Cross-platform log analysis tool |
| Scribe | Server for aggregating log data that is streamed in real time from clients |
| Chukwa | Hadoop sub-project devoted to large-scale log collection and analysis |
This article has been revamped in line with our recent announcement.
 Explore our comprehensive directory of recommended free and open source software. Our carefully curated collection spans every major software category.This directory is part of our ongoing series of informative articles for Linux enthusiasts. It features hundreds of detailed reviews, along with open source alternatives to proprietary solutions from major corporations such as Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk. You’ll also find interesting projects to try, hardware coverage, free programming books and tutorials, and much more. Know a useful open source Linux program that we haven’t covered yet? Let us know by completing this form. |