Developer(s) Treasure Data Repository github.com/fluent | Stable release v0.12.15 | |
Initial release October 2011;5 years ago (2011-10) Operating system | ||
Fluentd is a cross platform open source data collection solution originally developed at Treasure Data. It is written primarily in Ruby.
Contents
Overview
Fluentd is a Big Data tool and deals with semi- or unstructured data sets in real time. Like Kafka, it analyzes event logs, application logs, and clickstreams. According to Suonsyrjä and Mikkonen, the "core idea of Fluentd is to be the unifying layer between different types of log inputs and outputs." According to its official website, Fluentd is available on Linux and Mac OSX and experimentally on Windows.
History
Fluentd was created by Sadayuki Furuhashi as a project of the Mountain View-based firm Treasure Data. Written primarily in Ruby, its source code was released as open source in October 2011.
Users
Fluentd is one of the data collection tools recommended by Amazon Web Services who noted in their 2013 white paper that while its architecture is very similar to Apache Flume or Scribe, Fluentd had better documentation and support and was easier to both install and maintain. Google Cloud Platform's BigQuery recommends Fluentd as default real-time data-ingestion tool, and uses Google's customized version of Fluentd, called google-fluentd, as a default logging agent.