In a containerized environment like Kubernetes, Pods and the containers within them, can be created and deleted automatically via ReplicaSet’s. And now also I will be using that environment.

In a previous series of articles, I talked about an environment, I prepared on my Windows laptop, with a guest Operating System, Docker and Minikube available within an Oracle VirtualBox appliance, with the help of Vagrant. I leave it up to you to decide which product is most suitable for (log) data collection in your situation. This time I won’t be using Fluentd for log aggregation. Installed as an agent on your servers, Filebeat monitors the log files or locations that you specify, collects log events, and forwards them to either to Elasticsearch or Logstash for indexing. įilebeat is a lightweight shipper for forwarding and centralizing log data.

In 2015, a family of lightweight, single-purpose data shippers were introduced into the ELK Stack equation. Logstash is a server -side data processing pipeline that ingests data from multiple sources simultaneously, transforms it, and then sends it to a “stash” like Elasticsearch. The Elastic Stack is the next evolution of the ELK Stack. In my previous article I already spoke about Elasticsearch (a search and analytics engine) and Kibana (which lets users visualize data with charts and graphs in Elasticsearch). “ELK” is the acronym for three open source projects: Elasticsearch, Logstash, and Kibana. One popular centralized logging solution is the Elasticsearch, Fluentd, and Kibana (EFK) stack.įluentd is an open source data collector, which lets you unify the data collection and consumption for a better use and understanding of data. In this article I will talk about the installation and use of Filebeat (without Logstash). In a new series of articles, I will dive into using Filebeat and Logstash (from the Elastic Stack) to do the same. Besides log aggregation (getting log information available at a centralized location), I also described how I created some visualizations within a dashboard. In my last article I described how I used ElasticSearch, Fluentd and Kibana (EFK).

Install filebeat on kubernetes