Index management Logstash vs Elasticsearch

Hello,

I have ELK stack deployed all on one host for testing purposes and two Filebeat klients sending data from other hosts to it.

First one is sending Apache and Icinga monitoring logs from a linux machine directly do Elasticsearch. The second one is sending logs from custom application on windows machine to Logstash where I set up grok filtres for parsing and then redirection to Elasticsearch.

The thing i dont understand is that the first Filebeat (sending directly to Elasticsearch) has created one index (filebeat-7.8.0-2020.06.22-000001) that it's using from beginning for a few days now while the second Filebeat (sending tru Logstash) is creating a new index every day (filebeat-7.8.0-2020.06.XX).

I would expect the same behaviour from both Filebeats and that is creating new index every day. Why is the first filebeat still sending data to the same index and why is there 000001 in the name of it while there never was a identically named index?

I'm quite new to ELK stack, so please excuse me if this is something that is documented in detail. I've searched in the official documentation and on google but couldn't help myself out.

The difference would be that Filebeat is using ILM, while Logstash doesn't do that by default.

You can tell Logstash to use it as well.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.