Elastic search Indexes

Elastic Stack Elasticsearch
1

Hi Team,

I am new to elastic search and ELK configuration. By following the documentation i was able to configure the ELK. I am pushing my application logs to Elastic search and the view the data in Kibana.

Now i wanted to understand the best practices to follow for this to be deployed for real time applications. I wanted to put my log information from Cluster of nodes to the Elastic search and over a period of time my application log information is going to increase drastically.

I am looking at what are the best practices to create an index so that i should be able to delete 1 month old data from elastic search index. I dont want to delete the whole index only part of the data in that needs to be deleted.

Regards
Manju

2

I would say daily indices. I have Logstash creating a new Elasticsearch index everyday and then I can set a "retention period" with Curator to delete indices older than a certain amount of days.

3

Thanks for your reply. Do you have any configuration or documentation which explains me to create daily indices automatically.

4

It's the default for Elasticsearch output in Logstash

https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-index

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.