Hi, I have multi services which generating at least 2 million of log message per day, I’m using logstash to form a doc and I create new index for every new day log, but I’m feeling it should have better way to handle it? Should I put all the log into one index regardless how many day? And make it 3…

Almost, you just need to adjust the time parameter, see https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-index

Discuss the Elastic Stack

How should I index the 100 billion log

Elastic Stack Logstash

warkolm (Mark Walkom) April 27, 2018, 2:45am 2

No, use time based indices.

Topic		Replies	Views
Disable index rotation in logstash Logstash	6	2357	July 6, 2017
Is it possible to create two index for the same day using logstash templete Logstash	8	819	September 25, 2019
Best practice for indexing a daily log Logstash	2	1823	March 4, 2018
Best Practices - Confused Logstash	4	667	May 22, 2017
Logstash creating day based indexes Logstash	5	1159	July 6, 2017

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries
Trademarks
Terms
Privacy
Brand
Code of Conduct

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.

How should I index the 100 billion log

Related topics