Hi All Elasticsearch Gurus,
I am a newbie and learning the Elastic Stack and hence need inputs and thoughts from experts..
Assuming an Elastic Stack environment wherein the Logstash component parses/transforms data from various sources (web apps, mobile apps, switches, routers, OS logs etc) and ships to Elasticsearch cluster to be further viewed in Kibana for monitoring and troubleshooting....
I plan to create index ( in the output section of logstash conf file) in the following manner for all input sources
and similarly for other sources. (i.e. an index per source per day)
I have some questions for the above use
Q1. Is this a good practice? What are the disadvantages or potential future problems if data grows? ( since no. of indices will increase and replica shards will increase too)
Q2. What is recommendation for such use case?
Q3. Is creating an index/source better approach? ( e.g. webapp-index instead of daily webapp-index-yyyy.mm.dd)
Any pointers or suggestions is greatly appreciated.
Thank you in advance,