Hi there,
I've spent several hours searching the wider internet, including a lot of the "elastic" docs and discussion topics but I still haven't been able to find a clear answer to my question.
I intend to use an Index Template (or templates) in Elasticsearch. Each index will have an alias and I will have a cron job setup to call the Rollover API for each index/alias based on appropriate criteria for each. i.e. some will be based on time, others based on the number of documents etc...
When I use the Output for Elasticsearch in Logstash, I have an option to use "index =>" and specify the name of an Elasticsearch index, including names that would automatically use a Beat metric in the name, or a date and/or time etc. There's loads of useful options for naming the index, thanks! What I want to understand though is, if I set my "index =>" parameter within my Logstash Output to be the name of the alias sitting "over" an index, will it correctly go into Elasticsearch using that alias and the Index sitting behind it? OR, will it create a new index with the name specified in the Output "index =>" field? OR, worse, yet, will it just error out because I'm trying to create (dynamically) and write to an Index for which there is an Alias with the same name?
For my context, I'm hoping to have the Indices have names such as - and then the Alias will just be .
e.g. my Index for today will be "filebeat-dns-2018-09-21" and when the Rollover API is called with a "trigger" of "1d" (unless there is a better way to roll this over each day?) it will become "filebeat-dns-2018-09-22" just after midnight. The Alias that sits over this would be filebeat-dns.
After rollover, the previous day's index would no longer be associated with the "filebeat-dns" index and instead be associated with the "filebeat-dns-search" index, moved off to a "search" node.
So - can I point the Output field for "index =>" in Logstash to the "filebeat-dns" Alias and all my logs for the daily index will continue to end up in the correct Index behind the Alias?
My overall goal here is to be as "hands-off" as possible, tagging things as close to the source as possible and having them end up in the correct Index that's rolled-over on the ES nodes etc.
Thanks in advance!!