I'm pretty sure I'm missing something obvious, but ths has taken far too much time already, and I haven't found anyone with the same problem, so I hope somebody is willing to help this beginner.
I am setting up a monitoring system for our servers, and right now I have a filebeat collecting info and sending it to a central logstash, who passes it on elasticsearch. I'm also planning to use at least metricbeat, and possibly a few other beats. For now the logstash doesn't to anything except passing on the data. Later I intend to add filters to label important events, and possibly other processing, but that's not relevant right now.
What I'm strugging with is how to name the indices. It seems obvious to me that the filebeat data should be stored in elasticsearch with an index matching the 'filebeat-' pattern, since that is how 'filebeat setup' sets up the elasticsearch patterns, visualisations, and dashboards. However, by default logstash puts the data in an index matching the 'logstash-' pattern, which means all the filebeat setup doesn't work.
I am using the most recent version of all the software packages.
I am surprised that this 'ELK' setup doesn't work using defaults for this, but clearly I have to choose one of the two approaches:
-
Configure logstash so that it puts filebeat data with a 'filebeat-' index pattern in elasticsearch (and puts metricbeat data with a 'metricbeat-' index pattern in elasticsearch, and so on).
-
Somehow arrange that 'filebeat setup' sets up the elasticsearch patterns, visualizations, and dashboards using indices that match the 'logstash=*' patterns.
I have not been able to do either of these two, and I don't even know which one of the two approaches I should go for. Can anybody please help me to get this working?
Thanks in advance.
