New Elasticsearch installation and initial index pattern - what is it related to?


I am new in Elasticsearch world so please forgive my ignorance. I have just finished new ELK installation and after logging in to Kibana it is asking me for "index pattern". If I have 10 hosts shipping their logs to ELK and I will use filebeat on 5 of them and some other solution (i.e. syslog) on the other 5 what should I put in to the "index pattern" ? Is the "index pattern" somehow related to what is sending the logs or where it is sent from ? If the former should I use "filebeat-*" as the index pattern ? If the latter -> "hostname-*" as an index pattern ?

Thanks in advance

FYI we’ve renamed ELK to the Elastic Stack, otherwise Beats and APM feel left out! :wink:

It sounds like it, yes.

You can head to Console, in Dev Tools, and then run GET _cat/indices and then paste the response in here for us to check if you want to be 100% sure.

But is the index pattern related to what is sending or where it is sent from ?
Also what about the other 5 hosts (taken from the example above) which don't use filebeat ? Is the index pattern "filebeat-*" filter them out ?


I don't know how you have that setup, so I can't comment.

Yes to what ? I asked two question above.

I have 10 hosts sending their logs to Elasticsearch. Five of them are using filebeat for that purpose, the rest 5 don't. Let's say I use index pattern "filebeat-*". Is it kind of filter to accept logs ONLY from the first 5 hosts and refuse logs from those without filebeat ?

No, it's simply where filebeat defines as the source for the files.

For the others, you need to define your own index.

Can I have the same index for both kind of servers: those with filebeat installed and those without it ?

Yes, but it's not recommend. We suggest having an index for each different data type. So put apache logs together, but system logs in another index and so on.

Thanks @warkolm. So keep related data together.
The data (/var/log/*) from above example comes from servers but in different way: with filebeat or without so it makes sense to keep them together anyway.
But thanks for the hint.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.