Adding filebeat index patterns to use filebeat index or just use custom index without adding filebeat index patterns

[BentCoder]

Hi,

1. If I use filebeat-%{+yyyy.MM.dd} as index name, every day I get new index created in ES. I then can use filebeat-* in Kibana to see logs. For this, I have to add index patterns and load template.

2. If I use custom my-custom-index as index name, ES will have only one dedicated index. I then can use my-custom-index in Kibana to see logs. For this, I don't have to add or load index patterns/template.

Example:

output {
    elasticsearch {
        hosts => ["localhost:9200"]
        sniffing => true
        manage_template => false
        #index => "my-custom-index"
        #index => "filebeat-%{+yyyy.MM.dd}"
    }
}

The questions is: In terms of checking/visualising logs in Kibana, would there be any difference if I use my-custom-index or filebeat-*? I tried both and haven't seen any difference so what is the point of adding index patterns for options 1 (filebeat index version)?

Note: I asked similar question here but got all superficial/shortcut answers.

Thanks

[magnusbaeck (Magnus Bäck)]

In terms of checking/visualising logs in Kibana, would there be any difference if I use my-custom-index or filebeat-*?

No.

I tried both and haven’t seen any difference so what is the point of adding index patterns for options 1 (filebeat index version)?

The point of having one index per day (or month or hour or whatever) is that it helps keep your indexes to a reasonable size and it makes for efficient deletion of old data. Depending on how queries are made they can also be made more efficiently if you can narrow down to a subset of the data based on the time.