Winlogbeat not creating daily indexes

Elastic Stack Beats
1

I have taken over an elasticsearch 6.6.2 environment which was a proof of concept to centralize event_logs monitoring.

I have upgraded the servers to Windows 2019 and elasticsearch to 7.1.0 (and now 7.1.1) and added a dedicated ingest node (and included other event_logs), but now the indexes are not ‘daily’ anymore but just one big index. Even when the date rolled over last night, it still kept using the single index. This was working correctly in 6.6.2, so I am wondering if I am missing something.

Previously we had indexes like:
winlogbeat-6.6.2-2019.06.02
winlogbeat-6.6.2-2019.06.01
winlogbeat-6.6.2-2019.05.31
even all the way back to 2018 while the PoC was started in February 2019, so it seems that the older events caused indexes with past dates to be created.

Now however, I have just a big index containing all event-dates with the date of when the winlogbeat (7.1.0) were started:
winlogbeat-7.1.0-2019.06.04-000001
Is this something new in version 7?

I tried adding the settings (to a new winlogbeat with latest version) :
setup.template.name: "winlogbeat-7.1.1"
setup.template.pattern: "winlogbeat-7.1.1-*"
output.elasticsearch:
hosts: ["elastic-in:9200"]
index: "winlogbeat-%{[agent.version]}-%{+yyyy.MM.dd}"

Ran the winlogbeat setup --template and verified that the template was there.
But that still creates a winlogbeat-7.1.1-2019.06.05-000001 (so with the number at the end)

Has this behavior changed in version 7?
Is it because I now have a dedicated ingest node?
Do I need additional settings? (copied the winbeatlog.yml from the 6.6.2 PoC and added event_log names)

2

Hi @srdegeus
Yes this changed in v7. No it is not because you have a dedicated ingest node. You can get the same behavior as in your old version with the settings setup.ilm.enabled=false inside winlogbeat.yml file. BUT!

Have a look at the Feature that is default enabled in Elastic Stack >7 https://www.elastic.co/guide/en/beats/winlogbeat/current/ilm.html

This causes your index to rollover into a new index when the boundries in the policy arrives.
It is suggested to use ilm, so the index will only rollover when the set parameters of index size, created still time or even number of docs in index arrives. So you will allways have similar indexes, than if you daily create a new index. (example: monday to friday big indices and on saturday and sunday empty indices..)

1 Like
3

Thank you so much! This explains a lot.

I will read some more into index lifecycle management. As you describe it seems a more intelligent way of handling data.

Thanks for answering this.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.