How to generate month instead of weekly indexes in winlogbeat?


(Anh) #1

Hi all,

If I use winlogbeat to send Windows log directly to ES, what is the config syntax in winlogbeat.yml to generate monthly indexes? I have tried the syntax below but it did not work

index: "[winlogbeat-]YYYY.MM"

Thanks,


(Andrew Kroh) #2

This is currently not possible with beats. There is an open enhancement request: https://github.com/elastic/beats/issues/921

You need to send the data through Logstash if you want to use monthly indexes.


(Anh) #3

Thanks, being able to generate monthly or weekly indexes would be great because I want to limit the number of indexes and shards in a cluster.


(Mark Walkom) #4

You can do some of that if you modify the template for beats.


(Anh) #5

I'm having 2 shards for 2 data nodes and 1 replica now. Should I even go for 1 shard and 1 replica? I have winlogbeat write to both ES data nodes.


(Mark Walkom) #6

Sure, that'd work fine.


(Anh) #7

Will it somewhat affect indexing because data will be written to the ES instance that hosts the primary shard?


(Mark Walkom) #8

No because it needs to write to the replica anyway.


(Anh) #9

Yes, but index request must reach a primary shard first and then a replica to be considered completed. So in a setup of 2 data nodes with 1 primary shard and 1 replica, if I have logstash write to both data nodes in the configuration, Logstash will actually write to only the one node the hold the primary shard. Compared to 2 primary shards and 1 replica, in which each node hosts a primary shards, then Logstash can send data to both node simultaneously.

I'm just not sure if it really makes any difference.


(Mark Walkom) #10

And then ES sends the exact same thing to the replica. The two shards do the same amount of work, and that would be the same if you have 1 or two primaries with replicas


(Anh) #11

Thanks,


(system) #12

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.