How to generate month instead of weekly indexes in winlogbeat?

Elastic Stack Beats
1

Hi all,

If I use winlogbeat to send Windows log directly to ES, what is the config syntax in winlogbeat.yml to generate monthly indexes? I have tried the syntax below but it did not work

index: "[winlogbeat-]YYYY.MM"

Thanks,

2

This is currently not possible with beats. There is an open enhancement request: https://github.com/elastic/beats/issues/921

You need to send the data through Logstash if you want to use monthly indexes.

3

Thanks, being able to generate monthly or weekly indexes would be great because I want to limit the number of indexes and shards in a cluster.

4

You can do some of that if you modify the template for beats.

5

I'm having 2 shards for 2 data nodes and 1 replica now. Should I even go for 1 shard and 1 replica? I have winlogbeat write to both ES data nodes.

6

Sure, that'd work fine.

7

Will it somewhat affect indexing because data will be written to the ES instance that hosts the primary shard?

8

No because it needs to write to the replica anyway.

9

Yes, but index request must reach a primary shard first and then a replica to be considered completed. So in a setup of 2 data nodes with 1 primary shard and 1 replica, if I have logstash write to both data nodes in the configuration, Logstash will actually write to only the one node the hold the primary shard. Compared to 2 primary shards and 1 replica, in which each node hosts a primary shards, then Logstash can send data to both node simultaneously.

I'm just not sure if it really makes any difference.

10

And then ES sends the exact same thing to the replica. The two shards do the same amount of work, and that would be the same if you have 1 or two primaries with replicas

11

Thanks,

12

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.