Winlogbeat many shards

Elastic Stack Elasticsearch
1

Hi All,

i am new with ELK, i have a test cluster with single data node, i have configured one server to use winlogbeat now i am receiving a lots of small shards, i encountered similar issues as the below thread, i am trying to figure out what is best to change from a daily indexing to monthly or yearly index. Can someone please guide me to it.

here is my cluster health:

"cluster_name" : "Diamond",
"status" : "yellow",
"timed_out" : false,
"number_of_nodes" : 3,
"number_of_data_nodes" : 1,
"active_primary_shards" : 1682,
"active_shards" : 1682,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 1682,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 50.0
Many thanks

2

Hey,

you can use the elasticsearch output configuration of beats to specify the index name incuding a timespace... see https://www.elastic.co/guide/en/beats/libbeat/current/config-file-format-namespacing.html

--Alex

3

Thank you Alex, appreciate your help, my current logstash beat conf output is:
output {
elasticsearch {
hosts => "192.168.1.1.10:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"

document_type => "%{[@metadata][type]}"

}

do i have to change it to the below to perform a weekly indexing:

output {
elasticsearch {
hosts => "192.168.1.1.10:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.ww}"

document_type => "%{[@metadata][type]}"

}

4

Maybe this will help someone:

use this

index => "%{[@metadata][beat]}-%{+yyyy.ww}"

instead of

index => "%{[@metadata][beat]}-%{+YYYY.MM.ww}"

green open winlogbeat-2017.11

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.