Instead of having winlogbeat produces daily index, we can create separate index for separate winlogbeat clientlogs? Or any other idea?
Creating lots of small indices and shards is inefficient, wastes resources and will cause performance problems down the line. Why do you want to do this?
The reason is,
we have 33 clients and one server we using winlogbeat to ship log from the client and send it to logstash and then to elasticsearch
so we want to create 33 separate index for 33 client(logs) to store
Then we need to SEARCH specific client machine logs means easy for the user know that's why we are recommended to do this?
any idea?
We are using java High client Api to fetch data
I would recommend storing the data in a single index together with a tag identifying the user and then add a filter when you query. If you need separate indices, make sure you reduce the number of primary shards to 1 and switch to e.g. monthly or weekly indices instead of daily (depending on your retention period).
Now i had a Question with your idea
You said that 'tag identifying the user' what does it mean? can you expalin?
How to change creating daily indices to monthly or weekly?
We store in a single index the index have all the 33 client logs with the index named with winlogbeat-version-date if it's create daily indices
Then where to tag?
Add a field to each document that indicates the client. You should be able to to this in Logstash or through an ingest pipeline.
You are saying that to create extra fields in the logfile from the client
Yes.
How to do?
I need to know this is possible or not? to create separate index for separate machine(to clarify doubt)
By default, an elasticsearch output will use daily indices. The default value for the index option is "logstash-%{+YYYY.MM.dd}". You could change that to monthly using
index => "logstash-%{+YYYY.MM}"Okay
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.