Tuning ELK Performance and controlling Logs

Elastic Stack Elasticsearch
1

Hi All,

I need solution or guidance for below queries, to improve the performance. Kindly help.

  1. We can control logs in kibana, like log only if Error, is there any other way to do the same for logstash?
  2. Similarly Is it possible to deprecate index in run time, Like, i want to clear index data that are older than a week?
  3. Also my Elasticsearch data file is around 50Gb, is there any method to reduce the size other than deleting the index.
  4. There is only one node currently available, how to create more nodes and replicas? Do i need to run another elasticsearch instance in same machine?
2

Similarly Is it possible to deprecate index in run time, Like, i want to clear index data that are older than a week?

You might want to look at curator: GitHub - elastic/curator: Curator: Tending your Elasticsearch indices

Also my Elasticsearch data file is around 50Gb, is there any method to reduce the size other than deleting the index.

Running optimize might help a bit, curator can ke used to automate it.

There is only one node currently available, how to create more nodes and replicas? Do i need to run another elasticsearch instance in same machine?

You can have multiple shards on a single node. However having multiple replicas of the same shard is not helpful on a single node since you can lose both at once. You should look into starting up another node on another machine.

3

Hi Adrien,

Thanks a lot will check on that.

Is there any other way like adding in few parameter/function in config. will be really helpful.