Hi,
We are having Elasticsearch cluster of 5 nodes.
8GB Ram, 4 Cores each.
Elasticsearch is configured to take 4GB heap size.
Our indexes are daily managed, each contains about 22 Different Types of log files.
(*) Extracting Fields from 22 different log files.
Size of such a Daily index is about 250GB.
Nodes are as follows:
2 Nodes are Master + Ingest
1 Node is Master + Data
2 Nodes are Data only
Each index is 6 Primary Shards + Replication Factor of 1.
3 Data nodes, each data node will have 2 Primary Copies and 2 Replicas. Total of 4 Shards per index.
Kibana is installed on all data nodes, for redundancy, one kibana index for the cluster.
I am creating Aliases with filters according to the different log types, creating logic groups of several types.
Thoses logic groups, are then added to the Kibana Index patterns, which the users are using to search within.
The Search performance seems to be poor, taking some time to run and retrieve results to the Kibana.
My Questions:
How can I optimize the search ? Which Metrics should I query ?
Are there any Elasticsearch parameters worth changing to support better search performance (The Indexing performance should be degraded as well) ?
Thanks,
Ori