3 nodes cluster

[Vladislav_Trayanov]

Hey all,

I have a big trouble to understand the best 3 nodes cluster elasticsearch configuration.

At the moment I have 3 node cluster with 3 data + master eligible nodes with 4 gigs of JVM and standard HDD. I keep logs for 45 days. I think the data is not more 20GB

{
"cluster_name": "elasticsearch",
"status": "green",
"timed_out": false,
"number_of_nodes": 3,
"number_of_data_nodes": 3,
"active_primary_shards": 1311,
"active_shards": 2622,
"relocating_shards": 0,
"initializing_shards": 0,
"unassigned_shards": 0,
"delayed_unassigned_shards": 0,
"number_of_pending_tasks": 0,
"number_of_in_flight_fetch": 0,
"task_max_waiting_in_queue_millis": 0,
"active_shards_percent_as_number": 100
}

My biggest problem is that the setup it doesn't seems optimal since the nodes keep crashing because of the JVM or high load etc. especially when some of my fluentd was down for a while and then I start it. It keeps my nodes under high pressure.

So my point is to leave one only Master node and 2 only data nodes.
What do you think, is it better or you can suggest something else?

Thanks,
Vlad

[Vladislav_Trayanov]

or maybe I need coordinating node?

[dadoonet (David Pilato)]

I think the data is not more 20GB

In total?

Definitely 2622 shards is a way too much.
You need to reduce the pressure.

1 single shard can probably hold 20gb...

So I'd use 1 shard per index. With daily indices, you'll end up with 90 shards on 3 nodes which sounds more reasonable.

[Vladislav_Trayanov]

Hey, thanks for your answer.
At the moment I have 5 shards per index, so I need to decrease them to 1 right?

{
"logfiles-2017.12.12": {
"settings": {
"index": {
"creation_date": "1513036859653",
"number_of_shards": "5",
"number_of_replicas": "1",
"uuid": "q08OO2GWQJSuvkqW9FUPiQ",
"version": {
"created": "6000099"
},
"provided_name": "logfiles-2017.12.12"
}
}
},

[dadoonet (David Pilato)]

Yes.

[Vladislav_Trayanov]

Thank you a lot David. I think this will help me a lot.