i am running a single elasticsearch host with nearly 1 billion documents (apache logs) in it
All documents are stored in different indices that are rotated daily: a_log-YEAR.MONTH.DAY
I do close all unused indices and open them if i need 'em.
I have some large Kibana visualizations where Elasticsearch needs to filter and count a lot Data.
Sometimes Elasticsearch crashes. The service is running on a hypervisor with 4 cores and 25gb ram.
How much RAM do i need to handle so many documents. And how many nodes should i add? (to my single on )