ES version: 5.5.1
Is this a coincidence or something else?
Because we leverage the OSs ability to cache commonly accessed files in the remainder of the memory.
So... how do I decide the heap size of ES? 2gb by default is ok?
That's a massive - it depends.
What sort of data, how much, what sort of queries, what version, what JVM, what SLAs do you have?
The data is nginx access log, about 100gb every day, java is open-jdk-1.8.0.
I use ELK to collect and analysis nginx logs, how to get the info you mentioned above...?
We generally use 3/4 of the physically available RAM as the JVM HeapSize for Elasticsearch..
More RAM = Better as Lucene can keep more segments in RAM.
But depends on the queries.. If you create indeces per day and only access the last few days you wan´t need as much RAM as if you access all the data all the time.. Ideally you want to be able to keep all interactively browsed data in memory.
As much as i love Free Software, don´t use open-jdk it´s much slower than the properitary OracleJVM..
Really?
It's very slow when I view the dashboard in 7 days...
The out of the box settings for Elasticsearch do not have the JVM triggering a garbage collection until the heap reaches 70%. The goal for most users of Elasticsearch should be to keep the heap at or near 50% after a GC has completed. If the heap is near, or above 70% on a regular or continual basis (or cannot clean up RAM after a GC so it stays around 70%), that's evidence of memory pressure, which indicates a need for more heap, or more nodes, or perhaps both.
Great!!! Thanks a lot for your explanation, remove many my doubts! 
@warkolm @Spacefish @theuntergeek
By the way, what about the heap size of logstash?
What about it exactly would you like to know?
Maybe it should be percent of physical memory?
Not really, it only needs to hold a small amount of data in memory, so unless you have very large events, a few GB should be sufficient.
How to check the size of events?
Depends on so many cases.
A Basic one: A single line in your log file is an event. The size of the line becomes the size of the event.
It varies if you are creating new variables, or using multi line and so many other ways
Thanks, I found Events Received and Events Emitted on monitoring of kibana...
Replace openjdk by oracleJDK, load average decrease much, amazing...Thanks a lot!!!
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.