Thanks for your reply, see comment in line...
Den torsdagen den 2:e oktober 2014 kl. 23:52:13 UTC+2 skrev Mark Walkom:
You should drop your heap to 31GB, over that you don't compress java
pointers and GC will suffer.
Ok, I thought the limit was 32? I'll drop to 31 then.
- That will help as it reduces document size.
*I'll investigate what fields can be removed and then drop them. *
- Definitely, if you don't need near-realtime access then open it up,
we run at 60sec but could probably go to 2 or more minutes.
Will reduce to 10s to see if it makes a difference.
- This could be risky, doubly so given you are only running two nodes.
To elaborate on point 3, and on general note, you should really run 3 or
more nodes incrementing up to odd numbers (3,5,7,9....). The reason for
this is that is that it helps to prevent split brain across your cluster.
I actually have three nodes for reason stated above but only two data
nodes, third node is slower HW.
You should use disable bloom filtering on your indexes as it will give you
a bit of a boost, Elasticsearch curator can handle that for you.
Interesting, haven't seen this option before, thanks!
However you are probably reaching the limits of your cluster, 2 billion
docs is a fair bit of data. What versions of ES and java are you on? What
amount of data (GB) is it?
I'm running ES 1.3.2 and Java 1.7.0_65 and collect ~70GB per 24h.
I'll test the suggestions above but will add two more nodes anyway as we
have more logs to through in.
Thanks for your input, I'll post the result when done!
Br
Mathias Adler
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com <javascript:>
web: www.campaignmonitor.com
On 3 October 2014 00:04, Mathias Adler <mathia...@rebtel.com <javascript:>
wrote:
Hi All,
I'm quite new to ES and using it as part of the ELK stack. I'm running a
two node cluster where each node got 24core and 64GB RAM (32 allocated to
Java), We have an index rate @ ~3000/s and total document count per 24h @
200miljon (and storing 10days).
When mem usage gets close to 80% I start having search problems in Kibana
and get all kinds of exception and all gets better when mem usage gets
lower again.
So, one way is of course to scale out, but my question is, what mem
tuning can be done and will it make a big difference?
1, Drop unused data fields in logstash already, will that make any
difference?
2, Reduce index.refresh_interval, will that reduce mem usage?
3, Set replicas to zero and get all primary shards spread over both
nodes, will that impact mem usage?
What else can be done to lower mem usage?
Anyone out there having the same type of load or higher (document count
and index rate), how does your set up look like?
Br
Mathias
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2ba446f8-6017-4863-b462-a075b60780f2%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/2ba446f8-6017-4863-b462-a075b60780f2%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b68e68d5-93cc-4350-8f17-de137010fb4e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.