Horrible performance with elasticsearch and bro-ids


(James-4) #1

Topic says it...I have 6 terms and 1 histogram. I'm looking at bro-ids
conn.log piping direct to elasticsearch. I'm looking at just the previous
hour. It takes one minute, fourteen seconds to refresh the Kibana page.
My cpu is at:

top - 17:06:37 up 21 days, 2:22, 3 users, load average: 3.06, 1.72, 1.32
Tasks: 164 total, 2 running, 161 sleeping, 1 stopped, 0 zombie
Cpu(s): 20.1%us, 0.5%sy, 0.0%ni, 79.3%id, 0.0%wa, 0.0%hi, 0.0%si,
0.0%st
Mem: 4042516k total, 3913556k used, 128960k free, 206104k buffers
Swap: 5361660k total, 35572k used, 5326088k free, 1879196k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4865 elastics 20 0 15.0g 1.6g 518m S 162 42.6 24:10.02 java

CPU is Intel(R) Xeon(R) CPU L5420 @ 2.50GHz 8 core. Mem stats:

         total       used       free     shared    buffers     cached

Mem: 3947 3829 118 0 199 1848
-/+ buffers/cache: 1781 2166
Swap: 5235 34 5201

Any assistance would be wonderful thank you.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/395fa45b-2647-43b4-ad98-f2815bda1413%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(James-4) #2

So for those keeping score, setting

ES_HEAP_SIZE=

to something larger then default (256 megs) did a world of good.

On Thursday, August 7, 2014 11:18:01 AM UTC-6, digit...@gmail.com wrote:

Topic says it...I have 6 terms and 1 histogram. I'm looking at bro-ids
conn.log piping direct to elasticsearch. I'm looking at just the previous
hour. It takes one minute, fourteen seconds to refresh the Kibana page.
My cpu is at:

top - 17:06:37 up 21 days, 2:22, 3 users, load average: 3.06, 1.72, 1.32
Tasks: 164 total, 2 running, 161 sleeping, 1 stopped, 0 zombie
Cpu(s): 20.1%us, 0.5%sy, 0.0%ni, 79.3%id, 0.0%wa, 0.0%hi, 0.0%si,
0.0%st
Mem: 4042516k total, 3913556k used, 128960k free, 206104k buffers
Swap: 5361660k total, 35572k used, 5326088k free, 1879196k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4865 elastics 20 0 15.0g 1.6g 518m S 162 42.6 24:10.02 java

CPU is Intel(R) Xeon(R) CPU L5420 @ 2.50GHz 8 core. Mem stats:

         total       used       free     shared    buffers     cached

Mem: 3947 3829 118 0 199 1848
-/+ buffers/cache: 1781 2166
Swap: 5235 34 5201

Any assistance would be wonderful thank you.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f1dfad09-ddb6-444c-846b-baf174427be8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #3