Elasticsearch client using too much cpu

Elastic Stack Elasticsearch
1

hi,
im pretty sure i am doing something wrong here as it doesnt make sense. can
someone tell what it is?

my ES client is taking 100% cpu on a dual core ec2 m1.large instance (so
effectively 50%). while the actual elasticsearch process is hardly taking
any cpu.

my client is designed in the following way - there is one ES client worker
thread which instantiates ES Client once in its constructor. Inside the run
method, it dequeue messages from a local java blockingqueue and uses the
client to do 2 things - a prepareGet and then prepareIndex. if it gets
something from prepareget, it uses the sourcefield map to update the new
document along with new fields that it wants to update. and then does
prepareIndex.execute().actionGet();

there is only 1 thread talking to ES service. is this expected?

thanks

2

Can you profile the client and see where CPU is spent? VisualVM has a nice sampling profiler without needing to set anything up.

On Sunday, February 19, 2012 at 9:27 PM, T Vinod Gupta wrote:

hi,
im pretty sure i am doing something wrong here as it doesnt make sense. can someone tell what it is?

my ES client is taking 100% cpu on a dual core ec2 m1.large instance (so effectively 50%). while the actual elasticsearch process is hardly taking any cpu.

my client is designed in the following way - there is one ES client worker thread which instantiates ES Client once in its constructor. Inside the run method, it dequeue messages from a local java blockingqueue and uses the client to do 2 things - a prepareGet and then prepareIndex. if it gets something from prepareget, it uses the sourcefield map to update the new document along with new fields that it wants to update. and then does prepareIndex.execute().actionGet();

there is only 1 thread talking to ES service. is this expected?

thanks

3

attaching the visualvm profiler dump in xml.. i have never used this tool
before, so not sure if i got the right data.. but it looks like lot of time
spent in threadpoolworker run and a future in es.

thanks

On Mon, Feb 20, 2012 at 5:09 AM, Shay Banon kimchy@gmail.com wrote:

Can you profile the client and see where CPU is spent? VisualVM has a
nice sampling profiler without needing to set anything up.

On Sunday, February 19, 2012 at 9:27 PM, T Vinod Gupta wrote:

hi,
im pretty sure i am doing something wrong here as it doesnt make sense.
can someone tell what it is?

my ES client is taking 100% cpu on a dual core ec2 m1.large instance (so
effectively 50%). while the actual elasticsearch process is hardly taking
any cpu.

my client is designed in the following way - there is one ES client worker
thread which instantiates ES Client once in its constructor. Inside the run
method, it dequeue messages from a local java blockingqueue and uses the
client to do 2 things - a prepareGet and then prepareIndex. if it gets
something from prepareget, it uses the sourcefield map to update the new
document along with new fields that it wants to update. and then does
prepareIndex.execute().actionGet();

there is only 1 thread talking to ES service. is this expected?

thanks

4

Kind of hard to tell based on that, maybe you can do more "heavy" profiling of the app?

On Monday, February 20, 2012 at 8:57 PM, T Vinod Gupta wrote:

attaching the visualvm profiler dump in xml.. i have never used this tool before, so not sure if i got the right data.. but it looks like lot of time spent in threadpoolworker run and a future in es.

thanks

On Mon, Feb 20, 2012 at 5:09 AM, Shay Banon <kimchy@gmail.com (mailto:kimchy@gmail.com)> wrote:

Can you profile the client and see where CPU is spent? VisualVM has a nice sampling profiler without needing to set anything up.

On Sunday, February 19, 2012 at 9:27 PM, T Vinod Gupta wrote:

hi,
im pretty sure i am doing something wrong here as it doesnt make sense. can someone tell what it is?

my ES client is taking 100% cpu on a dual core ec2 m1.large instance (so effectively 50%). while the actual elasticsearch process is hardly taking any cpu.

my client is designed in the following way - there is one ES client worker thread which instantiates ES Client once in its constructor. Inside the run method, it dequeue messages from a local java blockingqueue and uses the client to do 2 things - a prepareGet and then prepareIndex. if it gets something from prepareget, it uses the sourcefield map to update the new document along with new fields that it wants to update. and then does prepareIndex.execute().actionGet();

there is only 1 thread talking to ES service. is this expected?

thanks

Attachments:

  • ESProfilerResults1.xml