So you mean that CPU usage is high on Couchbase nodes but not on elasticsearch, right?
Or is the high CPU usage on elasticsearch nodes?
Did you try to run the hot threads command suggested by Jason?
What are your elasticsearch settings? How much RAM did you give to the heap?
--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr
Le 5 décembre 2013 at 14:21:11, Corey Nolet (cjnolet@gmail.com) a écrit:
David,
I pulled the elasticsearch nodes from the couchbase nodes but I didn't notice any difference in the CPU usage on the couchbase nodes.
On Thursday, December 5, 2013 2:14:56 AM UTC-5, David Pilato wrote:
I did not understand if the load comes from elasticsearch process or from couchbase process?
I guess you already check that, right?
Note: I'd isolate elasticsearch on another nodes, mainly because of IO. I guess that couchbase has a lot of reads to do and in the same time elasticsearch has a lot of writes.
--
David 
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 5 déc. 2013 à 07:57, Jason Wee peic...@gmail.com a écrit :
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-hot-threads.html
curl on your es nodes and that should give hint what is hogging the cpu.
hth
/Jason
On Thu, Dec 5, 2013 at 9:44 AM, Corey Nolet cjn...@gmail.com wrote:
I've got a 6 node HDFS cluster. Each node has 8 cores and 32gb of ram. On the same nodes, I'm running Couchbase 2.2 and Elasticsearch 0.90.5 with the Couchbase transport plugin 1.1.0. I have Couchbase doing real-time ingest/caching from Twitter Storm and indexing the documents in elasticsearch for further mining. I'm using about 18gb of ram from my entire cluster for Couchbase. Without ElasticSearch running, Couchbase seems to idle between 1% and 16% CPU utilization with occasional (expected) spikes at around 30% to 60%. I'm okay with that.
What I'm trying to figure out, however, is why the XDCR to ElasticSearch seems to pump my nodes up to a constant 60% to 99% CPU utilization. I've tried lowering the max concurrent replications to 2 and even changing the checkpoint sizes and I've noticed no decrease in utilization. The utilization has been so high, in fact, that my Zookeeper nodes can't keep a lock and they start expiring important connections.
I tried pulling Elasticsearch off of my 6 nodes and putting it elsewhere (on a separate 4 nodes). This also didn't seem to have an impact. I read on a slideshow that 24-cores is recommended but that's a little absurd of a requirement. I'm doing about 250,000 documents every 3 minutes or so and ELasticsearch seems to be keeping up. The short of it is... i really just need the ability to do term queries of the documents and N1QL isn't nearly as fast with the intersecting searches as I need it to be (i'm looking for subsecond here).
Any ideas? I've done a ton of research and all I've seen are issues in verisons <2.2 in jira that appear to have been fixed.
Thanks!
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a6920681-dd08-41f9-a781-c17bcefdcdb1%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7aae5306-cbd1-45c8-83f6-9f02533101b9%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52a08309.14330624.bd3d%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.
