Some typical things to check:
- Make sure the ES (java) process is not swapping. You can force it not to swap by setting bootstrap.mlockall to true.
- Which operating system are you running? Ubuntu 10.04 is known to have problems on ec2.
- Are you indexing while you do the search?
- Where do you store the data directory? Is it on local drive or EBS? Maybe EBS is misbehaving?
On Tuesday, June 14, 2011 at 10:28 PM, Eric Mill wrote:
Any ideas of why a query might work fine, and then suddenly spike out of control? I have the logs, running at DEBUG level, from an example incident. The same query gets executed, and on the 10th or so run, the memory starts piling up, and top shows the process taking anywhere from 100 to 177% of the CPU while this is happening.
ElasticSearch running out of control · GitHub
I can get it to occur pretty easily after I restart the server, but only on my EC2 instance. On my local machine, it's fine.
The query is a dis_max of 6 text/phrase queries on different fields, on an index of about 17,000 documents, where each document ranges from a few kilobytes to a few megabytes (not many of those). The data directory takes up 2.4GB of space.
Sorry for all the emails - I guess today's my day for questions!
-- Eric