I increased the refresh interval to 60sec.. that didnt help.. i see bunch
of error messages in elasticsearch.log file that look like below. could
that be the reason for slow search? now i see slowness even when there is
not much indexing happening. these messages occur twice/thrice a minute.
[2012-07-09 00:00:55,238][WARN ][index.merge.scheduler ]  [facebook] failed to merge
java.io.IOException: Input/output error:
Caused by: java.io.IOException: Input/output error
at sun.nio.ch.FileDispatcher.pread0(Native Method)
... 12 more
also, i was not able to install bigdesk - due to below error.
sudo bin/plugin -install lukas-vlcek/bigdesk/1.0.0
-> Installing lukas-vlcek/bigdesk/1.0.0...
Failed to install lukas-vlcek/bigdesk/1.0.0, reason: failed to download
i would really appreciate any help i can get here..
On Sat, Jul 7, 2012 at 8:08 AM, Radu Gheorghe firstname.lastname@example.org:
On Saturday, July 7, 2012 3:05:03 AM UTC+3, T Vinod Gupta wrote:
I have a single node ES (0.18.7 version) setup. unfortunately, i didn't
change the default config much and it has 5 shards.. and now i have quite a
bit of production data stored on it (12GB). what we are seeing is reduced
throughput over time and search times sometimes as high as few minutes.. im
looking at some help on how to bring the situation under control as we are
constantly indexing data and also serving realtime customer requests.
is it possible to reduce the number of shards from 5 to 2 somehow?
does that work once the system is already in place?
i read somewhere that it could be due to threadpool pressure. but the
node stats ( curl -XGET 'http://localhost:9200/_**
**') is not giving thread pool information. how do i get around
identifying the root cause?
I would start by looking at BigDesk and in the logs.
- my throughput is around 300-400 index calls per sec. how do i make it
It depends a lot on how you data looks like. But increasing the
refresh_interval should always help.
- if i were to optimize such that my gets and search calls are faster,
is it possible? it can be at the expense of slower index calls.
How does your data and searches look like?
If you find your storage slow, you might benefit from compressing your
source. I would also try upgrading ES to a newer version. I find it faster,
although I don't have a clear benchmark to show that. Please note that
upgrading needs some care. Quote:
- Upgrading from 0.18 requires issuing a full flush of all the indices
in the cluster (curl host:9200/_flush) before shutting down the
cluster, with no indexing operations happening after the flush.
- The local gateway state structure has changed, a backup of the state
files is created when upgrading, they can then be used to downgrade back to
0.18. Don’t downgrade without using them.
this is on a dual core machine (ec2 m1.large instance) and i gave ES 4GB
ram. has there been any benchmarking done on ec2 instances.
let me know if any further info is needed.