ES slowness and calls taking a long time to return

Elastic Stack Elasticsearch
2

Hi,

On Saturday, July 7, 2012 3:05:03 AM UTC+3, T Vinod Gupta wrote:

Hi,
I have a single node ES (0.18.7 version) setup. unfortunately, i didn't
change the default config much and it has 5 shards.. and now i have quite a
bit of production data stored on it (12GB). what we are seeing is reduced
throughput over time and search times sometimes as high as few minutes.. im
looking at some help on how to bring the situation under control as we are
constantly indexing data and also serving realtime customer requests.

questions

  1. is it possible to reduce the number of shards from 5 to 2 somehow? does
    that work once the system is already in place?

  2. i read somewhere that it could be due to threadpool pressure. but the
    node stats ( curl -XGET '
    http://localhost:9200/_clustr/nodes/stats?pretty=true') is not giving
    thread pool information. how do i get around identifying the root cause?

I would start by looking at BigDesk and in the logs.

  1. my throughput is around 300-400 index calls per sec. how do i make it
    higher?

It depends a lot on how you data looks like. But increasing the
refresh_interval should always help.

  1. if i were to optimize such that my gets and search calls are faster, is
    it possible? it can be at the expense of slower index calls.

How does your data and searches look like?

If you find your storage slow, you might benefit from compressing your
source. I would also try upgrading ES to a newer version. I find it faster,
although I don't have a clear benchmark to show that. Please note that
upgrading needs some care. Quote:
Upgrade Notes:

  • Upgrading from 0.18 requires issuing a full flush of all the indices
    in the cluster (curl host:9200/_flush) before shutting down the cluster,
    with no indexing operations happening after the flush.
  • The local gateway state structure has changed, a backup of the state
    files is created when upgrading, they can then be used to downgrade back to
    0.18. Don’t downgrade without using them.

this is on a dual core machine (ec2 m1.large instance) and i gave ES 4GB
ram. has there been any benchmarking done on ec2 instances.

let me know if any further info is needed.

thanks