ElasticSearch Optimizing questions

Hey,

  1. Please be more specific. You still should provide queries and reproducible examples in order to understand the issue and not just write text. Also, how do you know and figured out that 'the whole database is retrieved'? Can you explain, what makes you think that (I have no idea how I would try to validate that claim, but maybe you found some evidence, so I am interested in that)? Also, please paste the exact query that you are testing this (this is what I referred to as context, as this helps to understand what you are after).

  2. My advise here: Do not tune any GC parameters, unless you fully understand what they are for. Configuring the heap size if fine. But if you run into issues with performance, this is extremely rare based on the default GC configuration. I know there is a lot of content out in the internet, but be careful about that.

  3. The same applies as for the first point. More context is needed. From what I see based on that data, you may want to split your shards a little smaller, as you know have a single shard with 190gb in size. That said, this does not explain why your scroll search is getting slower. Can you provide the initial query as well, that you used with scroll search? Are you trying to do a full export of your data? Also, about what dimension of slow are we talking about? How fast is it at the beginning, how does it decrease? Does it hang or is it still running, albeit slow?

Also, can you explain your node setup? You only seem to have two shards, so maybe not all of your nodes in the cluster are getting utilized properly - or is this a single node setup?

--Alex