Hey all, we had a query that returned only a handful of results but needed all that it could find, so we had the size parameter set to 2147483647 (int max value).
Of course that is not good practice and even had to tweak maxresultwindow in order to be able to get it working in 5.x (legacy code). We fixed this by putting a top of 9999 results on all queries of this type.
We found two interesting things:
-
The query in question went from 90-300ms to 15-30ms of execution time. Dunno why because they return the exact same results.
-
Since we switched from int maxvalue to 9999 we see the heap percentage holding on better in two ways: we don't see any GC overhead notices in the logs, and it takes longer for the heap to reach 75% (before, 12hs to 22hs, now 30hs+), that's when old gen GC kicks in.
Now, the not so good part is that prior the change old gen GCs used take no more than 300ms and now they take between 3s to 4s.
What i want to know:
- Is it a normal collection time?
- Why does size affect query response times if it returns the same results?
- Why reducing size parameter makes GC collections take longer?
My setup:
- ES 5.3.2
- Oracle JDK 1.8u55
- Window Server 2012 R2
- 3 nodes cluster.
- Two main indices, 3 shards and 3 replicas each. 10GB total.
- Each node has total 16GB RAM with 6GB reserved to ES.
Thanks a lot in advance!