Hi
We have a multi tenant SaaS solution and we expose our use of elasticsearch through an API. We noticed that clients who use the API directly, and not through our front-end, don't bother with paging and just use a count of 500k or 1m even if the actual count is much lower. I was wondering about the effect of not limiting the count since we're thinking of making a breaking change and limit this to 100.
I am still not expert an in ES but surely when not paging the process time
will be higher because more documents have to be brought back in the
response. However, depending on what kind of queries you perform, the
subsequent queries will be faster. I am thinking of filter bool queries
with caching and bitset query representation. I doubt that a size of 1m
will have any effect if there are only 100 documents indexed.
I wouldn't advise removing the paging for the sake of bandwidth and network
traffic especially if you have many clients using the same server, may be a
very optimized server setup can help.
We do a full query only when doing statistical calculation requiring all
documents.
On Wednesday, December 3, 2014 11:40:05 PM UTC-5, Ron Sher wrote:
Hi
We have a multi tenant SaaS solution and we expose our use of
elasticsearch through an API. We noticed that clients who use the API
directly, and not through our front-end, don't bother with paging and just
use a count of 500k or 1m even if the actual count is much lower. I was
wondering about the effect of not limiting the count since we're thinking
of making a breaking change and limit this to 100.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.