Can you be more specific about the definition of a "powerful" server?
Note, there are also different numbers for different workloads (just
indexing few docs per sec, bulk indexing, just a few searches per sec, huge
search load etc) so it really depends.
Joerg, I think what Adam means is how many open indexes an ES instance can
hold before lagging or crashing (assuming max open files is set correctly
etc).
Small indexes usually mean less segments per index, so the question may
boil down mostly to number of open IR/IW?
Can you be more specific about the definition of a "powerful" server?
Note, there are also different numbers for different workloads (just
indexing few docs per sec, bulk indexing, just a few searches per sec, huge
search load etc) so it really depends.
Sure, you can create hundreds of thousands of indices, without docs, or
with just one doc. Closing an index even frees the resources of the index
management. This is not useful, just an edge case.
I'm afraid this is not the answer to the question.
The number of open indices (shards) is limited by disk space and maximum
number of file descriptors. But that is also theoretical. Using all indices
at once will use more resources than file descriptors. It depends on the
index / query characteristics (RAM for shards, fields, term count, term
distribution etc.) That is not directly related to "powerful" servers. Even
the weakest server can create as much indices as the strongest one.
Putting workload on indices is a different story.
Jörg
On Mon, Mar 3, 2014 at 11:21 AM, Itamar Syn-Hershko itamar@code972.comwrote:
Joerg, I think what Adam means is how many open indexes an ES instance can
hold before lagging or crashing (assuming max open files is set correctly
etc).
Small indexes usually mean less segments per index, so the question may
boil down mostly to number of open IR/IW?
Hi Joerg & Itamar,
Thanks for your reply. What I am trying to get is an order of magnitue.
Suppose a server has 20GB RAM. Each index has about 20K documents, each
document is about 500 words, average term distribution. The question is
whether the server would be able to smoothly handle tenths, hundreds, or
thousands (or more?) of open indices. Suppose each index gets about 10 new
documents per minute, and serves about 30 queries per minute.
Thank you!
בתאריך יום שני, 3 במרץ 2014 19:04:22 UTC+2, מאת Jörg Prante:
Sure, you can create hundreds of thousands of indices, without docs, or
with just one doc. Closing an index even frees the resources of the index
management. This is not useful, just an edge case.
I'm afraid this is not the answer to the question.
The number of open indices (shards) is limited by disk space and maximum
number of file descriptors. But that is also theoretical. Using all indices
at once will use more resources than file descriptors. It depends on the
index / query characteristics (RAM for shards, fields, term count, term
distribution etc.) That is not directly related to "powerful" servers. Even
the weakest server can create as much indices as the strongest one.
Putting workload on indices is a different story.
Jörg
On Mon, Mar 3, 2014 at 11:21 AM, Itamar Syn-Hershko <ita...@code972.com<javascript:>
wrote:
Joerg, I think what Adam means is how many open indexes an ES instance
can hold before lagging or crashing (assuming max open files is set
correctly etc).
Small indexes usually mean less segments per index, so the question may
boil down mostly to number of open IR/IW?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.