I quite often find that the real number of actually simultaneous things is
often fewer than you might first think - each query only lasts a few ms,
often the results can be cached using squid or varnish, lots of times people
are searching common content for the same stuff, each of which reduces the
number of simultaneous queries that you have to do.
Actually I'm indexing email, so these thougts that may be valid for a
public index of common materials doesn't work.
People may search the same thing, but they will at least search in a
differente filtered part of the index or in a totally different index.
My design will is to create an index for each mail domain, and to add
the destination as a filter to get the mailbox contents. This lets me
share identical emails sent to more than one destination on the same
domain.
It even lets me have more security in case of an ES (or query, more
probably) bug. In fact if you search on the same domain you search on
a specific index. That may let me be sure that, at least, if the
query/index goes wrong, I may not get results from other companies.
In order to support
100's of users, you might find that really only 1 or 2 queries are actually
happening at the same time... I mean YMMV...
OK, that's a good starting point, thanks :). Actually for what I know
we're dealing with from 10k to 100k mailboxes with various
dimensions... It means that we may get, dealing with your suggestions,
something like from 100 to 1k simultaneous queries. Beware that I'd
accept now even to be slow, it does'nt matter.
My philosophy is that if something works slowly but is stable, it can
be improved. If it is not stable it is useless optimize it, as it will
be always unstable...
Is that right or with ES I should change my approach?
Thanks!
On 3 October 2013 16:56, Massimiliano Perantoni
massimiliano.perantoni@gmail.com wrote:
Actually we'll have to stand the load of hundreds of simultaneous
queries...
2013/10/3 James Richardson james@time4tea.net:
Well, I have tested that it works with the load I need. I don't really
see
the point in creating some imaginary load levels and just stressing it
out.
If it works for what I needs that's good enough... The more important
thing
(for me in this instance) is that it works enough and the heap size is
small enough that it can run on a cheap vm.
James
On 3 October 2013 16:47, Massimiliano Perantoni
massimiliano.perantoni@gmail.com wrote:
Have you ever stressed the engine to test its reactions?
2013/10/3 James Richardson james.time4tea@gmail.com:
For what it's worth, i am searching 1m documents, about 5G, using a
max
heap
of 350m and its all working just fine. This includes facets over the
entire
index...
--
You received this message because you are subscribed to a topic in
the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
