Performance with an arbitrary number of indicies

Elastic Stack Elasticsearch
1

Lets say that I was providing a service to customers that requires
customers to sign up to use my service. I'm also using elasticsearch to
store analytics data about each customer.
The number of documents recorded for each customer is completely arbitrary
(some customers may have 1million + documents, some may have less than
1000).

Which solution would be better?

  1. Give each customer their own index
  2. Put all customer's data into a single index

Argument for solution #1: In my personal tests, I can see that if I have
1million documents in index A, and 1000 documents in index B, then queries
run on index B aren't being slowed down by the amount of documents in index
A. However, when I put them all into the same index and separate them by a
unique key, then index B's queries are slowed down by the number of
documents in index A.

*Argument for solution #2: *I've only heard that having an arbitrary number
of indicies is bad for performance in the long term.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/15de33c8-d0e5-484f-9c16-f68be2bdb005%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Why not use aliases?

This way you can move larger customers to their own index if need be.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 11 October 2014 01:54, jnortey jeremy.nortey@gmail.com wrote:

Lets say that I was providing a service to customers that requires
customers to sign up to use my service. I'm also using elasticsearch to
store analytics data about each customer.
The number of documents recorded for each customer is completely arbitrary
(some customers may have 1million + documents, some may have less than
1000).

Which solution would be better?

  1. Give each customer their own index
  2. Put all customer's data into a single index

Argument for solution #1: In my personal tests, I can see that if I
have 1million documents in index A, and 1000 documents in index B, then
queries run on index B aren't being slowed down by the amount of documents
in index A. However, when I put them all into the same index and separate
them by a unique key, then index B's queries are slowed down by the number
of documents in index A.

*Argument for solution #2: *I've only heard that having an arbitrary
number of indicies is bad for performance in the long term.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/15de33c8-d0e5-484f-9c16-f68be2bdb005%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/15de33c8-d0e5-484f-9c16-f68be2bdb005%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624YQG0vKqDVH8t6k4MbJXJJ7%3D41OvHvw93eW46eTbLn03w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.