Multi-tenacy

Elastic Stack Elasticsearch
1

Hi,

I am considering using elastic search in a multi-tenant application,
should I have any concerns using an index per customer? Will thousands
of seperate indexes cause any problems? Other than those scaling can
fix?

Also can I secure each index separately?

Any documentation/samples for multi-tenacy.

Best.

Mike

2

Hi Mike,

Multi tenancy is mentioned here in the doc in you haven't seen it already
http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/#Multiple_Indices_and_Types

API makes it easy to work with that type of configuration. I can't comment
on the impact of having thousands of indices, but if the twitter example is
any indication, this should not be a problem.
http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/#Multiple_Indices_and_Types
Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skype

On Wed, Mar 24, 2010 at 12:56 PM, Michael olivieri.nicole@gmail.com wrote:

Hi,

I am considering using Elasticsearch in a multi-tenant application,
should I have any concerns using an index per customer? Will thousands
of seperate indexes cause any problems? Other than those scaling can
fix?

Also can I secure each index separately?

Any documentation/samples for multi-tenacy.

Best.

Mike

3

Hi,

Multiple indices support is basically bounded by memory on each
node/server. Each node in the cluster holds the cluster state (and the
derived state from it, for example, a parsed version of the mapping
definition). The cluster state includes all the indices, their mappings and
so on. To be honest, I have not tried to profile and see what the impact of
having thousands of indices, but the system is certainly built to handle it
if memory permits. Moreover, its going to be different from one app to the
other, since, for example, the mappings used (or automatically created)
affect the final overhead of each index within the cluster state.

I do know of places that can be optimized to retain less memory, but
thats the perfect case of premature optimization. What I would suggest is,
if you can, run a simple test, with the amount of servers, and see how it
holds. I am certainly here to help and optimize if needed. I do plan to run
a similar test as soon as I can.

In terms of scaling, assuming each node has enough memory to store the
cluster state, then yes, the more nodes you will add to the system, the
better.

Last, as for security, there isn't one built into elasticsearch, so you
will have to build it on the client side. I definitely plan to add security
features to elasticsearch, as multi tenancy demands it, but its not going to
be soon (i.e. in the next couple of months). As a side node, see Hadoop for
example, they just started to work on a security model :), and they are the
perfect example for multi tenancy.

-shay.banon

On Wed, Mar 24, 2010 at 7:47 PM, Berkay Mollamustafaoglu
mberkay@gmail.comwrote:

Hi Mike,

Multi tenancy is mentioned here in the doc in you haven't seen it already

http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/#Multiple_Indices_and_Types

API makes it easy to work with that type of configuration. I can't comment
on the impact of having thousands of indices, but if the twitter example is
any indication, this should not be a problem.
http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/#Multiple_Indices_and_Types
Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skype

On Wed, Mar 24, 2010 at 12:56 PM, Michael olivieri.nicole@gmail.comwrote:

Hi,

I am considering using Elasticsearch in a multi-tenant application,
should I have any concerns using an index per customer? Will thousands
of seperate indexes cause any problems? Other than those scaling can
fix?

Also can I secure each index separately?

Any documentation/samples for multi-tenacy.

Best.

Mike