Multiple indexes or 1 index

Elastic Stack Elasticsearch
1

I am working on building a common index that can serve multiple applications. The use case is that I have a few relational tables (example: products, reviews, etc). This data needs to be served to multiple applications built around different usecases that needs to search for the same products table but search by different fields and return different results.

For example: I need the ability to let application A search only fields 1, 2 and 3 in products and return a document with 1,2,3,4,5 columns.

Another application B can search only by fields 3, 4, 5 and would need to return different document with different columns.

The same applies to reviews as well.

I was wondering if I can build a common index for both, since it is the same products table that I will be indexing. How do I control what can be searched by application. UI may not be the right approach to limit the user to search by other fields because if they know the searchable fields they can search by them. Also, how do I limit what is returned to application A vs application B, if I build only 1 index.

Building multiple indexes for the same products table defeats the purpose of building a shared platform.

TL;DR - how to build a shared index and serve multiple applications and control what they can search by. Can be done by the UI of the application of course, but is it the right approach. Is source filter the only way to limit the content returned? Is it a common use case to identify and build one giant superset index that solves all consuming applications needs and limit the search and results this way.

I am wondering what the best practices in this situation.

Thanks.

2

I think that the most elegant way is by adding X-Pack security feature (commercial plugin) where you can have a fine control (per field) of what a user can see/query and can not see/query.
Here each application will have his own user with the right roles and rights associated with it.

3

Thanks! But was wondering if the idea of building and indexing a superset of all the columns that would be needed across all applications, is that norm, would it cause any implications on applications that need only a few searchable columns and may take a performance hit compared to applications that need to search across more columns? Are there any downsides to creating multiple/separate indexes for different applications (even though it is going against the same database table), other than 1 downside of duplication of data?

4

I see. Unless you have tons of indices that should not be a problem either way.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.