Elastic search, multiple indexes vs one index

Elastic Stack Elasticsearch
1

Hi Everybody,
I am trying to use Elasticsearch to store objects which all have similar fields and some field values are same in various objects. I have location specific data with me.

For ex. For a lot of objects a location field (city) would be same. In the end every object would be associated with location and neighbourhood and other localised features. I want to understand how to create indices properly wrt speed of search query and also respecting data storage issues.
According to me and i might be completely off track.

Strategy-1:

-Create a single index in which all objects would be stored and when we want to query we can filter the index based on various fields.

Strategy-2:

  • Create indices based on locations and have objects inside them. This would speed up my query but would increase indices , I am not sure how the memory usage gets affected by this one.

Just to give it more perspective , the locations data i have currently is around 50-60 but might increase to 500 or more.

Currently i am getting more inclined towards the second strategy as it would speed up my search queries as well when i want to get aggregation stats but not sure how it would scale up efficiently.

Thanks

2

Are you having performance issues now?

Also, why not look at routing with option 1?

3

Thanks for the reply.

Quite frankly. I have just started with Elasticsearch , initially i have a postgres but for faster searches and statistics based on fields. I am moving to Elasticsearch.
So currently i am trying to develop a mapping such that its
1.) Get statistics like top 5 locations with lowest rent fast (I still don't know how to use aggregation for this though :slight_smile:
2.) Use data visualisation effectively for ex. plotting a line chart only for a particular location.

4

Put your data into ES and then use Kibana to build a visualisation of what you think you want.
Then you can view the query and see how the actual agg is built and copy that :slight_smile:

5

Thanks !
But how do you get queries from Visualization for ex. data table (we can use this to get top 5) but how do you find the elasticsearch specific query for it.

6

There's a little ^ arrow at the bottom of each visualisation, click that and there is a Request button, click that.

1 Like
7

Thanks alot ! I was able to get the query. :blush:
For now i am thinking of going ahead with multiple indexing , it will aid in visualisation location wise. Do you see any significant downside in this approach.