Optinal way to index and search data

Elastic Stack Elasticsearch
1

Lets say that I am creating an application that will search text from a huge data set.
Say my document is something like this:

{
"type": "string", 
"message": "String"
.......
}

The type field is used to limit my search to a defined list and pass in the filter query only the types I want to search, then I perform a text search for message. I expect my type set to grow very big.

I can do 2 things.

  • Have one index per type
  • One index for all types and many partitions.

If one index per type, do many queries in parallel depending on my desired type, however I will have a huge number of shards.

Else having one index for all types, I'm worried that as data grows, the query time will increase because of all the data the query has to go through and filter types.

What is the best method and where do you find the balance?

Thanks a lot!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.