Localized data with Shard Knowledge


(ElasticSearch Users mailing list) #1

Hi,

I have a query about using ElasticSearch.

I have part of my data (about 90%) to be indexed in ElasticSearch and the
remaining 10% to be stored elsewhere (like a DB/NoSQL DB etc.) since that
10% data will be very frequently changing data.

I have the following queries:

1.] Should I be keeping the remaining 10% of my data on non-ES/Lucene
persistent store? Is it okay to store/index those fields in ES/Lucene even
if they are frequently updating? With NRT readers, will it be efficient at
reflecting changes and perform at par when I have to search for those
changes in different threads?

2.] Is there any concept of localization in ES? Can I segregate my data
based on some logical partition and apply filters and search only on that
subsection? Not sure how shard routing works, but if I route search
requests to multiple shards using SearchRequestBuilder, how do I aggregate
the results from multiple shards? Probably this is not the solution I am
looking for. My use case is to not worry too much about how ES is
organizing my data, but I still want to set specific filters for data that
will exist on a corresponding shard. If this is confusing, please let me
know, and I will rephrase.

Thanks,
Sandeep

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f6dcfbb2-f065-43f7-9e0d-bfc5e3be43a3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #2