Localized data with Shard Knowledge

(ElasticSearch Users mailing list) #1


I have a query about using ElasticSearch.

I have part of my data (about 90%) to be indexed in ElasticSearch and the
remaining 10% to be stored elsewhere (like a DB/NoSQL DB etc.) since that
10% data will be very frequently changing data.

I have the following queries:

1.] Should I be keeping the remaining 10% of my data on non-ES/Lucene
persistent store? Is it okay to store/index those fields in ES/Lucene even
if they are frequently updating? With NRT readers, will it be efficient at
reflecting changes and perform at par when I have to search for those
changes in different threads?

2.] Is there any concept of localization in ES? Can I segregate my data
based on some logical partition and apply filters and search only on that
subsection? Not sure how shard routing works, but if I route search
requests to multiple shards using SearchRequestBuilder, how do I aggregate
the results from multiple shards? Probably this is not the solution I am
looking for. My use case is to not worry too much about how ES is
organizing my data, but I still want to set specific filters for data that
will exist on a corresponding shard. If this is confusing, please let me
know, and I will rephrase.


You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f6dcfbb2-f065-43f7-9e0d-bfc5e3be43a3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

(system) #2