Hi I came into a problem with searching among multiple indexes.
Let's say we have 2 indexes both indexes have almost same structure but one index got more fields what are not exist in first index.
What I need to do is apply filters to first index and search in second index for expiration field greather than, if I will use must results will be just data what contain expiraiton field, if I use should the effect is not what is needed so in this case I will get all filtered data including data from second index but not filtered by expiration is > but all of them,
Do elasticsearch have some option to set filter for specific index let's say index.expiration > and this query will be applied just for second index ?
Thanks.
Hi @Kamil_Hurajt,
I have to admit that I did not understand your problem. Can you please come up with a concrete and completely reproducible example?
One thing you seem to look for is filtering fields that do not exist. If that's the case you can have a look at the docs for the exists query.
Daniel
HI @danielmitterdorfer , thanks for reply and sorry for not well explained problem.
I'll try to explain it a bit more clear.
index1 have this fields
- user
- settings (nested)
- likes (array of user ids)
- dislikes (array of user ids)
- later( array of user ids )
index2 have same fields excluded later and extra field expiraiton ( timestamp )
In MySQL problem will be solved by query:
SELECT index1.* from index1
LEFT JOIN
index2
ON index2.expiration > 2000
WHERE index1.like IS NOT IN (1,2,3) and index1.dislike IS NOT IN (1,2,3) and index1.later IS NOT IN (1,2,3)
So exactly result will be all filtered data from index1 based on where clause for index1 ( index1.like IS NOT IN (1,2,3) and index1.dislike IS NOT IN (1,2,3) and index1.later IS NOT IN (1,2,3) ) + data from index2 with expiration > 5000
That's what I need to reach in this situation with elastic search, split query for each index separately, to don't let query for index1 affect query for index2. If I'm searching in both indexes.
The description what exactly needed to be done is: In index1 we have users for search based on exact parameters. In index2 are stored just copies from index1 if user marked other user as "later", index2 have extra field "expiration" and one field less "later", what I need is to search in both indexes but get results from second index just in case when expiration is > then some number.
Hope that this explanation will give you better overview and I'm sorry for my bad explanation.
Hi @Kamil_Hurajt,
now I get you. No worries. 
The best you can probably do is to use the Multi Search API and issue one query for index1 and one for index2.
And to take a step back: I do not know the larger context of your application but based on your description I am not sure whether Elasticsearch is the right solution for your problem in this case (I don't know which other features you use).
Daniel
Hi @danielmitterdorfer , thanks to point me for right resource!
But in this case is right solution, in case of huge amount of data and huge amount of search for users in case of interests, settings, rating relation ships and activity and history. Also we're talking about huge amount of data in milions of rows, what need to be searched for bring right result.
I'm not sure, because i've just started learning big data, and I've start to work with ElasticSearch few years ago for other case as storage for analysed data crawled from interned and seearching throug them, ( social media care system ). But now I'm working on something where is lot of rules for searching and relational database is not best solution here i believe. Maybe i'm wrong
Hi @Kamil_Hurajt,
as I said I am not familiar with your use case and I don't want to shoehorn you into something just because you are asking here in this forum. I also did not imply that a relational database is the right solution either btw. 
If you have evaluated your options and Elasticsearch fits your use case best then great. I just wanted to bring this point up.
Daniel