Variable for MultiSearch

Eugene_Strokin · March 5, 2012, 8:26pm

First of all, thank you for adding MultiSearch to 0.19. This feature
already improved performance of some of my code by 2.5 times.
Sometimes I need to run 4 different queries strait, and now I'm just
making one call to do this, and it takes 4ms instead of 10ms.

But in other cases, which I believe could be common for other people
as well, I need to get a result from a first query, get some value,
and put it as a parameter to a following query (or filter actually).

So I was thinking, is it possible to somehow expose values of a
previous query to the next one?

I'll give a real use case example:
I have doc type: User. I'd need to find a/few such document(s) and
based on ID of the document retrieve some other documents.

I don't know the IDs upfront, since users selected based on some fancy
criteria. What I do right now, is just making a call to ES to get the
users, and generate a search for other type which contains the UserId
as a field. To improve this part as well using MutlySearch, I'd need
to pass those IDs to my other query somehow.

Do you have any suggestions/ideas?

Thank you.
Eugene S.

kimchy · March 5, 2012, 8:35pm

It feels like this is something that is best done on the "client" side in this case, there isn't a major benefit from somehow pushing it to have elasticsearch doing it, and having a "language" to inject values from the previous search execution to the next one, well, you might as well do that in your favorite lang of choice. In any case, this execution is still serial.

On Monday, March 5, 2012 at 10:26 PM, Eugene Strokin wrote:

First of all, thank you for adding MultiSearch to 0.19. This feature
already improved performance of some of my code by 2.5 times.
Sometimes I need to run 4 different queries strait, and now I'm just
making one call to do this, and it takes 4ms instead of 10ms.

But in other cases, which I believe could be common for other people
as well, I need to get a result from a first query, get some value,
and put it as a parameter to a following query (or filter actually).

So I was thinking, is it possible to somehow expose values of a
previous query to the next one?

I'll give a real use case example:
I have doc type: User. I'd need to find a/few such document(s) and
based on ID of the document retrieve some other documents.

I don't know the IDs upfront, since users selected based on some fancy
criteria. What I do right now, is just making a call to ES to get the
users, and generate a search for other type which contains the UserId
as a field. To improve this part as well using MutlySearch, I'd need
to pass those IDs to my other query somehow.

Do you have any suggestions/ideas?

Thank you.
Eugene S.

medcl_net · March 6, 2012, 2:17am

I think this feature can be called “query flow”,it can be very powerful,but also bring with risk...

From: Shay Banon
Sent: Tuesday, March 06, 2012 4:35 AM
To: elasticsearch@googlegroups.com
Subject: Re: Variable for MultiSearch

It feels like this is something that is best done on the "client" side in this case, there isn't a major benefit from somehow pushing it to have elasticsearch doing it, and having a "language" to inject values from the previous search execution to the next one, well, you might as well do that in your favorite lang of choice. In any case, this execution is still serial.
On Monday, March 5, 2012 at 10:26 PM, Eugene Strokin wrote:

First of all, thank you for adding MultiSearch to 0.19. This feature
already improved performance of some of my code by 2.5 times.
Sometimes I need to run 4 different queries strait, and now I'm just
making one call to do this, and it takes 4ms instead of 10ms.

But in other cases, which I believe could be common for other people
as well, I need to get a result from a first query, get some value,
and put it as a parameter to a following query (or filter actually).

So I was thinking, is it possible to somehow expose values of a
previous query to the next one?

I'll give a real use case example:
I have doc type: User. I'd need to find a/few such document(s) and
based on ID of the document retrieve some other documents.

I don't know the IDs upfront, since users selected based on some fancy
criteria. What I do right now, is just making a call to ES to get the
users, and generate a search for other type which contains the UserId
as a field. To improve this part as well using MutlySearch, I'd need
to pass those IDs to my other query somehow.

Do you have any suggestions/ideas?

Thank you.
Eugene S.

Eugene_Strokin · March 8, 2012, 10:18pm

Even though I'm in agreement that we shouldn't introduce some kind of
language for doing this, we still could take advantage of some
existing features.
may be it wouldn't be called multiSearch anymore, but very often we
have such case for example:
Some documents (type ABC) have IDs of another documents (type XYZ).
And we need to query a first type of the documents, but retrieve as a
result the second type.
I know we are NO SQL, and this looks like a join, but lets face it,
this is very common case (at least in my practice it is).
So we should be able to say: run my query on ABC documents, but give
me XYZ documents, and use ABC.someOtherDocId field as _id for XYZ.

So, there are such cases there we need to pass some data from result
of one query to another.
Right, it shouldn't be any kind of fancy thing, but such things like
IDs, or may be version number or score, etc...

On Mar 5, 9:17 pm, medcl2...@gmail.com wrote:

I think this feature can be called “query flow”,it can be very powerful,but also bring with risk...

From: Shay Banon
Sent: Tuesday, March 06, 2012 4:35 AM
To: elasticsearch@googlegroups.com
Subject: Re: Variable for MultiSearch

It feels like this is something that is best done on the "client" side in this case, there isn't a major benefit from somehow pushing it to have elasticsearch doing it, and having a "language" to inject values from the previous search execution to the next one, well, you might as well do that in your favorite lang of choice. In any case, this execution is still serial.

On Monday, March 5, 2012 at 10:26 PM, Eugene Strokin wrote:

First of all, thank you for adding MultiSearch to 0.19. This feature
already improved performance of some of my code by 2.5 times.
Sometimes I need to run 4 different queries strait, and now I'm just
making one call to do this, and it takes 4ms instead of 10ms.

But in other cases, which I believe could be common for other people
as well, I need to get a result from a first query, get some value,
and put it as a parameter to a following query (or filter actually).

So I was thinking, is it possible to somehow expose values of a
previous query to the next one?

I'll give a real use case example:
I have doc type: User. I'd need to find a/few such document(s) and
based on ID of the document retrieve some other documents.

I don't know the IDs upfront, since users selected based on some fancy
criteria. What I do right now, is just making a call to ES to get the
users, and generate a search for other type which contains the UserId
as a field. To improve this part as well using MutlySearch, I'd need
to pass those IDs to my other query somehow.

Do you have any suggestions/ideas?

Thank you.
Eugene S.