No hit using scan/scroll with has_parent filter


(Jean-Baptiste Lièvremont) #1

Hi folks,

I use a parent/child mapping configuration which works flawlessly with
"classic" search requests, e.g using has_parent to find child documents
with criteria on the parent documents.

I am trying to get all child document IDs that match a given set of
criteria using scan and scroll, which also works well - until I introduce
the has_parent filter, in which case the scroll request returns no hit
(although total_hits is correct).

Is it a known issue?

I can provide sample mapping files and queries with associated/expected
results. Please note that this behavior has been noticed on 0.90.6 but is
still present in 0.90.9.

Thanks, best regards,
-- Jean-Baptiste Lièvremont

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fd7c563e-34f7-4aa8-ab1a-460840ba2af0%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Martijn Van Groningen) #2

Hi Jean,

Can you share how you execute the scan request with the has_parent filter?
(via a gist or something like that)

Martijn

On 8 January 2014 15:17, Jean-Baptiste Lièvremont <
jean-baptiste.lievremont@sonarsource.com> wrote:

Hi folks,

I use a parent/child mapping configuration which works flawlessly with
"classic" search requests, e.g using has_parent to find child documents
with criteria on the parent documents.

I am trying to get all child document IDs that match a given set of
criteria using scan and scroll, which also works well - until I introduce
the has_parent filter, in which case the scroll request returns no hit
(although total_hits is correct).

Is it a known issue?

I can provide sample mapping files and queries with associated/expected
results. Please note that this behavior has been noticed on 0.90.6 but is
still present in 0.90.9.

Thanks, best regards,
-- Jean-Baptiste Lièvremont

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/fd7c563e-34f7-4aa8-ab1a-460840ba2af0%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2BA76TzaCJqG26SMNid2PawALjNSZgr%2ByS3NJEFfF3G%3DySjqkw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jean-Baptiste Lièvremont) #3

Hi Martijn,

Thanks for your answer. You can find in the gist below some HTTP
conversations made on my ES 0.90.6 node, as well as a link to the Java code
responsible for the calls:

Please note that the issue appears only when combining scan/scroll with
has_parent filter, as it seems to work using a has_parent query instead.

Best regards,
-- Jean-Baptiste Lièvremont

Le jeudi 9 janvier 2014 00:18:14 UTC+1, Martijn v Groningen a écrit :

Hi Jean,

Can you share how you execute the scan request with the has_parent filter?
(via a gist or something like that)

Martijn

On 8 January 2014 15:17, Jean-Baptiste Lièvremont <
jean-baptist...@sonarsource.com <javascript:>> wrote:

Hi folks,

I use a parent/child mapping configuration which works flawlessly with
"classic" search requests, e.g using has_parent to find child documents
with criteria on the parent documents.

I am trying to get all child document IDs that match a given set of
criteria using scan and scroll, which also works well - until I introduce
the has_parent filter, in which case the scroll request returns no hit
(although total_hits is correct).

Is it a known issue?

I can provide sample mapping files and queries with associated/expected
results. Please note that this behavior has been noticed on 0.90.6 but is
still present in 0.90.9.

Thanks, best regards,
-- Jean-Baptiste Lièvremont

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/fd7c563e-34f7-4aa8-ab1a-460840ba2af0%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8bafbc47-a68f-41fa-8730-d17cf1832011%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jean-Baptiste Lièvremont) #4

Going further on the subject, I have tried to work around the issue using a
filtered query, to no avail.

So basically, if I want to fetch all child document IDs that match a given
criterion on their parent, I have either to

  • use a query - which in my case requires that I duplicate the existing
    code that builds a filter, building a query instead
  • or do it in 2 requests, one to count the totalHits matching the criteria,
    the second using totalHits in the "size" parameter to fetch all IDs at once

Is there any other solution?

Le jeudi 9 janvier 2014 10:18:25 UTC+1, Jean-Baptiste Lièvremont a écrit :

Hi Martijn,

Thanks for your answer. You can find in the gist below some HTTP
conversations made on my ES 0.90.6 node, as well as a link to the Java code
responsible for the calls:
https://gist.github.com/jblievremont/8331460

Please note that the issue appears only when combining scan/scroll with
has_parent filter, as it seems to work using a has_parent query instead.

Best regards,
-- Jean-Baptiste Lièvremont

Le jeudi 9 janvier 2014 00:18:14 UTC+1, Martijn v Groningen a écrit :

Hi Jean,

Can you share how you execute the scan request with the has_parent
filter? (via a gist or something like that)

Martijn

On 8 January 2014 15:17, Jean-Baptiste Lièvremont <
jean-baptist...@sonarsource.com> wrote:

Hi folks,

I use a parent/child mapping configuration which works flawlessly with
"classic" search requests, e.g using has_parent to find child documents
with criteria on the parent documents.

I am trying to get all child document IDs that match a given set of
criteria using scan and scroll, which also works well - until I introduce
the has_parent filter, in which case the scroll request returns no hit
(although total_hits is correct).

Is it a known issue?

I can provide sample mapping files and queries with associated/expected
results. Please note that this behavior has been noticed on 0.90.6 but is
still present in 0.90.9.

Thanks, best regards,
-- Jean-Baptiste Lièvremont

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/fd7c563e-34f7-4aa8-ab1a-460840ba2af0%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/374c3350-ca10-4699-9aae-b72087aba07a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Search scroll returning no hits (possible bug in 6.2?)
(Martijn Van Groningen) #5

I'm a bit confused the first query with has_parent doesn't return hits, but
the second does according to your gist or I'm I misreading it.

I just did a little test here and has_parent and search with search_type
scan seems to work correctly.

On 10 January 2014 17:53, Jean-Baptiste Lièvremont <
jean-baptiste.lievremont@sonarsource.com> wrote:

Going further on the subject, I have tried to work around the issue using
a filtered query, to no avail.

So basically, if I want to fetch all child document IDs that match a given
criterion on their parent, I have either to

  • use a query - which in my case requires that I duplicate the existing
    code that builds a filter, building a query instead
  • or do it in 2 requests, one to count the totalHits matching the
    criteria, the second using totalHits in the "size" parameter to fetch all
    IDs at once

Is there any other solution?

Le jeudi 9 janvier 2014 10:18:25 UTC+1, Jean-Baptiste Lièvremont a écrit :

Hi Martijn,

Thanks for your answer. You can find in the gist below some HTTP
conversations made on my ES 0.90.6 node, as well as a link to the Java code
responsible for the calls:
https://gist.github.com/jblievremont/8331460

Please note that the issue appears only when combining scan/scroll with
has_parent filter, as it seems to work using a has_parent query instead.

Best regards,
-- Jean-Baptiste Lièvremont

Le jeudi 9 janvier 2014 00:18:14 UTC+1, Martijn v Groningen a écrit :

Hi Jean,

Can you share how you execute the scan request with the has_parent
filter? (via a gist or something like that)

Martijn

On 8 January 2014 15:17, Jean-Baptiste Lièvremont <
jean-baptist...@sonarsource.com> wrote:

Hi folks,

I use a parent/child mapping configuration which works flawlessly with
"classic" search requests, e.g using has_parent to find child documents
with criteria on the parent documents.

I am trying to get all child document IDs that match a given set of
criteria using scan and scroll, which also works well - until I introduce
the has_parent filter, in which case the scroll request returns no hit
(although total_hits is correct).

Is it a known issue?

I can provide sample mapping files and queries with associated/expected
results. Please note that this behavior has been noticed on 0.90.6 but is
still present in 0.90.9.

Thanks, best regards,
-- Jean-Baptiste Lièvremont

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/fd7c563e-34f7-4aa8-ab1a-460840ba2af0%
40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2BA76TxNLAq6XMtC7oco%3D1bTOYYaRbedbfrJBrd7G7qCwcC_Bg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Martijn Van Groningen) #6

So this is related to:

On 14 January 2014 23:23, Martijn v Groningen <martijn.v.groningen@gmail.com

wrote:

I'm a bit confused the first query with has_parent doesn't return hits,
but the second does according to your gist or I'm I misreading it.

I just did a little test here and has_parent and search with search_type
scan seems to work correctly.

On 10 January 2014 17:53, Jean-Baptiste Lièvremont <
jean-baptiste.lievremont@sonarsource.com> wrote:

Going further on the subject, I have tried to work around the issue using
a filtered query, to no avail.

So basically, if I want to fetch all child document IDs that match a
given criterion on their parent, I have either to

  • use a query - which in my case requires that I duplicate the existing
    code that builds a filter, building a query instead
  • or do it in 2 requests, one to count the totalHits matching the
    criteria, the second using totalHits in the "size" parameter to fetch all
    IDs at once

Is there any other solution?

Le jeudi 9 janvier 2014 10:18:25 UTC+1, Jean-Baptiste Lièvremont a écrit :

Hi Martijn,

Thanks for your answer. You can find in the gist below some HTTP
conversations made on my ES 0.90.6 node, as well as a link to the Java code
responsible for the calls:
https://gist.github.com/jblievremont/8331460

Please note that the issue appears only when combining scan/scroll with
has_parent filter, as it seems to work using a has_parent query instead.

Best regards,
-- Jean-Baptiste Lièvremont

Le jeudi 9 janvier 2014 00:18:14 UTC+1, Martijn v Groningen a écrit :

Hi Jean,

Can you share how you execute the scan request with the has_parent
filter? (via a gist or something like that)

Martijn

On 8 January 2014 15:17, Jean-Baptiste Lièvremont <
jean-baptist...@sonarsource.com> wrote:

Hi folks,

I use a parent/child mapping configuration which works flawlessly with
"classic" search requests, e.g using has_parent to find child documents
with criteria on the parent documents.

I am trying to get all child document IDs that match a given set of
criteria using scan and scroll, which also works well - until I introduce
the has_parent filter, in which case the scroll request returns no hit
(although total_hits is correct).

Is it a known issue?

I can provide sample mapping files and queries with
associated/expected results. Please note that this behavior has been
noticed on 0.90.6 but is still present in 0.90.9.

Thanks, best regards,
-- Jean-Baptiste Lièvremont

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/fd7c563e-34f7-4aa8-ab1a-460840ba2af0%
40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
Met vriendelijke groet,

Martijn van Groningen

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2BA76TwQ64N1TyPZBxZE8ugWKa6b%2B5BPuJ8FqeK86zHsHtavFQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jean-Baptiste Lièvremont) #7

Yep, this seems definitely related to the linked issue. I am looking
forward to testing your fix, hopefully in 0.90.11 :slight_smile:

Regards,
-- JB.L

2014/1/14 Martijn v Groningen martijn.v.groningen@gmail.com

So this is related to:
https://github.com/elasticsearch/elasticsearch/issues/4703

On 14 January 2014 23:23, Martijn v Groningen <
martijn.v.groningen@gmail.com> wrote:

I'm a bit confused the first query with has_parent doesn't return hits,
but the second does according to your gist or I'm I misreading it.

I just did a little test here and has_parent and search with search_type
scan seems to work correctly.

On 10 January 2014 17:53, Jean-Baptiste Lièvremont <
jean-baptiste.lievremont@sonarsource.com> wrote:

Going further on the subject, I have tried to work around the issue
using a filtered query, to no avail.

So basically, if I want to fetch all child document IDs that match a
given criterion on their parent, I have either to

  • use a query - which in my case requires that I duplicate the existing
    code that builds a filter, building a query instead
  • or do it in 2 requests, one to count the totalHits matching the
    criteria, the second using totalHits in the "size" parameter to fetch all
    IDs at once

Is there any other solution?

Le jeudi 9 janvier 2014 10:18:25 UTC+1, Jean-Baptiste Lièvremont a
écrit :

Hi Martijn,

Thanks for your answer. You can find in the gist below some HTTP
conversations made on my ES 0.90.6 node, as well as a link to the Java code
responsible for the calls:
https://gist.github.com/jblievremont/8331460

Please note that the issue appears only when combining scan/scroll with
has_parent filter, as it seems to work using a has_parent query instead.

Best regards,
-- Jean-Baptiste Lièvremont

Le jeudi 9 janvier 2014 00:18:14 UTC+1, Martijn v Groningen a écrit :

Hi Jean,

Can you share how you execute the scan request with the has_parent
filter? (via a gist or something like that)

Martijn

On 8 January 2014 15:17, Jean-Baptiste Lièvremont <
jean-baptist...@sonarsource.com> wrote:

Hi folks,

I use a parent/child mapping configuration which works flawlessly
with "classic" search requests, e.g using has_parent to find child
documents with criteria on the parent documents.

I am trying to get all child document IDs that match a given set of
criteria using scan and scroll, which also works well - until I introduce
the has_parent filter, in which case the scroll request returns no hit
(although total_hits is correct).

Is it a known issue?

I can provide sample mapping files and queries with
associated/expected results. Please note that this behavior has been
noticed on 0.90.6 but is still present in 0.90.9.

Thanks, best regards,
-- Jean-Baptiste Lièvremont

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/fd7c563e-34f7-4aa8-ab1a-460840ba2af0%
40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
Met vriendelijke groet,

Martijn van Groningen

--
Met vriendelijke groet,

Martijn van Groningen

--
*Jean-Baptiste LIEVREMONT | *
*SonarSource *Core Developer
http://sonarsource.com

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANYhQu1Envd-ijshBn6KRKXBt7gPft%2BbxrzA-XcYOJMksnRDmQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Martijn Van Groningen) #8

Thanks :slight_smile: The work around for now is I guess to use the has_parent query
instead of the has_parent filter.

On 15 January 2014 09:17, Jean-Baptiste Lièvremont <
jean-baptiste.lievremont@sonarsource.com> wrote:

Yep, this seems definitely related to the linked issue. I am looking
forward to testing your fix, hopefully in 0.90.11 :slight_smile:

Regards,
-- JB.L

2014/1/14 Martijn v Groningen martijn.v.groningen@gmail.com

So this is related to:
https://github.com/elasticsearch/elasticsearch/issues/4703

On 14 January 2014 23:23, Martijn v Groningen <
martijn.v.groningen@gmail.com> wrote:

I'm a bit confused the first query with has_parent doesn't return hits,
but the second does according to your gist or I'm I misreading it.

I just did a little test here and has_parent and search with search_type
scan seems to work correctly.

On 10 January 2014 17:53, Jean-Baptiste Lièvremont <
jean-baptiste.lievremont@sonarsource.com> wrote:

Going further on the subject, I have tried to work around the issue
using a filtered query, to no avail.

So basically, if I want to fetch all child document IDs that match a
given criterion on their parent, I have either to

  • use a query - which in my case requires that I duplicate the existing
    code that builds a filter, building a query instead
  • or do it in 2 requests, one to count the totalHits matching the
    criteria, the second using totalHits in the "size" parameter to fetch all
    IDs at once

Is there any other solution?

Le jeudi 9 janvier 2014 10:18:25 UTC+1, Jean-Baptiste Lièvremont a
écrit :

Hi Martijn,

Thanks for your answer. You can find in the gist below some HTTP
conversations made on my ES 0.90.6 node, as well as a link to the Java code
responsible for the calls:
https://gist.github.com/jblievremont/8331460

Please note that the issue appears only when combining scan/scroll
with has_parent filter, as it seems to work using a has_parent query
instead.

Best regards,
-- Jean-Baptiste Lièvremont

Le jeudi 9 janvier 2014 00:18:14 UTC+1, Martijn v Groningen a écrit :

Hi Jean,

Can you share how you execute the scan request with the has_parent
filter? (via a gist or something like that)

Martijn

On 8 January 2014 15:17, Jean-Baptiste Lièvremont <
jean-baptist...@sonarsource.com> wrote:

Hi folks,

I use a parent/child mapping configuration which works flawlessly
with "classic" search requests, e.g using has_parent to find child
documents with criteria on the parent documents.

I am trying to get all child document IDs that match a given set of
criteria using scan and scroll, which also works well - until I introduce
the has_parent filter, in which case the scroll request returns no hit
(although total_hits is correct).

Is it a known issue?

I can provide sample mapping files and queries with
associated/expected results. Please note that this behavior has been
noticed on 0.90.6 but is still present in 0.90.9.

Thanks, best regards,
-- Jean-Baptiste Lièvremont

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/fd7c563e-
34f7-4aa8-ab1a-460840ba2af0%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
Met vriendelijke groet,

Martijn van Groningen

--
Met vriendelijke groet,

Martijn van Groningen

--
*Jean-Baptiste LIEVREMONT | *
*SonarSource *Core Developer
http://sonarsource.com

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2BA76TwyZ-GdXRTygNPx6nrm%3D0ARh2Xj%2B08FskKabCc_GwUjHA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #9