When using scroll with a has_child filter, the initial request returns the corre…ct total number of hits, but subsequent requests return no hits.
It looks like this problem was introduced in 0.90.6, and still occurs in 0.90.10. 0.90.5 works as expected.
The number of documents seems to play a part - in my initial test cases with only 2 parent documents, I couldn't reproduce the issue. However, creating 100 parents does reliably reproduce it. In my testing, 8 parent documents worked fine, but 9 did not.
It sounds very similar to the issue mentioned here: http://elasticsearch-users.115913.n3.nabble.com/No-hit-using-scan-scroll-with-has-parent-filter-td4047236.html
Here's a test script (requires jq(1) to grab the scroll ID from the first JSON result):
``` bash
#!/bin/sh
HOST='localhost:9200'
INDEX='test_scroll_jj'
CURL="curl -q --ipv4 --silent --show-error --fail"
$CURL -XDELETE "$HOST/${INDEX}?pretty=true" >/dev/null
$CURL -XPOST "$HOST/${INDEX}/?pretty=true" -d '
{
"mappings": {
"homes":{
"_parent":{
"type" : "person"
}
}
}
}' >/dev/null
for x in {1..100}; do # in my testing, 8 docs works, 9 fails
$CURL -XPUT "$HOST/${INDEX}/person/$x/?pretty=true" -d '{}' >/dev/null
$CURL -XPOST "$HOST/${INDEX}/homes?parent=$x&pretty=true" -d '{}' >/dev/null
done
$CURL -XPOST "$HOST/${INDEX}/_refresh?pretty=true" >/dev/null
echo "REQUEST ONE:"
SCROLL_RESULT=$($CURL -v -XPOST "http://$HOST/${INDEX}/person/_search?pretty=true&scroll=30s" -d'
{
"size" : 1,
"fields" : ["_id"],
"query" : {
"filtered" : {
"filter" : {
"has_child" : {
"type" : "homes",
"query" : {
"match_all" : {}
}
}
}
}
}
}')
echo $SCROLL_RESULT
scroll_id=$(echo $SCROLL_RESULT | jq -r '.["_scroll_id"]')
echo
echo "REQUEST TWO:"
$CURL -v "http://$HOST/_search/scroll?scroll=30s&scroll_id=$scroll_id&pretty=true"
```
The failing output on 0.90.10:
```
/tmp|⇒ /tmp/scrollbug.sh
REQUEST ONE:
* About to connect() to localhost port 9200 (#0)
* Trying 127.0.0.1...
* Adding handle: conn: 0x7fb832006e00
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7fb832006e00) send_pipe: 1, recv_pipe: 0
* Connected to localhost (127.0.0.1) port 9200 (#0)
> POST /test_scroll_jj/person/_search?pretty=true&scroll=30s HTTP/1.1
> User-Agent: curl/7.32.0
> Host: localhost:9200
> Accept: */*
> Content-Length: 321
> Content-Type: application/x-www-form-urlencoded
>
} [data not shown]
* upload completely sent off: 321 out of 321 bytes
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=UTF-8
< Content-Length: 520
<
{ [data not shown]
* Connection #0 to host localhost left intact
{ "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTs2OkM5SXlBenNyU0lXR21uX3JsN25XcHc7NzpDOUl5QXpzclNJV0dtbl9ybDduV3B3Ozg6QzlJeUF6c3JTSVdHbW5fcmw3bldwdzs5OkM5SXlBenNyU0lXR21uX3JsN25XcHc7MTA6QzlJeUF6c3JTSVdHbW5fcmw3bldwdzswOw==", "took" : 5, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 100, "max_score" : 1.0, "hits" : [ { "_index" : "test_scroll_jj", "_type" : "person", "_id" : "2", "_score" : 1.0 } ] } }
REQUEST TWO:
* About to connect() to localhost port 9200 (#0)
* Trying 127.0.0.1...
* Adding handle: conn: 0x7f8589806e00
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7f8589806e00) send_pipe: 1, recv_pipe: 0
* Connected to localhost (127.0.0.1) port 9200 (#0)
> GET /_search/scroll?scroll=30s&scroll_id=cXVlcnlUaGVuRmV0Y2g7NTs2OkM5SXlBenNyU0lXR21uX3JsN25XcHc7NzpDOUl5QXpzclNJV0dtbl9ybDduV3B3Ozg6QzlJeUF6c3JTSVdHbW5fcmw3bldwdzs5OkM5SXlBenNyU0lXR21uX3JsN25XcHc7MTA6QzlJeUF6c3JTSVdHbW5fcmw3bldwdzswOw==&pretty=true HTTP/1.1
> User-Agent: curl/7.32.0
> Host: localhost:9200
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=UTF-8
< Content-Length: 410
<
{
"_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTs2OkM5SXlBenNyU0lXR21uX3JsN25XcHc7NzpDOUl5QXpzclNJV0dtbl9ybDduV3B3Ozg6QzlJeUF6c3JTSVdHbW5fcmw3bldwdzs5OkM5SXlBenNyU0lXR21uX3JsN25XcHc7MTA6QzlJeUF6c3JTSVdHbW5fcmw3bldwdzswOw==",
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" : [ ]
}
}
```
And the expected output as per 0.90.5:
```
/tmp|⇒ /tmp/scrollbug.sh
REQUEST ONE:
* About to connect() to localhost port 9200 (#0)
* Trying 127.0.0.1...
* Adding handle: conn: 0x7fd04a006e00
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7fd04a006e00) send_pipe: 1, recv_pipe: 0
* Connected to localhost (127.0.0.1) port 9200 (#0)
> POST /test_scroll_jj/person/_search?pretty=true&scroll=30s HTTP/1.1
> User-Agent: curl/7.32.0
> Host: localhost:9200
> Accept: */*
> Content-Length: 321
> Content-Type: application/x-www-form-urlencoded
>
} [data not shown]
* upload completely sent off: 321 out of 321 bytes
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=UTF-8
< Content-Length: 523
<
{ [data not shown]
* Connection #0 to host localhost left intact
{ "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTsyMTpILV9IUWU2MlRrUzQyd2JRYzZLS3dROzIzOkgtX0hRZTYyVGtTNDJ3YlFjNktLd1E7MjI6SC1fSFFlNjJUa1M0MndiUWM2S0t3UTsyNDpILV9IUWU2MlRrUzQyd2JRYzZLS3dROzI1OkgtX0hRZTYyVGtTNDJ3YlFjNktLd1E7MDs=", "took" : 9, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 100, "max_score" : 1.0, "hits" : [ { "_index" : "test_scroll_jj", "_type" : "person", "_id" : "2", "_score" : 1.0 } ] } }
REQUEST TWO:
* About to connect() to localhost port 9200 (#0)
* Trying 127.0.0.1...
* Adding handle: conn: 0x7f8a92006e00
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7f8a92006e00) send_pipe: 1, recv_pipe: 0
* Connected to localhost (127.0.0.1) port 9200 (#0)
> GET /_search/scroll?scroll=30s&scroll_id=cXVlcnlUaGVuRmV0Y2g7NTsyMTpILV9IUWU2MlRrUzQyd2JRYzZLS3dROzIzOkgtX0hRZTYyVGtTNDJ3YlFjNktLd1E7MjI6SC1fSFFlNjJUa1M0MndiUWM2S0t3UTsyNDpILV9IUWU2MlRrUzQyd2JRYzZLS3dROzI1OkgtX0hRZTYyVGtTNDJ3YlFjNktLd1E7MDs=&pretty=true HTTP/1.1
> User-Agent: curl/7.32.0
> Host: localhost:9200
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=UTF-8
< Content-Length: 523
<
{
"_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTsyMTpILV9IUWU2MlRrUzQyd2JRYzZLS3dROzIzOkgtX0hRZTYyVGtTNDJ3YlFjNktLd1E7MjI6SC1fSFFlNjJUa1M0MndiUWM2S0t3UTsyNDpILV9IUWU2MlRrUzQyd2JRYzZLS3dROzI1OkgtX0hRZTYyVGtTNDJ3YlFjNktLd1E7MDs=",
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 100,
"max_score" : 1.0,
"hits" : [ {
"_index" : "test_scroll_jj",
"_type" : "person",
"_id" : "7",
"_score" : 1.0
} ]
}
}
```