Can't implement pagination with scroll in ElasticSearch 6.6.1

I try to implement pagination with using scroll. I read this article but does not help me.
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html

My problem is when I try to make a few requests the results are same. Scrolling is not working.

We have more than 20 billions records in the database.

My ElasticSearch version is: 6.6.1

PHP code snippet:

public function someFunction($client, $elasticResult, $limit)
{
    $requestQuery = {
       "scroll":"1m",
       "size":10000,
       "index":"my_index",
       "type":"my_type",
       "body":{
          "query":{
             "bool":{
                "must":{
                   "terms":{
                      "db":[ 13, 15, 19
                      ]
                   }
                }
             }
          },
          "sort":[
             "_doc"
          ]
       }
    }

    $client = $client->build();
    $response = $client->search($requestQuery);

    $elasticResult->setScrollTime('1m');

    $metadata = [];
    yield from $elasticResult->getPagingResults($response, $client, $metadata, $limit);
}

    public function getPagingResults(array $response, Client $client, &$result, int $limit = null): \Generator
    {
        $i = 0;

        while (isset($response['hits']['hits']) && \count($response['hits']['hits']) > 0) {
            foreach ($response['hits']['hits'] as $hit) {
                ++$i;

                if (null !== $limit && $i > $limit) {
                    break;
                }

                if (!empty($hit['_source'])) {
                    foreach ($hit['_source'] as $key => $value) {
                        if (!isset($result[$hit['_source']['db']][$key])) {
                            $result[$hit['_source']['db']][$key] = 0;
                        }

                        if ('' !== trim($value)) {
                            ++$result[$hit['_source']['db']][$key];
                        }
                    }
                }

                yield $hit;
            }

            if (null !== $limit && $i > $limit) {
                break;
            }

            $scrollId = $response['_scroll_id'];

            $response = $client->scroll([
                'scroll_id' => $scrollId,
                'scroll'    => '1m',
            ]);
        }
    }

If is necessary additional info, Please don't hesitate to ask.

I don't use PHP myself so I can't comment on your particular example, but in general you need a scroll ID to be able to scroll through a search result.

There is a PHP example here showing the use of a scroller. Hope that helps you.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.