Missing result using scroll in java API

Elastic Stack Elasticsearch
1

Hi,

I use Java API to works with ES. When I start using scroll API like that:

SearchResponse response = client.prepareSearch()
                            .setIndices(indexName)
                            .setTypes(type)
                            .setSearchType(SearchType.DEFAULT)
                            .setScroll(TimeValue.timeValueMinutes(1));
                            .setSize(LogEntryStorage.SCROLL_PAGE_SIZE)
                            .setExplain(true)
                            .execute().actionGet()

response = srb.execute().actionGet();
response = client.prepareSearchScroll(response.getScrollId())
                            .setScroll(TimeValue.timeValueMinutes(1)).execute().actionGet();

The result of response.getHits().getHits() is empty but response.getHits().getTotalHits() has number which is correct. How to get this values ? I need to close or refresh scroll somehow?

Thanks

2

There is no query apparently, right?
So it's a match_all here.

I'd first remove .setExplain(true) as it does not bring any value and might slow down your process anyway.

If you run one first time this, and have totalHits > 0 then you should have something in the response.getHits().getHits() I think.

Are you sure you did not run it a second time but there are no more records available?

3

Hi David

I see the problem, without scroll the total number of hits is 139.
When I change setSize to 200 then in scroll getHits is 0
but when i change setSize to number lower than totalHits for example 20, then each
hit on scroll i have bunch of date with 20 results.

It looks like a bug when setSize is higher than total size.

Are you sure you did not run it a second time but there are no more records available?

Yes, I'm sure

4

I'm surprised. Which version is that?

5

6.2.2

6

Can you reproduce it with just some REST commands? From Kibana Console?

7

Unfortunately not now.
P.S Great presentation on CodeEurope in 2017 in Cracow :wink:

1 Like
8

Haha! Thank you! I'll be back at CodeEurope in Poznan in June 12th. :stuck_out_tongue:

9

Do you have any possibility to check/test this strange behavior?
Or how to check it on Kibana? How to create scroll?

10

I tried this on 6.2.3:

DELETE test
POST test/doc/_bulk
{ "index": {} }
{ "foo": "bar" }
{ "index": {} }
{ "foo": "bar" }
{ "index": {} }
{ "foo": "bar" }
{ "index": {} }
{ "foo": "bar" }
{ "index": {} }
{ "foo": "bar" }
{ "index": {} }
{ "foo": "bar" }
{ "index": {} }
{ "foo": "bar" }
{ "index": {} }
{ "foo": "bar" }
{ "index": {} }
{ "foo": "bar" }
{ "index": {} }
{ "foo": "bar" }
{ "index": {} }
{ "foo": "bar" }
{ "index": {} }
{ "foo": "bar" }
{ "index": {} }
{ "foo": "bar" }
{ "index": {} }
{ "foo": "bar" }
{ "index": {} }
{ "foo": "bar" }
{ "index": {} }
{ "foo": "bar" }
{ "index": {} }
{ "foo": "bar" }
POST test/_search?scroll=1m
{
    "size": 100
}

It works well:

{
  "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAAFXFnJBbWhNdEtXU2MyTktwVFVjWGhiNVEAAAAAAAABVhZyQW1oTXRLV1NjMk5LcFRVY1hoYjVRAAAAAAAAAVgWckFtaE10S1dTYzJOS3BUVWNYaGI1UQAAAAAAAAFZFnJBbWhNdEtXU2MyTktwVFVjWGhiNVEAAAAAAAABWhZyQW1oTXRLV1NjMk5LcFRVY1hoYjVR",
  "took": 5,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 17,
    "max_score": 1,
    "hits": [
      {
        "_index": "test",
        "_type": "doc",
        "_id": "iO3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "ku3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "iu3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "je3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "kO3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "k-3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "ie3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "i-3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "ju3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "lO3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "lu3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "l-3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "mO3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "jO3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "j-3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "ke3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "le3GmmIBFZyT-UHdSJyi",
        "_score": 1,
        "_source": {
          "foo": "bar"
        }
      }
    ]
  }
}
11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.