Bug in elasticsearch search_after rest api?

[corbosman]

Hi all, im running into what I'm thinking might be a bug. I'm using the REST search api to return log entries stored in our elasticsearch cluster. This could run into the tens of thousands, so I'm using search_after to return subsets at a time. This is mostly just working fine, but I ran into a small issue in the response. The last page returns this for hits.total even though the actual number of documents in that specific "paginated" response is well below 10.000.

{
  'value' => 10000,
   'relation' => 'gte'
};

Is this a bug? I was expecting the see the actual number of returned documents there and use it to stop the loop. This is latest ES7.8

[dadoonet (David Pilato)]

This number shows the "total" number of hits whatever the "page" you're looking at.

So this is expected.

[corbosman]

So there is no way to see how many results you're getting back in your current query?

[dadoonet (David Pilato)]

You mean? The number of hits on the current page or the exact total number of hits?

[corbosman]

The number of hits on the current page, especially the last page. So lets say I have 45000 hits in total, page 1 to 4 have the maximum of 10000 hits, so I expect the last page to have 5000 hits. Is there a way to see that 5000 number anywhere as a response of that specific 5th query?

[dadoonet (David Pilato)]

No but you can count how many hits you have in the array in the response I guess?