# Sliced search not returning all hits

**URL:** <https://discuss.elastic.co/t/sliced-search-not-returning-all-hits/324513>\
**Category:** Elasticsearch\
**Created:** [February 2, 2023, 8:30am UTC](https://discuss.elastic.co/t/sliced-search-not-returning-all-hits/324513 "2023-02-02T08:30:38Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![jkruger](https://avatars.discourse-cdn.com/v4/letter/j/ba9def/32.png) [@jkruger](https://discuss.elastic.co/u/jkruger)\
**Post date:** [February 2, 2023, 8:30am UTC](https://discuss.elastic.co/t/sliced-search-not-returning-all-hits/324513/1 "2023-02-02T08:30:38Z")

</div>

Hello,

I am a new Elastic user, and I already ran into an issue. I am trying to extract all logs from a certain index. In order to deal with large indices I want to implement pagination using [search\_after](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/paginate-search-results.html#search-after) and [PiTs](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/paginate-search-results.html#search-after) (Points-in-Time).

Python lib version: 7.13.3  
ELK version: 7.17.1

My code looks like this:

```auto
  pit = client.open_point_in_time(index=log_index, keep_alive='1m')
  pit_id = pit['id']

  log_file_items = list()

  lines_skipped = 0

  last_hit_sort = None 

  for slice_id in range(2):
      query_body = {
          'size': 150,
          "track_total_hits": False,  
          'pit': {
              'id': pit_id,
              'keep_alive': '10s'
          },
          'slice': {
              "id": slice_id,
              "max": 2
          },
          "query": {
              "match": {
                  "log.message": "[specific-token]"
              },
          },
          "sort": [
              {"@timestamp": "asc"},
              {"_shard_doc": "desc"}
          ],
      }

      if last_hit_sort is not None:
          query_body['search_after'] = last_hit_sort

      result = client.search(body=query_body)

      print('total hits', slice_id, len(result['hits']['hits']))

      for row in result['hits']['hits']:
          [parsing data...]

      if len(result['hits']['hits']) > 0:
          last_hit_sort = result['hits']['hits'][-1]['sort']
      else:
          break

  # --- Cleanup pit
  pit_close = client.close_point_in_time(body={
      'id': pit_id
  })

```

Because I don't have a sample index yet with more than `10,000` hits I wanted to test it by limiting the query size to `150` hits. It should return `261` hits in total, but the first slice only gives me `139` hits. The second slice returns `0` hits.

The curious thing is that when I change the sorting of the timestamp from ascending to descending then it returns more results, but still not the full amount.

> "sort": [  
> {"@timestamp": "asc"},  
> {"\_shard\_doc": "desc"}  
> ],

changed to

> "sort": [  
> {"@timestamp": "desc"},  
> {"\_shard\_doc": "desc"}  
> ],

Returns `139` in the first slice, and `114` in the second slice, and in total `253` which means that `8` hits are still missing. I am not sure what I am doing wrong, but any help would be appreciated. I went over the docs multiple times and searched for this issue, but I simply cannot find what I am doing wrong.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 2, 2023, 8:31am UTC](https://discuss.elastic.co/t/sliced-search-not-returning-all-hits/324513/2 "2023-03-02T08:31:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.