ElasticSearch.NET - Overflow exception when using scrollAPI or search_after

[salunkerahul (Rahul)]

I'm attempting read all data from a particular index using the Scroll API(ElasticSearch.NET-7.4.2), but I end up with the following exception

    # FailureReason: Unrecoverable/Unexpected BadResponse while attempting POST on http://localhost:9200/_search/scroll?pretty=true&error_trace=true
     - [1] BadResponse: Node: http://localhost:9200/ Exception: OverflowException Took: 00:00:00.0490000

The code is as follows:

    [ElasticsearchType]
    public class SPEXEC
    {
        [Text(Name="event_type")]
        //[JsonProperty("event_type")]
        public  string EventType{ get; set; }

        [Text(Name="client_app_name")]
        //[JsonProperty("client_app_name")]
        public  string ClientAppName{ get; set; }
    }
    var settings = new ConnectionSettings(new Uri("http://localhost:9200"))
        .DefaultIndex("filebeat-7.5.2")
        .EnableDebugMode()
        .PrettyJson()
        .RequestTimeout(TimeSpan.FromMinutes(2));
    var searchResponse = client.Search<SPExec>(s => s.Index("filebeat-7.5.2-2020.03.19-000002").MatchAll().Scroll("3m").Size(1000));
    while (searchResponse.Documents.Any())
    {
        var resp = new ISearchResponse<SPExec>
        {
            Results = searchResponse.Documents,
            ElapsedMilliseconds = (int)searchResponse.Took,
        };
        searchResponse = client.Scroll<SPExec>("1m", searchResponse.ScrollId); // Overflow exception
    }
    client.ClearScroll(c => c.ScrollId(searchResponse.ScrollId));

The aim of using Scroll API is to read data from Elastic to generate reports. I don't encounter any errors when using scrolling within the Dev Console.

    POST filebeat-7.5.2-2020.03.19-000002/_search?scroll=1m
    {
      "query": {
        "match_all": {}
      }
      , "size": 1000
    }

    POST /_search/scroll 
    {
        "scroll" : "1m", 
        "scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAX3sWOGZwN1NKUVVUNlc1dEFxbDRSZmxtQQ=="
    }

    POST /_search/scroll 
    {
        "scroll" : "1m", 
        "scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAX3sWOGZwN1NKUVVUNlc1dEFxbDRSZmxtQQ=="
    }

If there something obvious that I've missed out when using Scroll API, do point me in the right direction. I've also run the application using ElasticSearch-7.6.1 and I end up with the same exception.

I also attempted to use the search_after, which as the documentation suggests is for real-time user requests which also failed with an Overflow exception.

var searchResponse = client.Search<SPEXEC>(s => s.Index(index)
        .MatchAll()
        .Size(1000)
        .Sort(ss => ss
        .Ascending(p => p.Timestamp)
        .Descending(p => p.ElasticID)));
while (true)
{
    if (searchResponse.Documents.Any())
    {
       hitTotal += searchResponse.Hits.Count;
       var searchAfter = searchResponse.Hits.Last().Sorts.ToArray();
       searchResponse = client.Search<SPEXEC>(s => s.Index(index)
          .MatchAll()
         .Size(1000)
         .Sort(ss => ss
                .Ascending(p => p.Timestamp)
                .Descending(p => p.ElasticID))
         .SearchAfter(searchAfter));
  }
  else
 {
    System.Threading.Thread.Sleep(5 * 60 * 1000);
 }
   }