# ScrollAll varying record counts and speed

**URL:** <https://discuss.elastic.co/t/scrollall-varying-record-counts-and-speed/141629>\
**Category:** Elasticsearch\
**Created:** [July 25, 2018, 4:55pm UTC](https://discuss.elastic.co/t/scrollall-varying-record-counts-and-speed/141629 "2018-07-25T16:55:26Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Robert\_McGirr](https://avatars.discourse-cdn.com/v4/letter/r/6f9a4e/32.png) [@Robert\_McGirr](https://discuss.elastic.co/u/Robert_McGirr)\
**Post date:** [July 25, 2018, 4:55pm UTC](https://discuss.elastic.co/t/scrollall-varying-record-counts-and-speed/141629/1 "2018-07-25T16:55:27Z")

</div>

I have a daily need to retrieve all documents from a particular index. I am using ScrollAll (below) but have 2 issues.

1. The number of docs in this index is currently static, but I receive varying doc counts when I run this code. The count in Kibana is constant. I really need this to be reliable.

2. The speed is slow, using 12 slices it takes 10+ minutes to get 1.6 Mil docs. I only need an Id field in the doc, but I haven't seen a way to get just that, if that would help. Any optimization advice for this would be appreciatred.

`List resp = new List();

```
        var eClient = ThirdPartyService.ElasticClient();

        // number of slices in slice scroll  
        var numberOfSlices = 12;

        var scrollObserver = eClient.ScrollAll<ThirdPartyProgram>("15m", numberOfSlices, s => s
            .MaxDegreeOfParallelism(numberOfSlices)
            .Search(search => search
                .Preference("iva")
                .Query(q => q
                    .Match(m => m
                    .Field(f => f.Source)
                    .Query(value)
                    )
                )
            )
        ).Wait(TimeSpan.FromMinutes(15), r =>
        {
            r.SearchResponse.Documents.ForEach(x => resp.Add(x.SourceId));
        });
                   

        var u = resp.DistinctBy(x => x).ToList();
        return resp;`
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 22, 2018, 4:55pm UTC](https://discuss.elastic.co/t/scrollall-varying-record-counts-and-speed/141629/2 "2018-08-22T16:55:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.