Error when using ScrollAll for millions of records

Elastic Stack Elasticsearch
1

Error scrolling search on /_search/scroll?pretty=true with slice 4 was not valid on scroll iteration 1

2

Code Used

var observable = fromElasticClient.ScrollAll<type>(
                    "1m"
                    , numberOfShards
                    , s => s
                        .MaxDegreeOfParallelism(numberOfShards / 2)
                        .Search(search => search
                                .Size(1000)
                                .Index(fromIndexName)
                                .AllTypes()
                                .MatchAll()
                        )
                );
var waitHandle = new ManualResetEvent(false);
Exception exception = null;
observable.Subscribe(new ScrollAllObserver<type>(
                    onNext: r =>
                    {
                        //Partial Update code here
                        Interlocked.Add(ref seenDocuments, r.SearchResponse.Hits.Count);
                    },
                    onError: e =>
                    {
                        exception = e;
                        logger.Error("Error " + e.Message);
                        waitHandle.Set();
                    },
                    onCompleted: () => waitHandle.Set()
                ));
                waitHandle.WaitOne();
                if (exception != null)
                    throw exception;
1 Like
3

Any upadte?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.