Best approach for Fetching large dataset using NEST

(Waleed) #1

I'm using NEST API for fetching data from elastic Index. My index contains more than 700000 records and I want to fetch the data in chunks. I have tried it with From and size parameter on each request I'm querying according to the page number and fetching 10 records per request. But at a certain stage, the query stops fetching the data, when the from parameter is larger than 50000. I have tried changing the "index.max_result_window" to 100000 and still couldn't resolve the issue.

Is this the correct way to query a large dataset or should I use any other approach?

        var settings = new ConnectionSettings(new Uri(""))
                if (page == null || page==1)
                    start = 0;
                    start = (int)page - 1;
                    start = start * 10 + 1;
                client = new ElasticClient(settings);
                var searchResponse = client.Search<parcels>(s => s
                .Query(q => q
                .MultiMatch(c => c
                .Fields(f => f.Field("Parcel_Code").Field("CityName").Field("PlanName")
                .Query(search) //user input will be passed here as the query

above is my code.
Any help would be appreciated. TIA

(system) closed #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.