Elastisearch 5.5.1 with c# throw remote exception error please provide me the solution for this

Elastic Stack Elasticsearch
1

Elastisearch 5.5.1 will fatch data from 10000 perfactly but above 10000 it will not fatch the data using c#, and return remote exception error can you please guide me the same.

2

Pagination is limited by default to 10k hits. And we don't recommend changing that value.

3

how to get elastic search more than 10000 record from elastic server with c# query

4

I think I answered. Not on C# though as I don't know it and its client but from an API perspective.

Did you look at scroll as I said i the dotNet client?

May be this page is relevant? https://www.elastic.co/guide/en/elasticsearch/client/net-api/5.x/sliced-scroll-search-usage.html#_object_initializer_syntax_example_18

Or may be @Martijn_Laarman can help?

5

You can use the scroll API to do this in the .NET client. The client API also exposes an IObservable<T> that can be used to scroll all the documents. Something like

var client = new ElasticClient();

var scrollAllObservable = client.ScrollAll<MyDocument>("1m", 10, sc => sc
    .Search(s => s
        .Query(q => q
            .MatchAll()
        )
    )
);

var handle = new ManualResetEvent(false);

var scrollObserver = new ScrollAllObserver<MyDocument>(
    // on next
    r =>
    {
        // do something with the documents
        r.SearchResponse.Documents
    },
    // on exception
    e =>
    {
        handle.Set();
        throw e;
    },
    // on completion
    () => handle.Set()
);

scrollAllObservable.Subscribe(scrollObserver); 
handle.WaitOne();
6

Elastic search 5.5.1 with c# Use Scroll But Does not Search Properly How To get more than 10000 record from elastic search with c# code ?

7

update my elastic search version but does not work proper in .net when search more than 10000 record from elastic search generate error how to solve c# with elastic search

8

It would probably help if you share the code you wrote. Please format it using </> and check it's correctly formatted in the preview window.

9 
var brandResults = ConnectionToES.EsClient().Search<Nyu_ESProductListingFiltering>(s => s
                         .From(0)
                         .Size(354734)
                         .Index("ezmaal")
                         .Type("Nyu_ESProductListingFiltering")
                         .Query(q => q.Term(t => t.Field("categoryId").Value(categoryId)) && q.Term(t => t.Field("psmFiltering").Value(1)))
                         .Aggregations(a => a.Terms("saId", t => t.Field(p => p.SaId).Size(354734)).Terms("saName", t => t.Field(p => p.SaName).Size(354734))));
10

Not sure what is the error but I’m pretty sure this will cause some problems

Size(354734)
11

var brandResults = ConnectionToES.EsClient().Search<Nyu_ESProductListingFiltering>(s => s
.From(0)
.Size(354734)
.Index("ezmaal")
.Type("Nyu_ESProductListingFiltering")
.Query(q => q.Term(t => t.Field("categoryId").Value(categoryId)) && q.Term(t => t.Field("psmFiltering").Value(1)))
.Aggregations(a => a.Terms("saId", t => t.Field(p => p.SaId).Size(354734)).Terms("saName", t => t.Field(p => p.SaName).Size(354734))));

my Query is mention above , it's executes in elasticsearch1.7.2 , where it does not executes in elasticsearch5.6.3

12

May be circuit breaker is protecting the cluster?
Try by removing size

13

when i do not set size it's giving me only 10 record as default but i need to get all record having in elasticsearch table and it is all most 354734 + records in the table

14

You have 2 size in your request.

One for the size of the resultset:

                     .From(0)
                     .Size(354734)

For this one, you must use scroll API as we already explained.

One for the Terms aggregation:

                     .Aggregations(a => a.Terms("saId", t => t.Field(p => p.SaId).Size(354734)).Terms("saName", t => t.Field(p => p.SaName).Size(354734))));

This one is more problematic as you may be won't have enough memory to run it.
May be it can work though.
I'd just ask myself about the use case and the way you are trying to solve it.

What are you trying to do here?

15

Hi,

Thank you for your suggestions and code Snippet.

I have tried that on my project. but it seems to not working for me.
Below is the code i have written to perform the feature.

Please do let me know what is missing.

 var scrollAllObservable = ConnectionToES.EsClient().ScrollAll<Nyu_ESProductListingFiltering>("1m", 10, 
                sc => sc
                .Search(s => s
                .Index("ezmaal")
                .Type("Nyu_ESProductListingFiltering")
                .From(0)
                .Size(10000)
                .Query(q => q.Term(t => t.Field("categoryId").Value(categoryId)) && q.Term(t => t.Field("psmFiltering").Value(1)))
                )
            );
16

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.