Load about 1 million document from elastic search in .net application using nest

Elastic Stack Elasticsearch
1

Hi there,

I am trying to get about 1.5 million document which includes less then 6 field from elasticsearch index to .net app and it is unreachable, super slow. Is there any way to get such number of documents in real time using nest api ?

2

Are you trying to load 1.5 million documents into memory in an app?

3

Yes, because I need some joins over returned documents and elasticsearch does not have sql like temporary tables

4

Loading 1.5 million documents to join data in memory is going to be challenging, irrespective of the data store.

It’s possible to get 1.5 million documents out of Elasticsearch using the .NET client’s observable ScrollAll() helper method, but you may want to consider denormalizing the data that is being joined, to persist it with the documents.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.