Hi Team, I am trying to fetch data using rest client in java side, but not able to fetch more than 10000 and even if i am trying to fetch data less then 10000 like 5000 or 7000 it is taking too much time. please let me know hoe we can achieve it.

size parameter I set as 50000. and each document looks like this: "_source": { "product": "G10 Rates", "time_id": 20121, "wallet": 0.000057, "entity_name": "Brigade Capital Management", "country_hq": "USA", "gap_tox": "1-3", "entity_id": 208625, "revenue": 0, "gap": 0.00001, "rank": "8+", …

Can you try with less documents like 1000? Also what is the exact scroll query you are running? Have a look at https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html#sliced-scroll Might help doing things in parallel. Some hardware questions: Do you have ssd d…

I used sliced scrolling but unable to connect transport client.my code is: TransportClient client = new PreBuiltTransportClient(settings) .addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("172.21.153.176"),9300)); Error: Exception in thread "main" java.lang.AbstractMeth…

This is not related. How did you connect previously? The fact that will use slice scroll in the future is totally unrelated IMO. Or I'm missing something in which case it could help if you share the full code and the full logs (stack trace).

I used slice scroll api which works t9.Thanks!!! but performance is still slow. Any suggestion to improve performance of scroll api.

What did you do at the end? How many parallel scrolls did you run? And how? What does "slow" mean? Do you monitor elasticsearch and your application to understand where is the bottleneck?

Hi, We are fetching 40000 documents from elasticsearch using scroll API. We have given 10 slices and scroll size is 4000. But slice API is taking more time to fetch all data from elasticsearch. After fetching all data from ES, we have java code to iterate data but that code is very faster. We need t…

2 seconds we want in millisecond

How much data do you have on the each node? How much RAM and heap do you have per node? What type of storage do you have? If you want to reduce the response time as much as possible, you probably want to make sure that the full data set can fit in the operating system file cache. If this is not fea…

How to get data more than 10000 in elasticsearch

Elastic Stack Elasticsearch

dadoonet (David Pilato) December 15, 2017, 9:06am 19

2 seconds looks very good to me.
What kind of use case are you trying to solve here?

Topic		Replies	Views
Get bulk data from elasticsearch Elasticsearch	4	836	January 4, 2018
How to fetch ~12M documents(may be even more) quickly from ES using scroll API? Elasticsearch	4	932	December 28, 2017
Scroll api performance is very slow Elasticsearch	2	1560	December 6, 2017
Get all documents from an index Elasticsearch	10	110918	June 21, 2017
Search 1M data in elasticsearch using pagination Elasticsearch	3	1256	July 30, 2017

How to get data more than 10000 in elasticsearch

Related topics