Timeout during index refresh

Elastic Stack Elasticsearch
1

I'm getting a timeout when refreshing an index after a large bulk ingest. I'm using the synchronous client with 8.1.3 Java APIs. It seems the refresh occurs, just waiting for completion times out. It is safe to ignore this exception or should I switch to an asynchronous client to make this request?

Is there a way to get the async client from the synchronous ElasticsearchClient or do I need to create it separately?

2

Hello!
Ideally the refresh operation shouldn't timeout, so: what are you using to perform the bulk ingest? The java client offers a helper called BulkIngester which can help with large datasets, it creates its own threadpool to handle the load concurrently and it's configurable.
If it doesn't work already with the standard configuration shown in the tutorial, you can try tuning it further using its globalSettings() method to define the refresh (putting it to "true" would refresh after every operation) and customize the timeout.

Let me know if this helps :slight_smile:

3

Thank you Laura. I am using the Bulk Ingester for an intfloat-multimodal-base model to generate predicted values for two text based fields. I think I catch and ignore the timeout at this point as there is nothing I can do about it and I hope it continues to finish the refresh. I may have ingested 300K records.