Elasticsearch Python Lib

Elastic Stack Elasticsearch
1

Hi, I'm using Elastic Search python lib (8.8.2) to bulk insert data into a index. There are almost 2lakh+ documents I need to insert.

I'm using parallel_bulk api to sync them, but as I track the process RAM usage I see Parallel_bulk creates threads to process the data and the virtual memory footprint of those threads keeps on increasing.

I have 32GB RAM on the machine where from data is pushed. And Elastic Search is on a separate machine with 32GB RAM.

Is there anyway to limit the memory usage of these threads????
Tried setting thread_count to 1 but it still does not respects it

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.