Hello, I want to export elasticsearch index to HDFS, via let's say PIG.
It's working fine, but it's slow, index with 3 shards, cluster with 3 nodes BUT a single client node (access point).
Before, I was using a small home made Java program that creates X workers (multi-thread) for each shard that search-scroll in same time, took 10 minutes to load the entire index.
Whereas with hadoop, I can see there is a single search-scroll process, and it took 30 minutes
Is it possible to "force" the number of workers, even with a single client node?