Hi,
I have 7 TB ( 32 billion records) to ingest in elastic search. What would be the best elastic search configuration we can use to load these data quickly.
Welcome to our community! 
When you say you want to load it quickly, how quick do you want it done in?
It took 40 hrs to do some transformation on the same data with Dataflow. Can we achieve same for ES ingestion? Currently we are using ,
Max no. of nodes: 12
Each node has 8 CPU/64GB RAM/1.7TB SSD
Index has 130 primary shards / 0 replicas (replicas will be set after the ingestion as this affects the ingestion rate when set)
What is the timing for that current sized cluster?
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.