# Elasticsearch bulk Ingestion

**URL:** <https://discuss.elastic.co/t/elasticsearch-bulk-ingestion/270754>\
**Category:** Elasticsearch\
**Created:** [April 20, 2021, 7:11pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-ingestion/270754 "2021-04-20T19:11:10Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rokshan\_Jahan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rokshan_jahan/32/87448_2.png) [@Rokshan\_Jahan](https://discuss.elastic.co/u/Rokshan_Jahan)\
**Post date:** [April 20, 2021, 7:11pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-ingestion/270754/1 "2021-04-20T19:11:10Z")

</div>

Hi,  
I have 7 TB ( 32 billion records) to ingest in elastic search. What would be the best elastic search configuration we can use to load these data quickly.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 20, 2021, 11:04pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-ingestion/270754/2 "2021-04-20T23:04:22Z")

</div>

Welcome to our community! 😃

When you say you want to load it quickly, how quick do you want it done in?

---

<div class="post-metadata">

**Author:** ![Rokshan\_Jahan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rokshan_jahan/32/87448_2.png) [@Rokshan\_Jahan](https://discuss.elastic.co/u/Rokshan_Jahan)\
**Post date:** [April 21, 2021, 4:43am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-ingestion/270754/3 "2021-04-21T04:43:55Z")

</div>

It took 40 hrs to do some transformation on the same data with Dataflow. Can we achieve same for ES ingestion? Currently we are using ,  
Max no. of nodes: 12  
Each node has 8 CPU/64GB RAM/1.7TB SSD  
Index has 130 primary shards / 0 replicas (replicas will be set after the ingestion as this affects the ingestion rate when set)

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 21, 2021, 10:29pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-ingestion/270754/4 "2021-04-21T22:29:52Z")

</div>

What is the timing for that current sized cluster?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 19, 2021, 10:30pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-ingestion/270754/5 "2021-05-19T22:30:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.