# Bulk import into ES

**URL:** <https://discuss.elastic.co/t/bulk-import-into-es/29482>\
**Category:** Elasticsearch\
**Created:** [September 17, 2015, 2:33pm UTC](https://discuss.elastic.co/t/bulk-import-into-es/29482 "2015-09-17T14:33:49Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![dceder906](https://avatars.discourse-cdn.com/v4/letter/d/d9b06d/32.png) [@dceder906](https://discuss.elastic.co/u/dceder906)\
**Post date:** [September 17, 2015, 2:33pm UTC](https://discuss.elastic.co/t/bulk-import-into-es/29482/1 "2015-09-17T14:33:49Z")

</div>

I have 50 million records to import into ES.

What sort of configuration would I need for this, how many nodes, clusters, etc? I am using NEST API for C#.

How can I monitor how optimally the import is being done?

---

<div class="post-metadata">

**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\
**Post date:** [September 17, 2015, 2:51pm UTC](https://discuss.elastic.co/t/bulk-import-into-es/29482/2 "2015-09-17T14:51:11Z")

</div>

> [@dceder906](#):
>
> What sort of configuration would I need for this, how many nodes, clusters, etc? I am using NEST API for C#.

It really depends on lots of things. Size of the documents, hardware, etc. You should try it. I'd start with 1 machine for that.

> [@dceder906](#):
>
> How can I monitor how optimally the import is being done?

You can read the log for things like merges being throttled.

You can check the stack traces for things you don't expect.

Mostly, though, you should be testing different batch sizes for the bulk api and trying to find a good one for your data.

I've had lots of success tuning the `refresh_interval` interval on the index up to `30s` or `-1` (infinity) when doing bulk imports. That speeds them up a ton. See [this](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-update-settings.html) for how to set it. If you set it to `-1` you have to manually call `_refresh` after the process is done.

---

<div class="post-metadata">

**Author:** ![ea\_fox](https://avatars.discourse-cdn.com/v4/letter/e/bbce88/32.png) [@ea\_fox](https://discuss.elastic.co/u/ea_fox)\
**Post date:** [September 17, 2015, 8:24pm UTC](https://discuss.elastic.co/t/bulk-import-into-es/29482/3 "2015-09-17T20:24:54Z")

</div>

hi, check these settings:  
index.translog.flush\_threshold\_ops  
index.translog.flush\_threshold\_size

while configurating nodes do not forget to check "bootstrap.mlockall: true" )))

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:49pm UTC](https://discuss.elastic.co/t/bulk-import-into-es/29482/4 "2017-07-05T23:49:35Z")

</div>