# Elasticsearch Bulk Insert through Python Client API

**URL:** <https://discuss.elastic.co/t/elasticsearch-bulk-insert-through-python-client-api/279032>\
**Category:** Elasticsearch\
**Created:** [July 19, 2021, 8:37am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-through-python-client-api/279032 "2021-07-19T08:37:33Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Anirudhan](https://avatars.discourse-cdn.com/v4/letter/a/91b2a8/32.png) [@Anirudhan](https://discuss.elastic.co/u/Anirudhan)\
**Post date:** [July 19, 2021, 8:37am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-through-python-client-api/279032/1 "2021-07-19T08:37:33Z")

</div>

Dear Team,  
I am trying to bulk insert 3000 records into an elastic search using BULK API. Whichever documents having the keyword version are not being inserted.  
For eg:  
[{"name": "XXXX", "description": "XXXX", "\_id": "53a8b9a3-7f7c-4dac-bc6f-15c48b3bcbd9", "source": 2, "source\_id": "jk4jJXgB\_6QR5Hf9NoZA", "c": 1617367373, "u": 1618978346}]  
The above record is inserted into the elastic search.  
{"name": "Git version 2.31.1", "publisher": "The Git Development Community", "path": "XXX", **"version": "2.31"** , "\_id": "d0ec7bab-fa59-46a1-93c2-94c0925e416f", "companyRef": {"id": "53a8b9a3-7f7c-4dac-bc6f-15c48b3bcbd9", "name": "XXX"}, "agentRef": {"id": "606711592b2f4b23de4f6291", "name": "XXX"}, "assetRef": {"id": "53a8b9a3-7f7c-4dac-bc6f-15c48b3bcbd9\_000C2964CC22", "name": "XXX"}, "u": XXX, "c": XXX}  
The above record is not getting inserted.  
Can someone please help.

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [July 19, 2021, 9:22am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-through-python-client-api/279032/2 "2021-07-19T09:22:28Z")

</div>

please share error messages, that are returned from such requests. Thanks!

---

<div class="post-metadata">

**Author:** ![Anirudhan](https://avatars.discourse-cdn.com/v4/letter/a/91b2a8/32.png) [@Anirudhan](https://discuss.elastic.co/u/Anirudhan)\
**Post date:** [July 21, 2021, 5:52pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-through-python-client-api/279032/3 "2021-07-21T17:52:34Z")

</div>

Hi  
Sorry for the delayed response.  
Below is the error message.  
**elasticsearch.exceptions.RequestError: RequestError(400, 'illegal\_argument\_exception', 'For input string: ""11.4.7001.0""')**

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [July 22, 2021, 1:36pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-through-python-client-api/279032/4 "2021-07-22T13:36:43Z")

</div>

how is that field mapped in your index?

---

<div class="post-metadata">

**Author:** ![Anirudhan](https://avatars.discourse-cdn.com/v4/letter/a/91b2a8/32.png) [@Anirudhan](https://discuss.elastic.co/u/Anirudhan)\
**Post date:** [July 22, 2021, 2:14pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-through-python-client-api/279032/5 "2021-07-22T14:14:59Z")

</div>

It mapped in my index as a string.

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [July 22, 2021, 2:30pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-through-python-client-api/279032/6 "2021-07-22T14:30:57Z")

</div>

I somewhat doubt this (the exception looks like a number format exception), can you provide the full stacktrace as the involved field is not shown and a reproduction, that is not bound to a programming language but can be done via Kibana Dev Tools?

---

<div class="post-metadata">

**Author:** ![Anirudhan](https://avatars.discourse-cdn.com/v4/letter/a/91b2a8/32.png) [@Anirudhan](https://discuss.elastic.co/u/Anirudhan)\
**Post date:** [July 26, 2021, 5:43am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-through-python-client-api/279032/7 "2021-07-26T05:43:21Z")

</div>

Hi  
I am able to insert the data, by looping through and inserting one by one. And i am able to insert the data using the es-pandas module as well.

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [July 26, 2021, 7:24am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-through-python-client-api/279032/8 "2021-07-26T07:24:59Z")

</div>

If the above exception happens during an index operation, then it is unlikely that this document has been indexed, as you mentioned in the first post.

If the above exception does not occur when you index one-by-one I assume that your mapping at some point is dynamic where it should not be. That means, that the value of a field is determiend by the first document that contains this value. If that happens to be a string, then you will not be able to index anything as a number and vice versa.

Think about defining your mapping upfront, to make ingestion and especially querying and data storage more reliable, when you know what type of a field your are querying.

Hope this helps!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 23, 2021, 7:25am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-through-python-client-api/279032/9 "2021-08-23T07:25:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
