# Where to find examples of bulk indexing for ES6? **URL:** \ **Category:** Elasticsearch\ **Created:** [April 8, 2018, 12:54pm UTC](https://discuss.elastic.co/t/where-to-find-examples-of-bulk-indexing-for-es6/127212 "2018-04-08T12:54:06Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![timogoosen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timogoosen/32/29750_2.png) [@timogoosen](https://discuss.elastic.co/u/timogoosen)\ **Post date:** [April 8, 2018, 12:54pm UTC](https://discuss.elastic.co/t/where-to-find-examples-of-bulk-indexing-for-es6/127212/1 "2018-04-08T12:54:06Z")
I have a bunch of notes on bulk indexing with examples and code that I used to use to use for indexing data into Elasticsearch via the bulk index API. None of the examples seem to work and I run into all kinds of errors. I wanted to know if anyone can point me to a good tutorial to help me figure out what I could possibly be doing wrong. I know since version 6 allot has changed for example this: [https://www.elastic.co/blog/strict-content-type-checking-for-elasticsearch-rest-requests](https://www.elastic.co/blog/strict-content-type-checking-for-elasticsearch-rest-requests) where ES6 checks the content type of the body of REST requests. This thus means that if you are going to be doing bulk index operations for example of a json file using curl you would have to add the content type now, thus the request would have to look something like this: ```auto $ curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary @out.json ``` Are there any other changes that I need to be aware of and are there any changes to the bulk index format that is accepted by Elasticsearch's \_bulk index API? Thanks ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 9, 2018, 2:55pm UTC](https://discuss.elastic.co/t/where-to-find-examples-of-bulk-indexing-for-es6/127212/2 "2018-04-09T14:55:55Z")
the format itself has not changed. See [https://www.elastic.co/guide/en/elasticsearch/reference/6.2/docs-bulk.html](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/docs-bulk.html) Can you be more specific what you mean with `all kinds of errors`, so we can see what we can do to improve documentation in that regard? Thanks a lot! ---
**Author:** ![timogoosen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timogoosen/32/29750_2.png) [@timogoosen](https://discuss.elastic.co/u/timogoosen)\ **Post date:** [May 3, 2018, 1:41pm UTC](https://discuss.elastic.co/t/where-to-find-examples-of-bulk-indexing-for-es6/127212/4 "2018-05-03T13:41:56Z")
I created another post about this that was a bit more descriptive. See here: [Looking for working example data set to bulk index into ES6](https://discuss.elastic.co/t/looking-for-working-example-data-set-to-bulk-index-into-es6/128678) Seems I havn't solved the issue yet. You can see a video of what I'm trying to do here: [https://asciinema.org/a/iqONeVdBwCjAO0cHJ07HPWlbU](https://asciinema.org/a/iqONeVdBwCjAO0cHJ07HPWlbU) It comes down to: I want to be able to index data and only specify the index and type. In the documentation it specifies that you don't need to specify the id each entry for a type. The video shows step by step what I'm trying to do and the error I'm getting. Let me know if you can't open the video and I'lll just copy paste all my steps in here. ---
**Author:** ![timogoosen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timogoosen/32/29750_2.png) [@timogoosen](https://discuss.elastic.co/u/timogoosen)\ **Post date:** [May 3, 2018, 3:29pm UTC](https://discuss.elastic.co/t/where-to-find-examples-of-bulk-indexing-for-es6/127212/5 "2018-05-03T15:29:59Z")
From what I understand the id's should be auto generated. See this doc: [https://www.elastic.co/guide/en/elasticsearch/guide/current/bulk.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/bulk.html) "If no \_id is specified, an ID will be autogenerated:" Then the documentation shows the request body where the id is not specified (implying it will be autogenerated): ``` { "index": { "_index": "website", "_type": "blog" }} { "title": "My second blog post" } ``` ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 3, 2018, 3:44pm UTC](https://discuss.elastic.co/t/where-to-find-examples-of-bulk-indexing-for-es6/127212/6 "2018-05-03T15:44:10Z")
These examples work for me: ```auto curl -XPOST localhost:9200/_bulk -H 'Content-Type: application/json' -d' { "index": { "_index": "website", "_type": "blog" }} { "title": "My first blog post" } ' ``` ```auto curl -XPOST localhost:9200/website/blog/_bulk -H 'Content-Type: application/json' -d' { "index": {}} { "title": "My second blog post" } ' ``` ---
**Author:** ![timogoosen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timogoosen/32/29750_2.png) [@timogoosen](https://discuss.elastic.co/u/timogoosen)\ **Post date:** [May 3, 2018, 8:00pm UTC](https://discuss.elastic.co/t/where-to-find-examples-of-bulk-indexing-for-es6/127212/7 "2018-05-03T20:00:56Z")
> [@Christian\_Dahlqvist](#): > > url -XPOST localhost:9200/website/blog/\_bulk -H 'Content-Type: application/json' -d' > { "index": {}} > { "title": "My second blog post" } > ' Your examples work for me too. If you have a look at my examples they were: > [@Timo Goosen](#): > > curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/\_bulk --data-binary @out.json Which means that my examples were making use of a different content type which is "Content-Type: application/x-ndjson" Where your examples are making use of :"Content-Type: application/json" i can give you the steps by step of what I've been doing so far,but it is also on the video that I linked earlier. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 3, 2018, 8:05pm UTC](https://discuss.elastic.co/t/where-to-find-examples-of-bulk-indexing-for-es6/127212/8 "2018-05-03T20:05:00Z")
Looking at the file at the 3:30 mark in your video, it looks like your document lines might not be correctly formatted. ---
**Author:** ![timogoosen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timogoosen/32/29750_2.png) [@timogoosen](https://discuss.elastic.co/u/timogoosen)\ **Post date:** [May 7, 2018, 8:31am UTC](https://discuss.elastic.co/t/where-to-find-examples-of-bulk-indexing-for-es6/127212/9 "2018-05-07T08:31:34Z")
I see someone else solved it also in my old thread. > [@Looking for working example data set to bulk index into ES6](https://discuss.elastic.co/t/looking-for-working-example-data-set-to-bulk-index-into-es6/128678/3): > > I got the auto generate id posting right with the bulk api via: $ chmod +x convert.py $ cat convert.py #!/usr/bin/env python src\_file = 'src\_file.json' dest\_file = 'dest\_file.json' metadata = '{"index": {"\_index": "bank\_accounts", "\_type": "account"}}' with open(src\_file) as open\_file: lines = open\_file.readlines() lines = [line.replace(' ', '') for line in lines] with open(dest\_file, 'w') as f: for each\_line in lines: f.write(metadata + '\n') f.writelines(each\_lin… Thanks for your help. Seemed like I had a small issue with how I was iterating over each document in my python code. Thanks for your help! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 4, 2018, 8:31am UTC](https://discuss.elastic.co/t/where-to-find-examples-of-bulk-indexing-for-es6/127212/10 "2018-06-04T08:31:38Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.