# Bulk via curl not working in data in file?
**URL:** https://discuss.elastic.co/t/bulk-via-curl-not-working-in-data-in-file/3490
**Category:** Elasticsearch
**Created:** [October 29, 2010, 3:32pm UTC](https://discuss.elastic.co/t/bulk-via-curl-not-working-in-data-in-file/3490 "2010-10-29T15:32:27Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)
#### Post date: [October 29, 2010, 3:32pm UTC](https://discuss.elastic.co/t/bulk-via-curl-not-working-in-data-in-file/3490/1 "2010-10-29T15:32:27Z")
Hi,
I am having troubles executing bulk operations via curl if the bulk command
sequence is in file.
The following works fine:
curl -XDELETE '[http://localhost:9200/test/?pretty=1](http://localhost:9200/test/?pretty=1)'
curl -XPUT '[http://localhost:9200/test/?pretty=1](http://localhost:9200/test/?pretty=1)' -d '{ index : {
number\_of\_shards : 3, number\_of\_replicas : 0 }}'
curl -XPUT '[http://localhost:9200/\_bulk](http://localhost:9200/_bulk)' -d '
{ "create" : {"\_index":"test","\_type":"one","\_id":"1"} }
{"name":"01","category":"01","subcategory":"01" }
{ "create" : {"\_index":"test","\_type":"one","\_id":"2"} }
{"name":"02","category":"01","subcategory":"02" }
{ "create" : {"\_index":"test","\_type":"one","\_id":"3"} }
{"name":"03","category":"01","subcategory":"02" }
{ "create" : {"\_index":"test","\_type":"one","\_id":"4"} }
{"name":"04","category":"02","subcategory":"01" }
{ "create" : {"\_index":"test","\_type":"one","\_id":"5"} }
{"name":"05","category":"02","subcategory":"01" }
{ "create" : {"\_index":"test","\_type":"one","\_id":"6"} }
{"name":"06","category":"01","subcategory":"02" }
'
gives me the following response:
{
"ok" : true,
"acknowledged" : true
}
{
"ok" : true,
"acknowledged" : true
}
{"items":[{"create":{"\_index":"test","\_type":"one","\_id":"1","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"2","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"3","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"4","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"5","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"6","ok":true}}]}
but if I put bulk operations into a file and run
curl -XPUT '[http://localhost:9200/\_bulk](http://localhost:9200/_bulk)' -d @bulk.data
then it gives me:
{"error":"ActionRequestValidationException[Validation Failed: 1: no requests
added]"}
Content of the file is the following:
$ cat bulk.data
{ "create" : {"\_index":"test","\_type":"one","\_id":"1"} }
{"name":"01","category":"01","subcategory":"01" }
{ "create" : {"\_index":"test","\_type":"one","\_id":"2"} }
{"name":"02","category":"01","subcategory":"02" }
{ "create" : {"\_index":"test","\_type":"one","\_id":"3"} }
{"name":"03","category":"01","subcategory":"02" }
{ "create" : {"\_index":"test","\_type":"one","\_id":"4"} }
{"name":"04","category":"02","subcategory":"01" }
{ "create" : {"\_index":"test","\_type":"one","\_id":"5"} }
{"name":"05","category":"02","subcategory":"01" }
{ "create" : {"\_index":"test","\_type":"one","\_id":"6"} }
{"name":"06","category":"01","subcategory":"02" }
$
(note the extra new line at the end of the file data)
Regards,
Lukas
---
### Author:  [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)
#### Post date: [October 29, 2010, 3:36pm UTC](https://discuss.elastic.co/t/bulk-via-curl-not-working-in-data-in-file/3490/2 "2010-10-29T15:36:51Z")
Ah, the subject should have been: "Bulk via curl not working IF data in
file?"
On Fri, Oct 29, 2010 at 5:32 PM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) wrote:
> Hi,
>
> I am having troubles executing bulk operations via curl if the bulk command
> sequence is in file.
>
> The following works fine:
>
> curl -XDELETE '[http://localhost:9200/test/?pretty=1](http://localhost:9200/test/?pretty=1)'
>
> curl -XPUT '[http://localhost:9200/test/?pretty=1](http://localhost:9200/test/?pretty=1)' -d '{ index : {
> number\_of\_shards : 3, number\_of\_replicas : 0 }}'
>
> curl -XPUT '[http://localhost:9200/\_bulk](http://localhost:9200/_bulk)' -d '
> { "create" : {"\_index":"test","\_type":"one","\_id":"1"} }
> {"name":"01","category":"01","subcategory":"01" }
> { "create" : {"\_index":"test","\_type":"one","\_id":"2"} }
> {"name":"02","category":"01","subcategory":"02" }
> { "create" : {"\_index":"test","\_type":"one","\_id":"3"} }
> {"name":"03","category":"01","subcategory":"02" }
> { "create" : {"\_index":"test","\_type":"one","\_id":"4"} }
> {"name":"04","category":"02","subcategory":"01" }
> { "create" : {"\_index":"test","\_type":"one","\_id":"5"} }
> {"name":"05","category":"02","subcategory":"01" }
> { "create" : {"\_index":"test","\_type":"one","\_id":"6"} }
> {"name":"06","category":"01","subcategory":"02" }
>
> '
>
> gives me the following response:
> {
> "ok" : true,
> "acknowledged" : true
> }
> {
> "ok" : true,
> "acknowledged" : true
> }
>
> {"items":[{"create":{"\_index":"test","\_type":"one","\_id":"1","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"2","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"3","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"4","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"5","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"6","ok":true}}]}
>
> but if I put bulk operations into a file and run
>
> curl -XPUT '[http://localhost:9200/\_bulk](http://localhost:9200/_bulk)' -d @bulk.data
>
> then it gives me:
>
> {"error":"ActionRequestValidationException[Validation Failed: 1: no
> requests added]"}
>
> Content of the file is the following:
>
> $ cat bulk.data
> { "create" : {"\_index":"test","\_type":"one","\_id":"1"} }
> {"name":"01","category":"01","subcategory":"01" }
> { "create" : {"\_index":"test","\_type":"one","\_id":"2"} }
> {"name":"02","category":"01","subcategory":"02" }
> { "create" : {"\_index":"test","\_type":"one","\_id":"3"} }
> {"name":"03","category":"01","subcategory":"02" }
> { "create" : {"\_index":"test","\_type":"one","\_id":"4"} }
> {"name":"04","category":"02","subcategory":"01" }
> { "create" : {"\_index":"test","\_type":"one","\_id":"5"} }
> {"name":"05","category":"02","subcategory":"01" }
> { "create" : {"\_index":"test","\_type":"one","\_id":"6"} }
> {"name":"06","category":"01","subcategory":"02" }
>
> $
>
> (note the extra new line at the end of the file data)
>
> Regards,
> Lukas
---
### Author:  [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)
#### Post date: [October 29, 2010, 3:42pm UTC](https://discuss.elastic.co/t/bulk-via-curl-not-working-in-data-in-file/3490/3 "2010-10-29T15:42:16Z")
I am running Linux (fedora)
On Fri, Oct 29, 2010 at 5:36 PM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) wrote:
> Ah, the subject should have been: "Bulk via curl not working IF data in
> file?"
>
> On Fri, Oct 29, 2010 at 5:32 PM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com)wrote:
>
> > Hi,
> >
> > I am having troubles executing bulk operations via curl if the bulk
> > command sequence is in file.
> >
> > The following works fine:
> >
> > curl -XDELETE '[http://localhost:9200/test/?pretty=1](http://localhost:9200/test/?pretty=1)'
> >
> > curl -XPUT '[http://localhost:9200/test/?pretty=1](http://localhost:9200/test/?pretty=1)' -d '{ index : {
> > number\_of\_shards : 3, number\_of\_replicas : 0 }}'
> >
> > curl -XPUT '[http://localhost:9200/\_bulk](http://localhost:9200/_bulk)' -d '
> > { "create" : {"\_index":"test","\_type":"one","\_id":"1"} }
> > {"name":"01","category":"01","subcategory":"01" }
> > { "create" : {"\_index":"test","\_type":"one","\_id":"2"} }
> > {"name":"02","category":"01","subcategory":"02" }
> > { "create" : {"\_index":"test","\_type":"one","\_id":"3"} }
> > {"name":"03","category":"01","subcategory":"02" }
> > { "create" : {"\_index":"test","\_type":"one","\_id":"4"} }
> > {"name":"04","category":"02","subcategory":"01" }
> > { "create" : {"\_index":"test","\_type":"one","\_id":"5"} }
> > {"name":"05","category":"02","subcategory":"01" }
> > { "create" : {"\_index":"test","\_type":"one","\_id":"6"} }
> > {"name":"06","category":"01","subcategory":"02" }
> >
> > '
> >
> > gives me the following response:
> > {
> > "ok" : true,
> > "acknowledged" : true
> > }
> > {
> > "ok" : true,
> > "acknowledged" : true
> > }
> >
> > {"items":[{"create":{"\_index":"test","\_type":"one","\_id":"1","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"2","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"3","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"4","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"5","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"6","ok":true}}]}
> >
> > but if I put bulk operations into a file and run
> >
> > curl -XPUT '[http://localhost:9200/\_bulk](http://localhost:9200/_bulk)' -d @bulk.data
> >
> > then it gives me:
> >
> > {"error":"ActionRequestValidationException[Validation Failed: 1: no
> > requests added]"}
> >
> > Content of the file is the following:
> >
> > $ cat bulk.data
> > { "create" : {"\_index":"test","\_type":"one","\_id":"1"} }
> > {"name":"01","category":"01","subcategory":"01" }
> > { "create" : {"\_index":"test","\_type":"one","\_id":"2"} }
> > {"name":"02","category":"01","subcategory":"02" }
> > { "create" : {"\_index":"test","\_type":"one","\_id":"3"} }
> > {"name":"03","category":"01","subcategory":"02" }
> > { "create" : {"\_index":"test","\_type":"one","\_id":"4"} }
> > {"name":"04","category":"02","subcategory":"01" }
> > { "create" : {"\_index":"test","\_type":"one","\_id":"5"} }
> > {"name":"05","category":"02","subcategory":"01" }
> > { "create" : {"\_index":"test","\_type":"one","\_id":"6"} }
> > {"name":"06","category":"01","subcategory":"02" }
> >
> > $
> >
> > (note the extra new line at the end of the file data)
> >
> > Regards,
> > Lukas
---
### Author:  [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)
#### Post date: [October 29, 2010, 3:58pm UTC](https://discuss.elastic.co/t/bulk-via-curl-not-working-in-data-in-file/3490/4 "2010-10-29T15:58:03Z")
So the trick is to use --data-binary
curl -XPUT '[http://localhost:9200/\_bulk](http://localhost:9200/_bulk)' --data-binary @bulk.data
thanks @clintongormley
On Fri, Oct 29, 2010 at 5:42 PM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) wrote:
> I am running Linux (fedora)
>
> On Fri, Oct 29, 2010 at 5:36 PM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com)wrote:
>
> > Ah, the subject should have been: "Bulk via curl not working IF data in
> > file?"
> >
> > On Fri, Oct 29, 2010 at 5:32 PM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com)wrote:
> >
> > > Hi,
> > >
> > > I am having troubles executing bulk operations via curl if the bulk
> > > command sequence is in file.
> > >
> > > The following works fine:
> > >
> > > curl -XDELETE '[http://localhost:9200/test/?pretty=1](http://localhost:9200/test/?pretty=1)'
> > >
> > > curl -XPUT '[http://localhost:9200/test/?pretty=1](http://localhost:9200/test/?pretty=1)' -d '{ index : {
> > > number\_of\_shards : 3, number\_of\_replicas : 0 }}'
> > >
> > > curl -XPUT '[http://localhost:9200/\_bulk](http://localhost:9200/_bulk)' -d '
> > > { "create" : {"\_index":"test","\_type":"one","\_id":"1"} }
> > > {"name":"01","category":"01","subcategory":"01" }
> > > { "create" : {"\_index":"test","\_type":"one","\_id":"2"} }
> > > {"name":"02","category":"01","subcategory":"02" }
> > > { "create" : {"\_index":"test","\_type":"one","\_id":"3"} }
> > > {"name":"03","category":"01","subcategory":"02" }
> > > { "create" : {"\_index":"test","\_type":"one","\_id":"4"} }
> > > {"name":"04","category":"02","subcategory":"01" }
> > > { "create" : {"\_index":"test","\_type":"one","\_id":"5"} }
> > > {"name":"05","category":"02","subcategory":"01" }
> > > { "create" : {"\_index":"test","\_type":"one","\_id":"6"} }
> > > {"name":"06","category":"01","subcategory":"02" }
> > >
> > > '
> > >
> > > gives me the following response:
> > > {
> > > "ok" : true,
> > > "acknowledged" : true
> > > }
> > > {
> > > "ok" : true,
> > > "acknowledged" : true
> > > }
> > >
> > > {"items":[{"create":{"\_index":"test","\_type":"one","\_id":"1","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"2","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"3","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"4","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"5","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"6","ok":true}}]}
> > >
> > > but if I put bulk operations into a file and run
> > >
> > > curl -XPUT '[http://localhost:9200/\_bulk](http://localhost:9200/_bulk)' -d @bulk.data
> > >
> > > then it gives me:
> > >
> > > {"error":"ActionRequestValidationException[Validation Failed: 1: no
> > > requests added]"}
> > >
> > > Content of the file is the following:
> > >
> > > $ cat bulk.data
> > > { "create" : {"\_index":"test","\_type":"one","\_id":"1"} }
> > > {"name":"01","category":"01","subcategory":"01" }
> > > { "create" : {"\_index":"test","\_type":"one","\_id":"2"} }
> > > {"name":"02","category":"01","subcategory":"02" }
> > > { "create" : {"\_index":"test","\_type":"one","\_id":"3"} }
> > > {"name":"03","category":"01","subcategory":"02" }
> > > { "create" : {"\_index":"test","\_type":"one","\_id":"4"} }
> > > {"name":"04","category":"02","subcategory":"01" }
> > > { "create" : {"\_index":"test","\_type":"one","\_id":"5"} }
> > > {"name":"05","category":"02","subcategory":"01" }
> > > { "create" : {"\_index":"test","\_type":"one","\_id":"6"} }
> > > {"name":"06","category":"01","subcategory":"02" }
> > >
> > > $
> > >
> > > (note the extra new line at the end of the file data)
> > >
> > > Regards,
> > > Lukas
---
### Author:  [@hartzler\_2](https://discuss.elastic.co/u/hartzler_2)
#### Post date: [October 29, 2010, 4:04pm UTC](https://discuss.elastic.co/t/bulk-via-curl-not-working-in-data-in-file/3490/5 "2010-10-29T16:04:58Z")
thanks for posting the solution! I was having the same issue.
On Fri, Oct 29, 2010 at 10:58 AM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) wrote:
> So the trick is to use --data-binary
> curl -XPUT '[http://localhost:9200/\_bulk](http://localhost:9200/_bulk)' --data-binary @bulk.data
>
> thanks @clintongormley
>
> On Fri, Oct 29, 2010 at 5:42 PM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com)wrote:
>
> > I am running Linux (fedora)
> >
> > On Fri, Oct 29, 2010 at 5:36 PM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com)wrote:
> >
> > > Ah, the subject should have been: "Bulk via curl not working IF data in
> > > file?"
> > >
> > > On Fri, Oct 29, 2010 at 5:32 PM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com)wrote:
> > >
> > > > Hi,
> > > >
> > > > I am having troubles executing bulk operations via curl if the bulk
> > > > command sequence is in file.
> > > >
> > > > The following works fine:
> > > >
> > > > curl -XDELETE '[http://localhost:9200/test/?pretty=1](http://localhost:9200/test/?pretty=1)'
> > > >
> > > > curl -XPUT '[http://localhost:9200/test/?pretty=1](http://localhost:9200/test/?pretty=1)' -d '{ index : {
> > > > number\_of\_shards : 3, number\_of\_replicas : 0 }}'
> > > >
> > > > curl -XPUT '[http://localhost:9200/\_bulk](http://localhost:9200/_bulk)' -d '
> > > > { "create" : {"\_index":"test","\_type":"one","\_id":"1"} }
> > > > {"name":"01","category":"01","subcategory":"01" }
> > > > { "create" : {"\_index":"test","\_type":"one","\_id":"2"} }
> > > > {"name":"02","category":"01","subcategory":"02" }
> > > > { "create" : {"\_index":"test","\_type":"one","\_id":"3"} }
> > > > {"name":"03","category":"01","subcategory":"02" }
> > > > { "create" : {"\_index":"test","\_type":"one","\_id":"4"} }
> > > > {"name":"04","category":"02","subcategory":"01" }
> > > > { "create" : {"\_index":"test","\_type":"one","\_id":"5"} }
> > > > {"name":"05","category":"02","subcategory":"01" }
> > > > { "create" : {"\_index":"test","\_type":"one","\_id":"6"} }
> > > > {"name":"06","category":"01","subcategory":"02" }
> > > >
> > > > '
> > > >
> > > > gives me the following response:
> > > > {
> > > > "ok" : true,
> > > > "acknowledged" : true
> > > > }
> > > > {
> > > > "ok" : true,
> > > > "acknowledged" : true
> > > > }
> > > >
> > > > {"items":[{"create":{"\_index":"test","\_type":"one","\_id":"1","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"2","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"3","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"4","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"5","ok":true}},{"create":{"\_index":"test","\_type":"one","\_id":"6","ok":true}}]}
> > > >
> > > > but if I put bulk operations into a file and run
> > > >
> > > > curl -XPUT '[http://localhost:9200/\_bulk](http://localhost:9200/_bulk)' -d @bulk.data
> > > >
> > > > then it gives me:
> > > >
> > > > {"error":"ActionRequestValidationException[Validation Failed: 1: no
> > > > requests added]"}
> > > >
> > > > Content of the file is the following:
> > > >
> > > > $ cat bulk.data
> > > > { "create" : {"\_index":"test","\_type":"one","\_id":"1"} }
> > > > {"name":"01","category":"01","subcategory":"01" }
> > > > { "create" : {"\_index":"test","\_type":"one","\_id":"2"} }
> > > > {"name":"02","category":"01","subcategory":"02" }
> > > > { "create" : {"\_index":"test","\_type":"one","\_id":"3"} }
> > > > {"name":"03","category":"01","subcategory":"02" }
> > > > { "create" : {"\_index":"test","\_type":"one","\_id":"4"} }
> > > > {"name":"04","category":"02","subcategory":"01" }
> > > > { "create" : {"\_index":"test","\_type":"one","\_id":"5"} }
> > > > {"name":"05","category":"02","subcategory":"01" }
> > > > { "create" : {"\_index":"test","\_type":"one","\_id":"6"} }
> > > > {"name":"06","category":"01","subcategory":"02" }
> > > >
> > > > $
> > > >
> > > > (note the extra new line at the end of the file data)
> > > >
> > > > Regards,
> > > > Lukas
---
### Author:  [@searchersteve](https://discuss.elastic.co/u/searchersteve)
#### Post date: [October 15, 2011, 12:25am UTC](https://discuss.elastic.co/t/bulk-via-curl-not-working-in-data-in-file/3490/6 "2011-10-15T00:25:04Z")
I would note that many articles out there seem to indicate you need to urlencode the contents of a datafile being posted via curl in binary mode.
As you have found -- and I have found -- this is not necessary when posting JSON as content.
---
### Author:  [@cn081](https://discuss.elastic.co/u/cn081)
#### Post date: [February 25, 2014, 10:13am UTC](https://discuss.elastic.co/t/bulk-via-curl-not-working-in-data-in-file/3490/7 "2014-02-25T10:13:17Z")
I needed to do something similar from within Ruby code w/o writing to a file first, so see code below for an example:
## example json data generated by the code below, please note that the meta fields and source fields are separated by a newline ([http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-bulk.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-bulk.html))
=begin
{:index =\> {"\_index" =\> "xxx", "\_type" =\> "yyy", "\_id" =\> "xxx-yyy-zzz"}}
{:field0 =\> [{"field1" =\> "value1", "field2" =\> "value2", "field3" =\> "value3"}, {"field1" =\> "value1", "field2" =\> "value2", "field3" =\> "value3"}, {"field1" =\> "value1", "field2" =\> "value2", "field3" =\> "value3"}]}
=end
host = 'localhost'
bulk\_contents \<\< {:index =\> meta\_data\_fields} # meta data fields contains fields such as \_id, \_type, and \_index
documents \<\< document\_fields # fields specific to the document to be modified
bulk\_contents \<\< {:field0 =\> documents} # please see above of an example of the generated json. The 'field0' key is part of a partial document
bulk\_contents\_as\_string = bulk\_contents.map(&:to\_json).join("\n")
bulk\_update\_request = "curl -s -XPOST 'http://#{host}:9200/\_bulk' --data-binary '#{bulk\_contents\_as\_string}\n'" # terminated by a newline
system("#{bulk\_update\_request} \> /dev/null") # or system(bulk\_update\_request) if you require the output
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:47am UTC](https://discuss.elastic.co/t/bulk-via-curl-not-working-in-data-in-file/3490/8 "2017-07-06T01:47:31Z")