Bulk insert file having many json entries into Elasticsearch

Elastic Stack Elasticsearch
1

Hi,

I just wanted to bulk insert into Elasticsearch. I have a file that has many json entries and looks like:
{"level":"debug","message":"POST : /info","timestamp":"2016-03-31T08:54:52.236Z"}
{"level":"debug","message":"POST : /info","timestamp":"2016-03-31T08:54:52.265Z"}
{"level":"debug","message":"POST : /info","timestamp":"2016-03-31T08:54:52.679Z"}
{"level":"debug","message":"POST : /info","timestamp":"2016-03-31T08:54:52.242Z"}

I tried inserting this file using _bulk api and got following error:
{
"error" : {
"root_cause" : [ {
"type" : "illegal_argument_exception",
"reason" : "Malformed action/metadata line [1], expected START_OBJECT or END_OBJECT but found [VALUE_STRING]"
} ],
"type" : "illegal_argument_exception",
"reason" : "Malformed action/metadata line [1], expected START_OBJECT or END_OBJECT but found [VALUE_STRING]"
},
"status" : 400
}

I could insert docs into ES:
curl -XPUT localhost:9200/_bulk --data-binary @shakespeare.json (from docs)

But, how can I insert data mentioned above into Elasticsearch?

Thanks!

1 Like
2

The file need to be properly formatted as a bulk request, it can not just take a file with one object per line.

3

You have to respect the BULK format. https://www.elastic.co/guide/en/elasticsearch/reference/2.3/docs-bulk.html

action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n
4

Just got it working by inserting action and metadata \n before every json.

Thanks!