Use _bulk to import json db

Elastic Stack Elasticsearch
1

Hello!
Im migratin ES from version 2 to version 6.2, and I did backups of all index with elasticdump, it create a json with this format.

{"_index":"conn-scripts-dns","_type":"devices","_id":"201711281431118946304","_score":1,"_source":{"@timestamp":"2017-11-28T14:31:01-0300","host":"sarha","modelname":"
dns-ns1","server":"10.30.10.30","time":"0.019"}}
{"_index":"conn-scripts-dns","_type":"devices","_id":"201711281431220209432","_score":1,"_source":{"@timestamp":"2017-11-28T14:31:01-0300","host":"auth","modelname":"dns-
ns1","server":"10.30.10.30","time":"0.025"}}

is posible run a bulk POST with this json format ?

I try:
curl -H 'Content-Type:application/json' -XPUT 'http://localhost:9200/_bulk' -d '{"_index":"conn-scripts-dns","_type":"devices","_id":"201711281431118946304","_score":1,"_source" {"@timestamp":"2017-11-28T14:31:01-0300","host":"sarha","modelname":"dns-ns1","server":"10.30.10.30","time":"0.019"}}'

but I got this error:
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Malformed action/metadata line [1], expected START_OBJECT or END_OBJECT but found [VALUE_STRING]"}],"type":"illegal_argument_exception","reason":"Malformed action/metadata line [1], expected START_OBJECT or END_OBJECT but found [VALUE_STRING]"},"status":400}

I have many GB in json to import.. please help.
thanks.

2

Even though it seems to have all the information required, it is not in the format the bulk API requires, so you probably need to create a script to transform it into the correct format before ingesting it. You could probably also get Logstash to parse the lines and use a few filters to perform the required transformation.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.