Unable to send json to elastic

Elastic Stack Logstash
13

Ok, the problem is related to the input file format.
As I said few messages ago, the JSON file you're trying to parse contains a list of JSON objects.

Logstash usually can work with files containing one JSON object per line.

E.g.

{ "fieldone": 1, "anotherfield": "http://..." ... }
{ "fieldone": 2, "anotherfield": "http://..." ... }
{ "fieldone": 3, "anotherfield": "http://..." ... }
{ "fieldone": 4, "anotherfield": "http://..." ... }

While your file seems to be as follows:

[{ "fieldone": 1,
  "anotherfield": "http://..." ... 
},
{
  "fieldone": 2,
  "anotherfield": "http://..." ... 
},
{
  "fieldone": 3,
  "anotherfield": "http://..." ... 
},
{ 
  "fieldone": 4,
  "anotherfield": "http://..." ... }]

The file can be converted to the correct format using Python for example:

import json
with open('infile.json') as input_file:
    data = json.load(input_file)
    with open('outfile.json', 'w') as out_file:
        for e in data:
           json.dump(e, out_file)
           out_file.write('\n')

Or otherwise, we need to find a way to detect the last ] which terminates the JSON list.

1 Like