How to deal with brackets json input filter

Elastic Stack Logstash
1

Hi there,

I'm trying to figure out how to deal with brackets in logstash. The code is as following:

{
"test1": "text",
"images": [
    "https://test2.com/images/I/test.jpg",
    "https://test2.com/images/I/test.jpg",
    "https://test2.com/images/I/test.jpg",
    "https://test2.com/images/I/test.jpg",
    "https://test2.com/images/I/test.jpg",
    "https://test2.com/images/I/test.jpg",
    "https://test2.com/images/I/test.jpg"
],
"test2": "text",
"test": 4.8
}

Any idea how to parse this in a proper way so i wont get errors?

{:error=>#<LogStash::Json::ParserError: Unexpected close marker ']': expected '}' (for root starting at [Source: (String)"]["; line: 1, column: 0])
 at [Source: (String)"]["; line: 1, column: 2]>, :data=>"]["}
2

That is not valid JSON, so a parser error is expected. What does your configuration look like? Are you reading a file? Is the JSON pretty printed?

3 
input {
    file {
      path => "/var/log/logstash/test.json"
      codec => json
      start_position => "beginning"
    }
  }
  
  
  output {
    elasticsearch {
      index => "test"
      hosts => ["https://192.168.2.6:9200"]
      cacert => "/etc/logstash/certs/logstash.pem"
      ssl_certificate_verification => false
      user => "test"
      password => "test"
4

Is a single JSON object spread across multiple lines?

5

No it's actually not

6

Then somewhere in your file you have a line that contains "][", which would be expected to cause that error.

7

Hmm strange because the only brackets in the file are the brackets of the field images:

[2021-05-13T00:00:37,436][ERROR][logstash.codecs.json     ][main][08ae9b307c2d0f73d6caf27e682f549d737fbda856d1ad084aa68cedaff5eb62] JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Unexpected close marker ']': expected '}' (for root starting at [Source: (String)"]["; line: 1, column: 0])
 at [Source: (String)"]["; line: 1, column: 2]>, :data=>"]["}

I didn't restart logstash after changing the config file. I have restarted it and tried again but no errors occur but also the data is not coming into elastic.

8

The in-memory sincedb (which records how much of the file logstash has processed) is persisted across restarts by default. You will need to append data to the file, or else use sincedb_path => "/dev/null"

9

Oke i figured out that the error was occured of the other index. But it still leaves me with the issue.

If i completely remove the json file and add new data to it it starts with [ and ends with ] do i need to remove these brackets or do i need to use another codec or something?

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.