# Logstash- How to parse formatted JSON arrays in log files

**URL:** <https://discuss.elastic.co/t/logstash-how-to-parse-formatted-json-arrays-in-log-files/339498>\
**Category:** Logstash\
**Created:** [July 28, 2023, 6:15am UTC](https://discuss.elastic.co/t/logstash-how-to-parse-formatted-json-arrays-in-log-files/339498 "2023-07-28T06:15:33Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![fisher\_he](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fisher_he/32/110259_2.png) [@fisher\_he](https://discuss.elastic.co/u/fisher_he)\
**Post date:** [July 28, 2023, 6:15am UTC](https://discuss.elastic.co/t/logstash-how-to-parse-formatted-json-arrays-in-log-files/339498/1 "2023-07-28T06:15:33Z")

</div>

Logstash version: 7.17.10  
Elasticsearch version:7.17.10

The logs are located in `/var/logs` directory and the format is as below:

xxx.log

```json
[
  {
    "t": "SYS",
    "dt": "2023-04-17 19:46:40.147 GMT-04:00",
    "c": "MenuSectionsViewController",
    "cpu": "cpu usage: 40%",
    "m": "dealloc",
    "msg": "",
    "mem": "total: 4177M used: 169M free: 2398M",
    "b": "8.25.1"
  },
  {
    "t": "SYS",
    "dt": "2023-04-17 19:46:40.154 GMT-04:00",
    "c": "OrderTicketView",
    "cpu": "cpu usage: 40%",
    "m": "dealloc",
    "msg": "",
    "mem": "total: 4177M used: 169M free: 2398M",
    "b": "8.25.1"
  }
]

```

This is my logstash.conf:

```auto
input {
  file {
    path => "/var/logs/*.log"
    start_position => "beginning"
    sincedb_path => "/dev/null"
    codec => multiline {
      pattern => "^\["
      negate => true
      what => "previous"
    } 
  }
}

filter{
    json { source => message }
}

output {
  stdout {
    codec => rubydebug
  } 
  elasticsearch {
    hosts => ["http://elasticsearch:9200"]
    index => "device-logs-%{+YYYY.MM.dd}"
  }
}

```

I will get a JSON parse error based on the previous config. details:

```auto
Error parsing json {:source=>"message", :raw=>"[\n {\n \"t\": \"SYS\",\n \"dt\": \"2023-04-17 19:46:40.147 GMT-04:00\",\n \"c\": \"MenuSectionsViewController\",\n \"cpu\": \"cpu usage: 40%\",\n \"m\": \"dealloc\",\n \"msg\": \"\",\n \"mem\": \"total: 4177M used: 169M free: 2398M\",\n \"b\": \"8.25.1\"\n },\n {\n \"t\": \"SYS\",\n \"dt\": \"2023-04-17 19:46:40.154 GMT-04:00\",\n \"c\": \"OrderTicketView\",\n \"cpu\": \"cpu usage: 40%\",\n \"m\": \"dealloc\",\n \"msg\": \"\",\n \"mem\": \"total: 4177M used: 169M free: 2398M\",\n \"b\": \"8.25.1\"\n },\n {\n \"t\": \"SYS\",\n \"dt\": \"2023-04-17 19:46:40.155 GMT-04:00\",\n \"c\": \"TicketSideBackground\",\n \"cpu\": \"cpu usage: 40%\",\n \"m\": \"dealloc\",\n \"msg\": \"\",\n \"mem\": \"total: 4177M used: 169M free: 2398M\",\n \"b\": \"8.25.1\"\n }", :exception=>#<LogStash::Json::ParserError: Unexpected end-of-input: expected close marker for Array (start marker at [Source: (byte[])"[

```

As you know, I want to use Logstash to parse JSON arrays in log files and output each JSON object as a separate document to Elasticsearch. My expected log documents are as follows:

doc\_1

```json
{
  "_index": "device-logs-2023.07.28",
  "_type": "_doc",
  "_id": "VVqImokBM0y425kVF-bp",
  "_version": 1,
  "_score": 1,
  "_source": {
    "path": "/var/logs/2.log",
    "tags": [
      "multiline",
      "_jsonparsefailure"
    ],
    "@timestamp": "2023-07-28T03:25:57.330Z",
    "host": "2b9233260805",
    "@version": "1",
    "message": {
          "t":"SYS",
          "dt": "2023-04-17 19:46:40.147 GMT-04:00",
           "c": "MenuSectionsViewController"
            ....
       }

  }
...

```

But unfortunately, I have tried multiple configurations and still haven't found the right solution, so I hope everyone can help me out.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 25, 2023, 6:16am UTC](https://discuss.elastic.co/t/logstash-how-to-parse-formatted-json-arrays-in-log-files/339498/2 "2023-08-25T06:16:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.