# Comma destroys perfect events | file { codec =\> multiline { pattern

**URL:** <https://discuss.elastic.co/t/comma-destroys-perfect-events-file-codec-multiline-pattern/152780>\
**Category:** Logstash\
**Created:** [October 17, 2018, 8:24am UTC](https://discuss.elastic.co/t/comma-destroys-perfect-events-file-codec-multiline-pattern/152780 "2018-10-17T08:24:34Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![humalog](https://avatars.discourse-cdn.com/v4/letter/h/cc9497/32.png) [@humalog](https://discuss.elastic.co/u/humalog)\
**Post date:** [October 17, 2018, 8:24am UTC](https://discuss.elastic.co/t/comma-destroys-perfect-events-file-codec-multiline-pattern/152780/1 "2018-10-17T08:24:34Z")

</div>

I want to index a json file. I use input file, with codec multiline. My filter is json filter, output is elasticsearch.  
The structure of my json file looks like this:

```auto
[
 { 
   "Plugin": 92438,
   "Plugin Name": "WordPad History",
   "Family": "Windows",
   "Severity": "Info",
   "Total": 1
 },
 {
   "Plugin": 92439,
   "Plugin Name": "Explorer Search History",
   "Family": "Windows",
   "Severity": "Info",
   "Total": 1
 },
 {
   "Plugin": 53360,
   "Plugin Name": "SSL Server Accepts Weak Diffie-Hellman Keys",
   "Family": "General",
   "Severity": "Info",
   "Total": 1
 }
]

```

My logstash.conf:

```auto
input {
  file {
    path => "/home/vagrant/vuln.json"
    start_position => "beginning"
    sincedb_path => "/dev/null"
    codec => multiline {
      pattern => "{"    
      negate => "true"
      what => "previous"
      auto_flush_interval => 1
    }
  }
}

filter {
  mutate {
    gsub => ["message", ",$",""]
   }
   json {
      source => "message"
    }
}

output {
  elasticsearch {
    index => "new-index"
    hosts => "127.0.0.1"
  }
}

```

My output is nearly perfect but just nearly. I dont understand the json filter....I get 3 events. First 2 events are different to the 3rd. The first 2 events write into the message field for example:  
message:

```auto
 { 
   "Plugin": 92439, 
   "Plugin Name": "Explorer Search History", 
   "Family": "Windows", 
   "Severity": "Info", 
   "Total": 1 
 }, 

```

The last event has no comma at the end of the message and it indexes perfectly. It automatically makes new field, for example Plugin: 53360, Plugin Name: "SSL Server Accepts Weak Diffie-Hellman Keys", Severity: "Info". This is a valid json file, why can't it build new field also if i there are commas behind the message? I get \_jsonparsefailure in the first 2 events, where no new fields are built, in the last event I don't get \_jsonparsefailure...

message :

```auto
 {
   "Plugin": 53360,
   "Plugin Name": "SSL Server Accepts Weak Diffie-Hellman Keys",
   "Family": "General",
   "Severity": "Info",
   "Total": 1
 }

```

What i need to do, to get new fields if there are commas at the end of the message? Is it possible  
mutate { gsub 0\> and json { } don't work together?

Best regards

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 14, 2018, 9:03am UTC](https://discuss.elastic.co/t/comma-destroys-perfect-events-file-codec-multiline-pattern/152780/3 "2018-11-14T09:03:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.