# Help Filter JSON file

**URL:** <https://discuss.elastic.co/t/help-filter-json-file/88300>\
**Category:** Logstash\
**Created:** [June 5, 2017, 4:35pm UTC](https://discuss.elastic.co/t/help-filter-json-file/88300 "2017-06-05T16:35:41Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rafael\_Pereira\_Silva](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rafael_pereira_silva/32/68703_2.png) [@Rafael\_Pereira\_Silva](https://discuss.elastic.co/u/Rafael_Pereira_Silva)\
**Post date:** [June 5, 2017, 4:35pm UTC](https://discuss.elastic.co/t/help-filter-json-file/88300/1 "2017-06-05T16:35:41Z")

</div>

Hi, I have a record that is generated by an application, this log comes as JSON.  
But the log is coming broken.

Follow Log:

(2017-06-05 11:53:22) {  
"\_\_v": 0,  
"\_id": "Joq3PHwo8AHiUueiZ73VXpavSo84azUg30tydFVWpQTJcvMD4ZhLLDyg9rJKFVTHOChTk4giWxBVU1V6jxTRbzJBB57J1Y4UYxO6",  
"request": {  
"payload": {  
"addressId": "2354235235346"  
},  
"path": "/x2/asd/s",  
"method": "post",  
"headers": {  
"content-length": "29",  
"connection": "Keep-Alive",  
"x-forwarded-server": "teste.local.local",  
"x-forwarded-host": "Teste.local.local",  
"incap-client-ip": "189.121.52.130",  
"x-forwarded-for": "181.15.58.30, 19.57.140.8",  
"incap-proxy-684": "OK",  
"user-agent": "okhttp/3.6.0",  
"accept-encoding": "gzip",  
"content-type": "application/json; charset=UTF-8",  
"origin": "CCI",  
"agent": "Android;6.0;XT1097;motorola;0.27.0\_homolog.CCI",  
"authorization": "authorizat234ionnasdaspdkasx asda, blu me ",  
"host": "meuhost.teste.teste"  
}  
},  
"statusCode": 500,  
"code": 0,  
"details": "(timers.js:596:5)\n\n{"isBoom":true,"isServer":true,"data":null,"output":{"statusCode":500,"payload":{"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred","code":0},"headers":{}}}",  
"level": "ERROR",  
"\_created\_at": "2017-06-05T14:53:22.840Z"

Important is the value : "message, level, details and hour"

This is my filter:

```
if [type] == "mobile-prod" {
      multiline {
          pattern => '^\s'
          what => "next"
          }
        }

```

can you help me with filter?

thanks.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 8, 2017, 5:07am UTC](https://discuss.elastic.co/t/help-filter-json-file/88300/2 "2017-06-08T05:07:37Z")

</div>

Don't use the multiline filter, use a multiline codec. The following configuration probably works:

```
pattern => "^\(%{TIMESTAMP_ISO8601}\) \{$"
what => "previous"
negate => true

```

In other words, unless the line looks like the marker of a new logical event, join it with the preceding line.

---

<div class="post-metadata">

**Author:** ![Rafael\_Pereira\_Silva](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rafael_pereira_silva/32/68703_2.png) [@Rafael\_Pereira\_Silva](https://discuss.elastic.co/u/Rafael_Pereira_Silva)\
**Post date:** [June 8, 2017, 3:03pm UTC](https://discuss.elastic.co/t/help-filter-json-file/88300/3 "2017-06-08T15:03:21Z")

</div>

I use this configuration in session {input or filter ?

Follow my file /logstash/config/logstash.conf

```
input {
    udp {                
        host => "0.0.0.0"
        port => 514     
        tags => "syslog"
    }                          
                                                                    
    lumberjack {                                                    
        port => 5043                                     
        ssl_certificate => "/logstash/config/logstash-forwarder.crt"
        ssl_key => "/logstash/config/logstash-forwarder.key"
        tags => "lumberjack"
    }   
}                                                                                      
                                                                                       
filter {                                                                               
      if [type] == "mobile-prod" { #this type of logstash forwarder conf in the client
          codec => multiline {                                        
              pattern => "^\(%{TIMESTAMP_ISO8601}\) \{$"              
              what => "previous"                                      
              negate => true                                          
              }                                                       
            }                                                         
output {                                                              
    elasticsearch {                                                   
        cluster => "c4logs"                                          
        protocol => "http"                                            
        host => "10.254.0.100"                                    
        port => "9200"                                            
        index => "logstash-%{+YYYY.MM.dd}"                         
    }                                                                 
}  
}

```

It is corret?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 9, 2017, 5:22am UTC](https://discuss.elastic.co/t/help-filter-json-file/88300/4 "2017-06-09T05:22:14Z")

</div>

For reliable operations you need to put the multiline processing closer to the source, i.e. in Filebeat or whatever is the origin of the logs. Using a multiline filter in this way is just a bad idea.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 7, 2017, 5:22am UTC](https://discuss.elastic.co/t/help-filter-json-file/88300/5 "2017-07-07T05:22:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.