# Multiline codec help for SYSLOG files

**URL:** <https://discuss.elastic.co/t/multiline-codec-help-for-syslog-files/88844>\
**Category:** Logstash\
**Created:** [June 9, 2017, 1:07pm UTC](https://discuss.elastic.co/t/multiline-codec-help-for-syslog-files/88844 "2017-06-09T13:07:34Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![rgacek](https://avatars.discourse-cdn.com/v4/letter/r/c6cbf5/32.png) [@rgacek](https://discuss.elastic.co/u/rgacek)\
**Post date:** [June 9, 2017, 1:07pm UTC](https://discuss.elastic.co/t/multiline-codec-help-for-syslog-files/88844/1 "2017-06-09T13:07:34Z")

</div>

Hello,

I am trying to combine the following multiline output into single event using Logstash:

01,03,APR 10 16:25:45:453530,timer,ERROR ,"GFI Timer Error ==\>"  
01,03,APR 10 16:25:45:453530,timer,ERROR ,"No 14: Date (MON 04/10/17 16:25:45:453015 ), Error Type (CANCEL\_INVALID\_BLCOK), cpu(1)"  
01,03,APR 10 16:25:45:453530,timer,ERROR ," modid(module(0)), flags(0x00000000), type(1), exp\_time(318070975), timeout(16744),"  
01,03,APR 10 16:25:45:453530,timer,ERROR ," data1(7c83), data2(1afda9c8), func(10090f04), create\_ra(0x1008e954), tm\_cancel\_ra(0x1008ffbc)"  
01,03,APR 10 16:25:45:453530,timer,ERROR ," bt: 1003c1e8 \<- 1008f840 \<- 1008ffdc \<- 1009055c \<- 10091cb8 \<- 101ff258"  
01,08,APR 10 16:25:45:452804,timer,ERROR ,"GFI Timer Error ==\>"  
01,08,APR 10 16:25:45:452804,timer,ERROR ,"No 09: Date (MON 04/10/17 16:25:45:452233 ), Error Type (CANCEL\_INVALID\_BLCOK), cpu(1)"  
01,08,APR 10 16:25:45:452804,timer,ERROR ," modid(module(0)), flags(0x00000000), type(1), exp\_time(324860186), timeout(16744),"  
01,08,APR 10 16:25:45:452804,timer,ERROR ," data1(7c83), data2(1afdade8), func(10090f04), create\_ra(0x1008e954), tm\_cancel\_ra(0x1008ffbc)"  
01,08,APR 10 16:25:45:452804,timer,ERROR ," bt: 1003c1e8 \<- 1008f840 \<- 1008ffdc \<- 1009055c \<- 10090688 \<- 101eabd4"  
01,10,APR 10 16:25:49:527170,DP\_CHECK,ERROR ,"tq\_check\_array\_index\_validity FAILED (23 same log suppressed)"

The pattern for the SYSLOG would be the following, or at least the timestamp  
01,03,APR 10 16:25:45:453530  
chassis, slot, timestamp

or just timestamp with milliseconds...  
APR 10 16:25:45:453530

My initial idea for pattern match was to remove the 6 digit milliseconds, but this will end up giving me same timestamp for multiple events.  
Another idea is to use [chassis,slot,timestamp}, but so far no luck...

I have tried the following:  
filter {  
grok {  
pattern =\> ["%{DATESTAMP:LOGTIME}"]  
}  
date {  
match =\> ["LOGTIME", "MMM dd HH:mm:ss:SSSSSS"]  
timezone =\> "UTC"  
remove\_field =\> ["LOGTIME"]  
}  
multiline {  
pattern =\> "(^%{NUMBER:chassis},%{NUMBER:slot},%{WORD:MONTH} %{NUMBER:DAY} %{TIME:LOGTIME})"  
negate =\> true  
what =\> "previous"  
}

grok {  
match =\> { "message" =\> "%{NUMBER:chassis},%{NUMBER:slot},%{WORD:MONTH} %{NUMBER:DAY} %{TIME},%{WORD:UNIT\_REPORTING},%{WORD:LOGLEVEL} ,%{GREEDYDATA:syslog\_message}"}  
}

and I have tried the following through CSV filter, per something I found on the web.. no luck so far:

filter {

# csv plugin takes the input lines from the file, and breaks down into defined columns below

```
csv {
   columns =>
["Chassis",
 "Slot_ID",
 "LOGTIME",
 "reporting_type",
 "event_level",
 "log_message"]

separator => "," 

```

# remove\_field =\> ["message"]

```
    }

```

# remove all 6 digits after seconds to meet time standard

```
    mutate {
       gsub => ["LOGTIME", ":\d{6}", ""]
	   add_field => { "LOG_TYPE" => "SYSLOG" }
           }

    mutate {
       convert => {
'Chassis' => 'integer'
'Slot ID' => 'integer'
               }
          }

```

date {  
match =\> ["LOGTIME", "MMM dd HH:mm:ss"]  
timezone =\> "UTC"

# remove\_field =\> ["LOGTIME"]

}

Can someone guide me and let me know the mistakes I am making?

Thank you

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 7, 2017, 1:07pm UTC](https://discuss.elastic.co/t/multiline-codec-help-for-syslog-files/88844/2 "2017-07-07T13:07:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.