Logtash Pattern not working

[mbvelo (charles)]

input {
 beats {
         port => 5044
   }    
}
filter {
    grok {
        match => ["mesage", "ts:(?<date>(([0-9]+)-*)+ ([0-9]+-*)+ [A-Z]+)\|INFO:%{{LOGLEVEL:LEVE}\|WebContainer:%{WORD:WebContainer}\|request:%{WORD:request}\|jsessionid:%{WORD:jsessionid}"]
    }
    date {
        match => ["ts","yyyy-MM-dd HH-mm-ss-SSS"]
        target => "@timestamp"
    }

date {
        match => ["date","yyyy-MM-dd HH-mm-ss-SSS z"]
        target => "@timestamp"
    }
  }
	
  output {
    elasticsearch {
    hosts => "localhost:9200"
    #index => "%{[@metadata]}-%{+YYYY.MM.dd}"
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
}
}

[magnusbaeck (Magnus Bäck)]

Is the field that you want to parse with grok really named mesage?

[mbvelo (charles)]

My log snippet
"input_type": "log",
"message": "2017-10-12 16:36:14,716 INFO [WebContainer : 2] x.x.x.x.x.OperationController [OperationController.java:113] processRequest UserInfo request: UserInfo{accessAccount='null', attributes={}, creationDate=Thu Oct 12 16:35:40 SAST 2017, userLocale=null, deviceLocalDateTime=null, deviceOSName='ANDROID', deviceOSVersion='null', appVersion='null', authenticated=false, userNumber='null', localCurrencyCode='x', timeZoneOffSet='0', phoneNumber=null, countryCode='null', activityRefNo='null', sessionId='xxxxxxxxxxx', lastLoginTime='null', deviceId='xxxxxxxxxxx', deviceModelName='null', originatorIPAddress='null', channel='S', ipAddress='xxxxxxxxxxx', localIpAddress='xxxxxxxxxxx', correlationId='null', email='null', iVal='null', jsessionid='xxxxxxxxxxx:xxxxxxxxx', enterpriseSessionId='null', wfpt='null', xfpt='null', nonce='ovLPayS4JzLaB3T6wCzIw-LpLaqJZe1uxm4UsoeVipw', userAgent='null', OpList=[]} ",Preformatted text

[magnusbaeck (Magnus Bäck)]

So your event contains a message field but you've configured your grok filter to parse a mesage field that probably doesn't exist.

[mbvelo (charles)]

@mbvelo

input {
beats {
port => 5044
}
}
filter {
grok {
match => ["mesage", "ts:(?<date>(([0-9]+)-)+ ([0-9]+-)+ [A-Z]+)|INFO:%{{LOGLEVEL:LEVE}|WebContainer:%{WORD:WebContainer}|request:%{WORD:request}|jsessionid:%{WORD:jsessionid}"]
}
date {
match => ["ts","yyyy-MM-dd HH-mm-ss-SSS"]
target => "@timestamp"
}

date {
match => ["date","yyyy-MM-dd HH-mm-ss-SSS z"]
target => "@timestamp"
}
}

output {
elasticsearch {
hosts => "localhost:9200"
#index => "%{[@metadata]}-%{+YYYY.MM.dd}"
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}

i updated the Grok filter 
I am getting this response

herewith my log - please help getting a pattern

2017 - 10 - 20 08: 53: 39, 785[WebContainer: 1] ACTIVITY OUT: xxxxxx ErrorCode = 00000 Field = 00000 Exception = null JSON = {
"field1": "xxxxxx",
"field2": "00000",
"field3": "value",
"field4": "value",
"Parent": [{
"child1": "value",
"child2": "value",
"child3": "value",
"Parent": {
"child1": "value",
"child2": "value"
},
"Parent1": {
"child1": "value",
"child2": "value"
},
"field5": "value",
"field6": "value",
"field7": "value",
"field8": "value",
"field9": "value",
"field10": "value",
"Parent2": {
"child1": "Value",
"child2": "Value"
}
}, {
"child1": "value",
"child2": "value",
"child3": "value",
"Parent": {
"child1": "value",
"child2": "value"
},
"Parent1": {
"child1": "value",
"child2": "value"
},
"field5": "value",
"field6": "value",
"field7": "value",
"field8": "value",
"field9": "value",
"field10": "value",
"Parent3": {
"child1": "value",
"child2": "value"
}
}, {
"child1": "value",
"child2": "value",
"child3": "value",
"Parent": {
"child1": "value",
"child2": "value"
},
"Parent1": {
"child1": "value",
"child2": "value"
},
"field5": "value,
"field5": "value",
"field5": "value",
"field5": "value",
"field5": "value",
"field5": "value",
"Parent4": {
"child1": "value",
"child2": "value"
}
}

[magnusbaeck (Magnus Bäck)]

match => ["mesage", "ts:(?(([0-9]+)-)+ ([0-9]+-)+ ...

No! Your configuration still says mesage instead of message. Over and out.

[mbvelo (charles)]

@mbvelo

input {

beats {

port => 5044

}

}

filter {

grok {

match => ["mesage", "ts:(?<date>(([0-9]+)-)+ ([0-9]+-)+ [A-Z]+)|INFO:%{{LOGLEVEL:LEVE}|WebContainer:%{WORD:WebContainer}|request:%{WORD:request}|jsessionid:%{WORD:jsessionid}"]

}

date {

match => ["ts","yyyy-MM-dd HH-mm-ss-SSS"]

target => "@timestamp"

}

date {

match => ["date","yyyy-MM-dd HH-mm-ss-SSS z"]

target => "@timestamp"

}

}

output {

elasticsearch {

hosts => "localhost:9200"

#index => "%{[@metadata]}-%{+YYYY.MM.dd}"

index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"

document_type => "%{[@metadata][type]}"

}

}

rectified - uploaded the pattern

geytting this errror

	"tags": [
		"beats_input_codec_plain_applied",
		"_grokparsefailure",
		"_geoip_lookup_failure"

[mbvelo (charles)]

 input {
 	beats {
 		port => 5044
 	}
 }
 filter {
 	grok {
 		match => ["message", "ts:(?<date>(([0-9]+)-)+ ([0-9]+-)+ [A-Z]+)|INFO:%{{LOGLEVEL:LEVE}|WebContainer:%{WORD:WebContainer}|request:%{WORD:request}|response:%{WORD:response}||jsessionid:%{WORD:jsessionid}"]
 	}
 	date {
 		match => ["ts", "yyyy-MM-dd HH-mm-ss-SSS"]
 		target => "@timestamp"
 	}

 	date {
 		match => ["date", "yyyy-MM-dd HH-mm-ss-SSS z"]
 		target => "@timestamp"
 	}
 }

 output {
 	elasticsearch {
 		hosts => "localhost:9200"
 		index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
 		document_type => "%{[@metadata][type]}"
 	}
 }