Video log processing logstash

[rach]

hi
can you please help me process the following video log?
I will have multiple dynamic fields (which not appear all the time), and this is my log:

02/04 10:07:59.216,DEBUG,10024,JDM,"Received: FR_JOB_RESULT msg, from service type: recognition, to service type: manager, class: Fre from 127.0.0.1:29923(recognition) to 127.0.0.1:37469(manager) for FR job (id #32352 session #21303) ,result type: Success ihd=224, fics=[3], current td=-1, eest=-1.0, proceime=604, result:[], QRs:[]",com.1.jdm.Jdm(495),Jdm

thank you!

[magnusbaeck (Magnus Bäck)]

It's a bit hard to help when you're not telling us which parts are dynamic, but I'm guessing everything up to and including "type: Success" is more or less static. Use a grok filter to extract fields from there and put the rest of the string in a field that you process with a kv filter.

[rach]

thanks for the answer.
my problem is that i don't know how to address the " (beginning in the Received and ending with []) so that it will be part of my message...
i tried this and it didn't work...:
input {
beats {
port => "5043"
}
}

filter {
grok {
match => { "message" =>"%{MONTHNUM:month}/%{MONTHDAY:day} %{TIME:time},%{WORD:level},%{WORD:proccess_id},%{WORD:component_type},%{GREEDYDATA:msg}" }
}
kv {
source => "msg"
value_split => ":"
}
mutate {
gsub => [
"msg", "[\"]", " "
]
}
}

output {
elasticsearch {
hosts => "localhost:9200"
index => "a-%{+YYYY.MM.dd}"
document_type => "try"
}
}