If make same head Multiline codec


(James Pang) #1

I would like to make if beginning of 63 chars the same to be same document. For example if following 8 lines start with
"2018/11/14-15:14:14.518292-09855-09855-432638034320556033-004-"
is one doc, if logstash find a new line is not start with that, create a new document.

Thanks

=======================================================================
2018/11/14-15:14:14.518292-09855-09855-432638034320556033-004- Sapi
2018/11/14-15:14:14.518292-09855-09855-432638034320556033-004- **********************************************************************
2018/11/14-15:14:14.518292-09855-09855-432638034320556033-004- SAPI: internal message <332> processing begin ...
2018/11/14-15:14:14.518292-09855-09855-432638034320556033-004- ====================
2018/11/14-15:14:14.518292-09855-09855-432638034320556033-004- 0x7f733f8bc530
2018/11/14-15:14:14.518292-09855-09855-432638034320556033-004- --------------------------------------------
2018/11/14-15:14:14.518292-09855-09855-432638034320556033-004-
2018/11/14-15:14:14.518292-09855-09855-432638034320556033-004- ====================
2018/11/14-15:14:14.518298-09855-09855-432638034320556033-004- Sapi PPI set as:
2018/11/14-15:14:14.518311-09855-09855-432638034320556035-004- Reading cache entry for ctx_id : d900000a0f380002
2018/11/14-15:14:14.518319-09855-09855-432638034320556035-004- Pers-Mgr PersistentObject constructed as: 0x7f618656dac0


Logstash - aggregate results
#2

Those 8 lines do not start with the same 63 characters. The last 3 have different time stamps. That said, if we break off the timestamp and then use the rest of the 63 you can do it using aggregate.

grok { match => { "message" => [ "^(?<time>.{26})(?<taskId>.{37})" ] } }
aggregate {
    task_id => "%{taskId}"
    code => "map['message'] ||= ''; map['message'] += event.get('message') + '
';"
    push_map_as_event_on_timeout => true
    timeout_task_id_field => "taskId"
    timeout => 2 # 2s timeout
    timeout_code => "event.set('[@metadata][wanted]', 1)"
}
if [@metadata][wanted] != 1 { drop {} }

(system) closed #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.