Split filter logstash

Elastic Stack Logstash
1

Hi,
Split filter works fine but i have a problem
here my log looks like

2312-15:44:07:813|V2.5.4|DOW  |WooalsewD6/TTxrff==|ss|ss|0110006|0Succeed|
1110-14:17:40:282|V2.5.1|G|1212|444|||EXCEPTION : System.ServiceModel.ProtocolException: .
   à System.Net.HttpWebRequest.GetResponse()
   à WebService.WebServi(S)NULL|TOTAL:9
2022-01-13 17:16:05   - request in formatHexDump is => 
000000: 01 12 01 04 ....         | 209155000
000016: 37 38 01 ......          | 78           ssa
000032: 30 33 3                  | 03355s05sjha
000048: 30 30       .......      | 00B12
000304: 30 31 30 03              | 010

2022-01-13 17:16:07   - type message: => A
2022-01-13 17:16:07   - code: => 12
2022-01-13 17:16:08   - end communication
2312-15:47:33:043|V2.5.4|AN |6aals4565s60s/s77e==|ss|ss|07556||0Succeed|

here my filebeat.yml

- type: log
  enabled: true
  paths:
    - D:\elastic_stack\LOGS\G_LOGS\*
  multiline.type: pattern
  multiline.pattern: '^\d{4}-\d{2}:\d{2}:\d{2}:\d{3}|V'
  multiline.negate: true
  multiline.match: after
  multiline.max_lines: 20000

here my logstash.conf

filter {
 if([message] =~ /^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}/) {
        grok { match => { "message" => "(?<other format>(^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s{3}-\srequest\sin\sformatHexDump\sis\s\S{2}\s)(\r\n|\r|\n)(.*(\r\n))*(^(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s{3}-\send)[^(\r\n)]*(\r\n)*)+)"    }} 
      }
if ([message] =~ /^\d{4}-\d{2}:\d{2}:\d{2}:\d{3}\|/ and "|" in [message]) {
    mutate {
      split => {"message" => "|"}
      add_field => { "date" => "%{[message][0]}" }
      add_field => { "version" => "%{[message][1]}" }
      add_field => { "action" => "%{[message][2]}" }
      add_field => {"ID" => "%{[message][3]}"}
      }
}

I want to use the split only for logs that start with this format 2312-15:44:07:813 and keep the reset in the message or just add another field ....

NOTE: It works if 2022-01-13 17:16:05 - request in formatHexDump is => ...... was first like :

2022-01-13 17:16:05   - request in formatHexDump is => 
000000: 01 12 01 04 ....         | 209155000
000016: 37 38 01 ......          | 78           ssa
000032: 30 33 3                  | 03355s05sjha
000048: 30 30       .......      | 00B12
000304: 30 31 30 03              | 010

2022-01-13 17:16:07   - type message: => A
2022-01-13 17:16:07   - code: => 12
2022-01-13 17:16:08   - end communication
2312-15:47:33:043|V2.5.4|AN |6aals4565s60s/s77e==|ss|ss|07556||0Succeed|

Any help would be sincerely appreciate!
Thanks.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.