Filtered Logstash for desired part

[Zerra_Triani]

0

I have a log, and I only want to retrieve the ones I have marked in red boxes. I've tried putting it in Elasticsearch but it inserts it every row in that log. How do you take the data that I have marked and combine it into a single row?

The following is an example of the log

Task with ID = 119 is waiting for the message to arrive on the queue 1294598175.
=>bpc_wait_for_event (comm_utils.c)
=>bpc_select (pipe_utils.c)
Using select() to wait for an event to occur at 10:00:02 
Event selector just had 1 fd(s) triggered at 10:00:02 
=>process_incoming_data (tcp_main.c)
=>reset_device_idle_timers (tcp_main.c)
=>receive_incoming_message (tcp_xfer.c)
=>receive_ncr_message (tcp_xfer.c)
=>tcp_receive_data (tcp_xfer.c)
=>bpc_sock_recv (comm_utils.c)
=>tcp_receive_data (tcp_xfer.c)
=>bpc_sock_recv (comm_utils.c)
30.32.31.30.46.33.33.38.34.30.30.39.38.41.38.30       0210F33840098A80
38.30.30.32.30.30.30.30.30.30.30.30.30.32.30.30       8002000000000200
30.30.30.36.31.36.36.30.33.34.39.34.38.38.32.30       0006166034948820
32.32.36.32.31.32.33.39.31.30.30.30.30.30.30.30       2262123910000000
37.37.39.35.37.36.30.30.30.32.32.34.30.39.35.39       7795760002240959
35.35.30.30.30.30.30.30.30.30.34.31.36.39.35.34       5500000000416954
30.39.35.39.35.35.30.32.32.34.36.30.31.31.33.39       0959550224601139
36.31.31.31.31.31.31.31.30.30.30.30.30.30.30.30       6111111100000000
30.30.38.30.36.34.35.30.30.30.35.30.30.30.34.38       0080645000500048
31.34.31.36.39.35.34.35.33.53.31.41.57.31.35.52       141695453S1AW15R
4E.33.36.30.30.30.34.30.39.30.30.31.33.31.36.35       N360004090013165
30.30.38.38.32.32.30.30.30.30.30.30.33.34.35.31       0088220000003451
30.30.33.34.35.31                                     003451          
Received 198 bytes of data from device #600, sending to CROUTer
=>raw_msgx2sv_msg (tag_utils.c)

    ************************ Header Fields *************************
    orgdev:     600             utrnno:     0
    orgid:      0               reversal:   0
    destid:     0               repeat:     0
    last_task_id:   119             fintran:    0
    msgtype:    51              phase:      0
    task_msgtype:   0               balances:   0
    respreq:    0               stood_in_for:   0
    resp_qid:   -1              issuer_posted:  0
    smsgno:     0               sv_trace:   0
    nwindicator:    0               timestamp:  0
    devinfo:    []
    hpan:       []
    fld_flags:  00000000000000000000
    sys_msg_no: 0

msgsnd_w_retry [dst task: COMMSINT, time: 24/02/2020 10:00:02.0358]: Msg sent to queue 1293844488

[Rios]

You can use ruby for regex.

      ruby {
	  code =>  "  event.set('msgfilter', event.get('message').scan(/\s{7}[a-zA-Z0-9]+(?=\r\n|\s*\r\nReceived)/) ) "
	  id => "ruby-counter"
     }
       # merge array of strings
       mutate { join => { "msgfilter" => "" }  }
	   # remove spaces
	   mutate { gsub => [ "msgfilter", " " , "" ] }

Result:

"msgfilter" => "0210F33840098A8080020000000002000006166034948820226212391000000077957600022409595500000000416954095955022460113961111111000000000080645000500048141695453S1AW15RN3600040900131650088220000003451003451"

[Zerra_Triani]

Before that, I'm sorry because I just learning to use elastic. I've tried the ruby filter you provided but the result is still the same. Did I write it wrong? Thank you

Here the logstash

input {
    beats {
        port => "5044"
    }
}
filter {
	ruby {
	  code =>  "  event.set('msgfilter', event.get('message').scan(/\s{7}[a-zA-Z0-9]+(?=\r\n|\s*\r\nReceived)/) ) "
	  id => "ruby-counter"
     }
       # merge array of strings
       mutate { join => { "msgfilter" => "" }  }
	   # remove spaces
	   mutate { gsub => [ "msgfilter", " " , "" ] }
}
output {
    elasticsearch {
        hosts => [ "localhost:9200" ]
    }
}