Simple Split filter

Elastic Stack Logstash
1

Hello! Dear collegues, need your help!
I have the input log

09:00:00.023|SMSG|***|ROUTING begin
FLD_029 **
VALIDITY_FLAG T
TIME_STAT |POS_ISO

09:00:00.024|DEBG|***|proc_new_msg()
FLD_029 **
VALIDITY_FLAG F
TIME_STAT |POS_ISO_IN_E-21600.06876

09:00:00.070|DEBG|1***|acq()
FLD_029 ***
VALIDITY F
TIME_STAT |POS_ISO_-21600.05620|

Every paragraph i want write to the one event, well i see that every paragraph separate "\n\n"

I right the folowing conf:

beats {

port => 5055
host => "**.**.**.**"
tags => ["gateway_ssl"]
}

filter {

if "gateway_ssl" in [tags] {

    mutate {"message", \n, " ; "}
split {
terminator => "\n\n"
add_tag => ["mutated"]	
}
 }

}

if "gateway_ssl" in [tags] {
elasticsearch {
hosts => "localhost:9200"
index => "gateway_ssl-%{+YYYY.MM.dd}"
}}

What am i do wrong? Could you help me correct my config?
Thank you a lot!

2

Are you consuming the entire file as a single event?

3

Hi, Badger! Glad to hear you)
No, not all file as single event. Just paragraph betwee two \n\n

Exm:
1 event:
09:00:00.023|SMSG|***|ROUTING begin
FLD_029 **
VALIDITY_FLAG T
TIME_STAT |POS_ISO

2 event:
09:00:00.024|DEBG|***|proc_new_msg()
FLD_029 **
VALIDITY_FLAG F
TIME_STAT |POS_ISO_IN_E-21600.06876

3 event:
09:00:00.070|DEBG|1***|acq()
FLD_029 ***
VALIDITY F
TIME_STAT |POS_ISO_-21600.05620|

After i will to need grok every event, i have prepered config

grok {
match => {
"message" => [
"%{TIME:time_transaction}|%{DATA:INFO}|%{NUMBER:number}|%{GREEDYDATA:type_transaction}",
"FLD_029: %{NUMBER:FLD_029_value}",
"VALIDITY_FLAG: %{WORD:VALIDITY_FLAG_value}",
"LOCALITY_FLAG: %{WORD:LOCALITY_FLAG_value}",
"RSW_CAPTURE_FLAG: %{WORD:RSW_CAPTURE_FLAG_value}"]}

   add_tag => [ "grokked" ]}
4

If you completely remove the filter {} section and use stdout { codec => rubydebug } as the output then what does a single event look like?

5

Well, i tried to delete all filter configuration.
And added stdout rubydebuger, now i see that every line its the single event. Its not i need((
But every event have the tag: beats_input_codec_plain_applied
Is it Ok?
Maybe i have another solutions to this task?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.