How to set start and end point of a prebuild grok pattern

Elastic Stack Logstash
1

My grok patter broken when there is a |(pipe) in my data .
here is my log

> 16:29:52.143 [kafka-producer-network-thread | producer-1] INFO c.h.h.d.e.ApiMessageProducer - ==========================================================================================================

and Here is My grok pattern

%{TIME:timestamp}\s%{DATA:thread}\s+%{DATA:log_level}\s+%{DATA:classs_path}\s+\-

when there is a pipe in my thread, pattern is broken and log level is become "|" rest of my pattern is miss printed so I want to escape |(pipe) in thread if there is one.

Also if there is a empty character in thread part also same thing happens, so how can I say that thread starts with [ and ends with ] characters

2

Think you are looking for this.

%{TIME:timestamp}\s\[%{DATA:thread}\]\s+%{DATA:log_level}\s+%{DATA:classs_path}\s+\-

Which gives you this.

{
  "classs_path": "c.h.h.d.e.ApiMessageProducer",
  "log_level": "INFO",
  "thread": "kafka-producer-network-thread | producer-1",
  "timestamp": "16:29:52.143"
}
3

I tried exact same and it was giving errors now it is working :).
that solves my problem thanks.
In any case how can I escape pipe character in this %{DATA:thread}
assume pipe is in the middle of text and I want to escape it.

4

What do you mean by escape it? Using a \ before a special character will escape it but not sure we are talking about the same thing because that character is in a field.

5

can we do something like if exist delete not continue

6

Still not sure I understand completely. If you are looking to drop the field thread if it contains a | then I would do the below.

input {
  generator {
      lines => [ '{"thread" :"kafka-producer-network-thread | producer-1"}' ]
      codec => json
      count => 1
  }
}
filter {
  if "|" in [thread] {
    mutate { remove_field => "thread" }
  }
}
output {
  stdout { codec => json_lines }
}
7

thank you it is not that much important,

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.