Hi, I'm new to using the elk so the question may be easy / make no sense. One pattern appears 3 times with different numbers inside my log file. I can use the grok filter with the match config to find the pattern at all three instances. But I want the field to be called a different name every time this pattern is found so I can make 3 different visuals on kibana. For example I tried something like:
grok {
break_on_match => false
match => { "message" => "%{NUMBER:blah:int}"}
match => { "message" => "%{NUMBER:blah2:int}"}
match => { "message" => "%{NUMBER:blah3:int}"}
}
Thinking it would find the first instance -> call it blah, find second instance -> call it blah2, then find third instance -> call it blah3. But this doesn't work. Is there any way to do such a thing?
It's not quite clear from your question is these three occurrences appear in the same line or in different lines. Please show an example from your log file that exhibits the data you want to extract.
I still don't have a way to do it but I can use Kibana to add the sums of matches for that pattern which is good enough for me. I am using cucumber testing (u don't need to know what that is) on three components so the output, which is in one log file, is given the same way 3 different times. I can't really break the match up by components because of the way it is ran so I need a way to separate each output by giving it a different name.
I wanted to do something that says first time you find this pattern call it this: "%{NUMBER:firstTotal} scenarios ( %{NUMBER:firstFailed} failed, %{NUMBER:firstSkipped} skipped)".
Second time call it: this: "%{NUMBER:secondTotal} scenarios ( %{NUMBER:secondFailed} failed, %{NUMBER:secondSkipped} skipped.
Etc..
This way I can have separate fields for the 3 instances.
I have the same problem.
I am reading multiline from input and trying to match all the instances of the pattern. but it only output the first instance and not the rest. I'm not sure where I am making the mistake. this is my conf file content:
input {
file {
# change the path in your local to make it work
path => "/usr/local/src/logstash/log1.in"
start_position => beginning
sincedb_path => "/dev/null"
codec => multiline {
pattern => "^show"
negate => true
what => next
}
}
}
filter {
grok {
# pay attention to the path
patterns_dir => ["./patterns/mypatterns"]
match => {"message" => ["(?m)configure vlan %{WORD:vlan} add ports %{PORTS:ports} %{TAG_INFO:tag_info}"]}
break_on_match => false
}
}
output {
stdout {
codec => rubydebug
}
}
and this is the content of log1.in
configure snmp sysName "NY_MPBN_SS_1"
configure snmp sysLocation "GSI New York"
configure snmp sysContact "JPoserio@globecommsystems.com"
configure timezone name New_Yor -240
configure sys-recovery-level switch reset
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
