Hi, I am just beginning my story with ELK, trying to understand grok and how to filter events
Here is what I try to accomplish, I try to gather syslogs from routers and switches, here is example
I 05/23/16 11:50:14 ports: port 14 is now off-line
I 05/23/16 11:49:38 ports: port 14 is now on-line
W 05/23/16 11:49:36 ports: port 14 PD Invalid Signature indication.
I 05/23/16 11:49:35 ports: port 14 is Blocked by STP
I 05/23/16 11:49:32 ports: port 14 is now off-line
I 05/23/16 11:49:26 ip: VLAN101: network enabled on 10.101.0.130
I 05/23/16 11:49:26 ip: VLAN101: changing IP address to 10.101.0.130
All I want to do now is to catch string like "is now off-line" and then send emial alert, second part will be using Kibana to go through all logs easily, they can be raw.
Log will be flowing from multiple devices, they will be different types so I just need to catch encountered string, mark it and execute (eg send email)
input {
tcp {
type => "syslog"
port => 514
}
}
input {
udp {
type => "syslog"
port => 514
}
}
filter {
if [type] == "syslog" {
grok { check if the single sting( or strings from dictionary) exist in this message }
if yes, mark it, tag it if not do nothing just send the raw event to elasticsearch}
output {
email {
elasticsearch {
}
from => "myemail@somewhere.com"
match => [
"problem1", "problem2","problem3"
]
subject => "%{tagged_event}"
to => "you@example.com"
via => "mail"
body => "Alert something something: %{@message}"
}
}
filter {
if "is now offline" in [message] {
mutate {
add_tag => ["port-offline"]
}
}
...
}
output {
...
if "port-offline" in [tags] {
email {
...
}
}
}
You'll probably want to have a grok filter first to extract the timestamp in the message field into its own field. You might also want to use a regexp match instead of just using the in operator.
body => "Alert something something: %{@message}"
Unless you're using Logstash 1.1 or something it's message and not @message.
I did small test and it does not seem to work, in case the syslogs comes in with "in now offline" in the message nothing happens which means that it does not work. Without that IF check I got event in the standard output
input {
udp {
type => "syslog"
port => 514
}
}
filter {
if "is now offline" in [message] {
mutate {
add_tag => ["port-offline"]
}
}
}
output {
elasticsearch {
}
if "port-offline" in [tags] {
stdout { }
}
The point of my test is to see if the pattern is found then show that event on stdout { }
Yes, but in order to debug why this doesn't work as expected I want to see what the resulting event looks like.
I don't know how it looks in elastichsearch, I am just using it with kibana so all I see are log going through
All your events are sent to ES, including those that should've been sent to stdout. What do those events look like? Use Kibana's raw view (or whatever it's called) to see the event's JSON representation.
Going deeper with this, how can I compare array of strings or some kind of patterns from external file with eg set of IP.
so instead of
if "is now off-line" in [message] {
....
}
I would need to test array of 30 fixed patterns (stored as a string) form any kind of source against that message field
If I got match next step will be do the same with set of 30 IPs and host field.
if "1.1.1.1" in [host] {
mutate {
add_tag => ["WH", "%{space}"]
}
translate {
field => "locationIP"
destination => "space"
dictionary_path => "/opt/logstash/bin/yaml/spaces_ip.yaml"
add_tag => ["space"]
fallback => "no match"
refresh_interval => "60"
} }
.....
What am I really comparing (entire file?) here against what ? Where is the argument that translate is using for comparison
How to use that value later? Like to compare the match from first column against IP string and second column with name against sth else?
I have noticed that there is no "space" tag in my output (elasticsearch -> kibana) event though all look good
What am I really comparing (entire file?) here against what ? Where is the argument that translate is using for comparison
The translate filter looks up the contents of a field (the field option) against the lookup table and stores the result in another field (the destination option). It looks like you're getting this part.
How to use that value later? Like to compare the match from first column against IP string and second column with name against sth else?
Not sure exactly what you mean here.
I have noticed that there is no "space" tag in my output (elasticsearch -> kibana) event though all look good
I don't know why your example didn't work. It would be easier to debug if you provided the actual event, preferably as presented by a stdout { codec => rubydebug } output.
That is strange for me also, to make it even more strange I noticed that the very first time I run this confing, the translate filter worked properly because I saw extra tag added. No no matter what I do now, extra separate config file, different input files, dictionary, makes this translate filter not working at all. I even tried to install it again, no success.
Ok I am getting closer to what I need here is my filter
filter {
mutate {
add_field => { "location_IP" => "%{host}" }
add_field => { "capture_msg" => "not much here yet" }
}
translate {
field => "location_IP"
destination => "site_name"
dictionary_path => "/opt/logstash/bin/yaml/spaces_ip.yaml"
add_tag => ["%{location_IP}"]
fallback => "no match"
refresh_interval => "60"
}
if [location_IP] == "no match" {
mutate {
add_tag => ["not matched"]
}
} else {
#would like to see if any string (like "is off-line") from predefined set in the file matches any string in the %{message} that I already receive via syslog
# my important_msg.yaml may look like below:
#
# "is now off-line": "is now off-line"
# "TELNET": "TELNET"
#
#now my idea is to create possibly loop that copares array of values from that file against %{message} and try to find it within it
translate {
field => "capture_msg" # somehow assign my_array[0] .... [x]
destination => "msg"
dictionary_path => "/opt/logstash/bin/yaml/important_msg.yaml"
add_tag => ["%{capture_msg}"]
fallback => "not important"
refresh_interval => "60"
}
}
How to approach this to have the easiest possible result here?
Not sure if my approach is correct, If my idea is correct how to load those predefined strings into variable array type and compare in the loop against message?
I figured it out , just used ruby filter and made a loop
There is one thing that bothers me, the official docs are so minimum, there is lack of multiple examples you could digest. It would be great to have more examples than add_tag and filed under basically every section
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.