"Extracting" Data from a Message

Elastic Stack Logstash
1

So I am coming from Splunk, currently using Graylog, and trying to set up the ELK Stack to analyze power usage. I have my Energy Monitor sending messages to the server properly. Here are a couple lines for an example:

Vera-35030422 EventWatcher: M [022] Home Energy Monitor, Watts = 1001.520
Vera-35030422 EventWatcher: M [022] Home Energy Monitor, Watts = 1002.240
Vera-35030422 EventWatcher: M [022] Home Energy Monitor, Watts = 764.040
Vera-35030422 EventWatcher: M [022] Home Energy Monitor, Watts = 5115.480
Vera-35030422 EventWatcher: M [022] Home Energy Monitor, Watts = 894.720

I am pulling my hair out trying to "extract" a few things from each message. It was relatively easy using regular expressions and capture groups with the other products I used.

Basically, I need to be able to pull (make fields) from:
-DeviceID 022 (variable)
-DeviceName "Home Energy Monitor" (there are two more so it's variable, ie. HEM, HEM2, HEM3).
-Watts ... just the float number at the end.

I need to convert the Watts to a float value rather than a string so that it can be graphed.

Anyone have any hints?

2

The grok filter is the Logstash plugin that provides regular expressions and capture groups. Something like this should work for you:

filter {
  grok {
    match => [
      "message",
      "%{NOTSPACE} %{NOTSPACE}: M \[%{INT:DeviceID:int}\] %{GREEDYDATA:DeviceName}, Watts = %{NUMBER:Watts:float}"
    ]
  }
}
1 Like
3

Thank You!! I guess I am just still trying to get my head around grok and patterns, that helps A LOT!

4

Just wanted to follow up that this worked great!