How do we convert from a hash to json before event reaches the grok filter

Elastic Stack Logstash
1

How do we convert from a hash to json before event reaches the grok filter. The JMX input plugin produces a Hash, but the Grok filter expects JSON.

input {
jmx {
path => "c:/LogStash/jmxconf/"
polling_frequency => 15
type => "jmx"
nb_thread => 4
codec => json
}

}

filter {
grok {
patterns_dir => "C:/LogStash/bak/patterns"
match => { "event" => "Memory.HeapMemoryUsage.committed", "metric_value_number"=>%{NUMBER:committed}" }
}
}

output {
stdout { codec => rubydebug }
}

many thanks,

Tony

2

It looks like you're trying to parse JSON with your grok filter. Use a json filter instead. Although I don't get why that's necessary since the jmx input has codec => json so it should already be decoding the JSON payload. Could you disable your grok filter and just show us the output from the stdout output so that we can see exactly what we're dealing with?

3

Thanks for looking into this. After looking at the JMX plugin code, it doesn't seem like the code does anything with "codec => json" setting. On a side note the output stdout code does. I disabled the Grok filter. and the output looks like this:

{
"@version" => "1",
"@timestamp" => "2015-09-18T16:32:48.191Z",
"host" => "usbgc-lab-vd1.ny.cantor.com",
"path" => "c:/LogStash/jmxconf/",
"type" => "jmx",
"metric_path" => "usbgc-lab-vd1.ny.cantor.com_9119.Memory.Verbose_bo
ol",
"metric_value_number" => 0
}
{
"@version" => "1",
"@timestamp" => "2015-09-18T16:32:48.228Z",
"host" => "usbgc-lab-vd1.ny.cantor.com",
"path" => "c:/LogStash/jmxconf/",
"type" => "jmx",
"metric_path" => "usbgc-lab-vd1.ny.cantor.com_9119.Memory.HeapMemory
Usage.committed",
"metric_value_number" => 192937984
}
{
"@version" => "1",
"@timestamp" => "2015-09-18T16:32:48.243Z",
"host" => "usbgc-lab-vd1.ny.cantor.com",
"path" => "c:/LogStash/jmxconf/",
"type" => "jmx",
"metric_path" => "usbgc-lab-vd1.ny.cantor.com_9119.Memory.HeapMemory
Usage.init",
"metric_value_number" => 162068032
}
{
"@version" => "1",
"@timestamp" => "2015-09-18T16:32:48.251Z",
"host" => "usbgc-lab-vd1.ny.cantor.com",
"path" => "c:/LogStash/jmxconf/",
"type" => "jmx",
"metric_path" => "usbgc-lab-vd1.ny.cantor.com_9119.Memory.HeapMemory
Usage.max",
"metric_value_number" => 7635730432
}
{
"@version" => "1",
"@timestamp" => "2015-09-18T16:32:48.258Z",
"host" => "usbgc-lab-vd1.ny.cantor.com",
"path" => "c:/LogStash/jmxconf/",
"type" => "jmx",
"metric_path" => "usbgc-lab-vd1.ny.cantor.com_9119.Memory.HeapMemory
Usage.used",
"metric_value_number" => 141016808
}
{
"@version" => "1",
"@timestamp" => "2015-09-18T16:32:48.273Z",
"host" => "usbgc-lab-vd1.ny.cantor.com",
"path" => "c:/LogStash/jmxconf/",
"type" => "jmx",
"metric_path" => "usbgc-lab-vd1.ny.cantor.com_9119.Memory.NonHeapMem
oryUsage.committed",
"metric_value_number" => 109641728
}
{
"@version" => "1",
"@timestamp" => "2015-09-18T16:32:48.280Z",
"host" => "usbgc-lab-vd1.ny.cantor.com",
"path" => "c:/LogStash/jmxconf/",
"type" => "jmx",
"metric_path" => "usbgc-lab-vd1.ny.cantor.com_9119.Memory.NonHeapMem
oryUsage.init",
"metric_value_number" => 24576000
}
{
"@version" => "1",
"@timestamp" => "2015-09-18T16:32:48.287Z",
"host" => "usbgc-lab-vd1.ny.cantor.com",
"path" => "c:/LogStash/jmxconf/",
"type" => "jmx",
"metric_path" => "usbgc-lab-vd1.ny.cantor.com_9119.Memory.NonHeapMem
oryUsage.max",
"metric_value_number" => 587202560
}
{
"@version" => "1",
"@timestamp" => "2015-09-18T16:32:48.319Z",
"host" => "usbgc-lab-vd1.ny.cantor.com",
"path" => "c:/LogStash/jmxconf/",
"type" => "jmx",
"metric_path" => "usbgc-lab-vd1.ny.cantor.com_9119.Memory.NonHeapMem
oryUsage.used",
"metric_value_number" => 60404040
}
{
"@version" => "1",
"@timestamp" => "2015-09-18T16:32:48.323Z",
"host" => "usbgc-lab-vd1.ny.cantor.com",
"path" => "c:/LogStash/jmxconf/",
"type" => "jmx",
"metric_path" => "usbgc-lab-vd1.ny.cantor.com_9119.Memory.ObjectPend
ingFinalizationCount",
"metric_value_number" => 0
}
{
"@version" => "1",
"@timestamp" => "2015-09-18T16:32:48.335Z",
"host" => "usbgc-lab-vd1.ny.cantor.com",
"path" => "c:/LogStash/jmxconf/",
"type" => "jmx",
"metric_path" => "usbgc-lab-vd1.ny.cantor.com_9119.Memory.ObjectName
",
"metric_value_string" => "java.lang:type=Memory"
}

4

Am I correct when I say the output just listed is a Hash? Sorry, i'm new to Logstash and JSON in general. I want to convert from Hash to JSON.

5

{
"@version" => "1",
"@timestamp" => "2015-09-18T16:32:48.280Z",
"host" => "usbgc-lab-vd1.ny.cantor.com",
"path" => "c:/LogStash/jmxconf/",
"type" => "jmx",
"metric_path" => "usbgc-lab-vd1.ny.cantor.com_9119.Memory.NonHeapMem
oryUsage.init",
"metric_value_number" => 24576000
}

Depending on what kind of output you're interested in it's arguable that there's nothing here to parse. You already have the metric name and its value in discrete fields.

Am I correct when I say the output just listed is a Hash? Sorry, i'm new to Logstash and JSON in general. I want to convert from Hash to JSON.

The internal representation of a message inside Logstash might be a hash, but that's not something you should be concerned with. What's relevant is that it's a set of key/value pairs. Each message is converted upon output to e.g. JSON or, in the case above, a Ruby-specific format that's hardly useful for anything except debugging.

The jmx input produces key/value pairs from the start so you don't need to do any parsing. A grok filter is used to extract values from a flat string into a bunch of key/value pairs, but that's again nothing you need to do.

6

Thanks for the information. With it and some probing i'm able to see what you say as well as produce an example to do what I need. However I do have a new question: In the output that I posted earlier displays multiple events, meaning each are individual separate events. ie:

{
"@version" => "1",
"@timestamp" => "2015-09-18T16:32:48.228Z",
"host" => "usbgc-lab-vd1.ny.cantor.com",
"path" => "c:/LogStash/jmxconf/",
"type" => "jmx",
"metric_path" => "usbgc-lab-vd1.ny.cantor.com_9119.Memory.HeapMemory
Usage.committed",
"metric_value_number" => 192937984
}
{
"@version" => "1",
"@timestamp" => "2015-09-18T16:32:48.243Z",
"host" => "usbgc-lab-vd1.ny.cantor.com",
"path" => "c:/LogStash/jmxconf/",
"type" => "jmx",
"metric_path" => "usbgc-lab-vd1.ny.cantor.com_9119.Memory.HeapMemory
Usage.init",
"metric_value_number" => 162068032
}

...
...
...

How do I merge them into a single event? As a result I can get committed, and init JMX memory information as one event and post it to Elastic Search as one event. A little more insight into what i'm trying to do. I have Kibana connected to elastic search and want to see that single event as a datapoint on one of my graphs.

many thanks,
Tony

7

That's actually not easily done. You might be able to use the aggregate filter but I think one problem is that it's hard to know which events should be merged. I don't think one can count on the various metrics to arrive in the same order every time, they might be interspersed by other events, and events might be processed in batch so that multiple sets of related events are processed at the same time. Getting this right is probably hard with the stock plugins.

8

I see. I figured that as much but wanted to hear from an expert. If I come up with a custom solution i will post it.

9

Hi, Were you able to find a solution to aggregate related JMX events into single event?