Add_field with json codec

Elastic Stack Logstash
1

Hi,

I'm working with json files and using the json codec to get at the values in each document. So far so good.

I now need to add some new fields to the document which will contain values taken from nested json.

For example I have:

if [outernode][timeoutvalue] {
			mutate {
				add_field => { "timeout" => "%{[outernode][timeoutvalue]}" }
			}
		}

Which does not work... All I want to do is create a new field called "timeout" and populate it with the value found in the [outernode][timeoutvalue] field which already exists. Ultimately I would be removing the [outernnode] field entirely... but only after I have decanted all the interesting values out of it and into new fields... If that makes sense.

How can I "get at" those nested values for use in add_field?

Thanks!

UPDATE:

This seems to work okay:

add_field => { "timeout" => "%{outernode[timeoutvalue]}" }

2

add_field => { "timeout" => "%{outernode[timeoutvalue]}" }

That's not supposed to work. The syntax you originally used is the correct one, assuming your event looks like this:

{
  "outernode": {
    "timeoutvalue": 123
  }
}
3

Thanks inside the outernode there are up to forty unique entries, each with a value of its own. What I am trying to do is get those fields and values out of the outernode structure and make new fields for them. Then remove the outernode field completely. This will dramatically reduce the weight of the document that I send to ES.

Not sure why my original syntax would not work. Just glad that the latter syntax does appear to work okay.

Thanks again Magnus

4

If you want to move all subfields into the top level (effectively simply removing the outernode level) have a look at Help parsing nested JSON with dynamic top key.

5

Hi Magnus,

Thanks for the pointer.

I have often considered the ruby filter to be potentially the most powerful of all filters. I'd like to explore its use further. Is there a place I can read about all the event properties and methods available in the scope of that filter?

Thanks

6

I don't think there's any reference documentation. When necessary I look at the source code (which nowadays is Java, I believe) of the class backing the event variable. That covers most things.