Problem to setup sub flieds

Elastic Stack Logstash
1

Hi,

I have a problem to generate sub fileds from a json message .

With the following filter I try to check if the field "details" exist and add the json fields to it.
If it doesn't exist the field sould be created with the sub fields, this working.

I tried the following filter but it doesn't work:

  if "#json" in [message] {
    if ["details"] {
      grok { match => {"message" => "[details](\{.*\})"} }
    }
    else {
      grok { match => {"message" => "(?<details>\{.*\})"} }
    }
  }

In the "if" case the fields won't be added as sub field, everything is connected and writen as string value to the "details" field. Where is my mistake in the "if" case?

Regards
Thorsten

2

Does the name of the field have double quotes in it? If not, you probably want

if [details] { ...
3

Thank you for the hint, now it seams that the if case is used but I get a grok parse failure. There must be a something wrong in the if section

4

To describe the whole thing in more detail, the "details" field does not always exist. If it exists, it already has various subfields and the values ​​from a JSON message should be added as further subfields. The case where the field doesn't exist works for my filter.

In principle, the Json messages always look like this

Some text. #json: {"Value1":10,"Value2":8,"Value3":{"Value4":["Foo","Bar"],"Value5":["Foo","Bla"]}}

I would expect that the if case generate the following fields, which it doesn't do because of a grok parsing error.

details.foo (existing field)
details.bar (existing field)
details.Value1
details.Value2
details.Value3
details.Value4
details.Value5

I actually thought it would be enough if I changed the string to the field, but that doesn't work. The problem must be here and I don't understand what's wrong with it:

... {"message" => "[details](\{.*\})"}...
5

Why do you expect the [details] field to exist? Can you show us the structure of an event from

output { stdout { codec => rubydebug } }
6

The problem is that some few of our functions have the [details] field occupied, which was not properly implemented by our developers, so it may be that the [details] field already exists with subfields. Therefore the information from the json messages must be added in this field as subfield. I've already forwarded this to the appropriate teams, but it may take a while to clean it up.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.