Logstash move subfield to top field

Elastic Stack Logstash
1 
 "_score" : 1.0,
        "_source" : {
          "Europe" : {
            "DateTime" : {
              "Date" : "08.08.2020",
              "Time" : "15"
            },
            "City" : [
              {
                "Data" : {
                  "Temp" : "32",
                  "Humidity" : "33",
                  "WindDirection" : "NE",
                  "WindSpeed" : "03"

                },
                "CityName" : "Amsterdam"
              },

Hi, I am a complete beginner and I started playing around with the logstash conf file. I was wondering what is the best way to move fields (Temp,Humidity,WindDirection,WindSpeed) to the same level as "CityName" while removing the "Data" object.

2

Something very similar to this, except that instead of "doc" it would be "[City][Data]" and instead of using just k in the event.set it would be "[City][#{k}]".

3

Thanks for the answer I tried it but I got an error:
Ruby exception occurred: undefined method 'each' for nil:NilClass

I assume this is something I need to account for, do you have a suggestion on how to do that? I am not really familiar with ruby.

4

That is telling you that you called event.get on a field that does not exist. Try "[Europe][City][Data]" and "[Europe][City][#{k}]".

5 
"Europe" => {
        "DateTime" => {
            "Time" => "15",
             "Date" => "08.08.2020"
        },
               "City" => [
            [ 0] {
                "Data" => {
                           "Humidity" => "33",
                     "WindDirection" => "NE",
                            "Temp" => "32",
                    "WindSpeed" => "03"
                },
                "CityName" => "Amsterdam"
            },
            [ 1] {
                "Data" => {
                           "Humidity" => "18",
                     "WindDirection" => "E",
                            "Temp" => "33",
                    "WindSpeed" => "06"
                },
                "CityName" => "Ankara"
            },

This is the output when I run the config file as you can see I have multiple cities could that be the issue?

6

Yes indeed! You can try this, which I have not tested.

ruby {
    code => '
        oldCities = event.get("[Europe][City]")
        cities = []
        oldCities.each { |x|
            x["Data"].each { | k, v |
                x[k] = v
            }
           x.delete("Data")
           cities << x
        }
        event.set([Europe][City]", cities)
    '
}
7

It works thank you so much :slight_smile:

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.