Nested Field Different Data Types

Elastic Stack Logstash
1

Heya, I have a source document that fails to load into elastic search.
The reason is that the source document contains text in a nested field that should be of type boolean.

I would like to process this log in logstash to make it loadable in elastic. This is what the source document looks like -->

{
  "date": "2018-01-01",
  "source": "ABC",
  "data": [
    {
      "id": 1,
      "field1": true,
      "field2": false,
      "field3": true
    },
    {
      "id": 2,
      "field1": true,
      "field2": false,
      "field3": nil
    }
  ]
}

Logically what I want to do is check if the nested field is "True" and if not set the value to "False". However I have not been able to break into the nested document to do any checks. I think it should look something like this however I'm just guessing -->

ruby     => "
  k = event.get('[data]')
  k.to_hash.each do  { 
  | index | 
  if  event.get('[data][index][field3]') == "True"
      event.set('[data][index][field3]', "True")
  else
      event.set('[data][index][field3]', "False")
  end
        }"

I would love some help.

2

Your base logic is indeed correct, minor Ruby script errors aside. Something like this should do what you want

    ruby {
        code => "
            k = event.get('[data]')
            k.each_index { | index |
                if  event.get('[data]['+index.to_s+'][field3]') == 'true'
                    event.set('[data]['+index.to_s+'][field3]', true)
                else
                    event.set('[data]['+index.to_s+'][field3]', false)
                end
            }
        "
    }
3

Wow - thanks for the response Paz.

It runs perfectly.

I must say I would never have got that - thank you so much.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.