Update field with new values is not possible using aggregate filter

I am trying to update a JSON object called "attributes" inside aggregate filter.

In some cases, I may or may not have attributes in Index, if it is not available means I will insert "attributes" as new JSON object field in index. If already exists, I need to update all the values inside "attributes" field, but due to below code it is updating the values which are mapped with "attributename" column but not removing some of the already existing attributes.
How can we update the attributes field completely without any old data?

For example,
I have an existing attributes JSON object like below:

attributes: { "Name": "Jack", "Age": "27", "City": "Chennai" }

The new attributes values comes from SQL will be having below values:

attributes: { "Name": "Snow", "Age": "20" }

The current result I am getting is like below:

attributes: { "Name": "Snow", "Age": "20",  "City": "Chennai" }

The exepected result I need it is like below:

attributes: { "Name": "Snow", "Age": "20" }

The below code I tried so far:

filter {
       if "my_employee_index" in [tags] {
        aggregate {
            task_id => "%{componentinstance.id}"
            code => "
                map['id'] = event.get('componentinstance.id')
                map['attributes'] ||= {}
				map['attributes'][event.get('attributename')] = event.get('attributevalue') /* Here I am mapping attributename with attribute value received in SQL Statement */
                event.cancel()
            "
            push_previous_map_as_event => true
            timeout => 3
        }

        mutate {
            add_field => { "custom_index_name" => "my_employee_index" }
        }
    } else {
        mutate {
            add_field => { "custom_index_name" => "%{type}" }
        }
    }
}
output {
    elasticsearch {
      hosts => ["http://localhost:9200"]
      index => "%{custom_index_name}"
      document_id => "%{componentinstance.id}"
      document_type => "%{type}"
      action => "update"
      doc_as_upsert => true
    }
}

Thanks in Advance! :slight_smile:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.