Update nested array in existing document

Elastic Stack Logstash
1

Hello !
I have documents with fields like that :

  • doc.site containing a string like "paris"
  • doc.ips.values containing a list of strings ["10.0.0.1", "10.0.0.2"], or which can sometimes be empty []
  • doc.ips.last_checked containing a date like "2021-08-17T19:34:51.000Z"

And I want to create a new event (document) in Elasticsearch if a document with doc.site doesn't already exists.
If a document with the same doc.site value exist, I want to update some fields.

For example, lets image i receive this new document in logstash :

  • doc.site = "paris"

  • doc.ips.values = ["10.0.0.3"]

  • doc.ips.last_checked = "2021-08-18T19:34:51.000Z"

  • Then I want to update doc.ips.values and only add the values that didn't already exists inside (i.e. I only want to add the value "10.0.0.3" to the array. Then, the array withh contain ["10.0.0.1", "10.0.0.2", "10.0.0.3"]

  • I also want to replace the content of doc.ips.last_checked with the value received in the most recend document.

Here is my config which is not working :

elasticsearch {
      hosts => ["localhost:9200"]
      index => "myindex"
      action => "update"
      document_id => "%{[doc][site]}"
      doc_as_upsert => "true"
      script => 'ctx._source.doc.ips.values.add("%{[doc][ips][values]}");'
    }

As you can see on the image below, it's not working at all as expected.
image

Furthermore, when I send a document with a value like doc.ips.values = [], it's adding (empty) to my array...

Could someone help me ? Thanks a lot in advance !

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.