Watcher: How to use Mustache-Syntax/dynamic values from aggregations in chain-input

[apt-get_install_skil]

Hey there,

in my filebeat-indices I have documents that contain an ip-address field 'destination_ip' and other documents that contain the same ip-field as well as the corresponding uri of that ip-address. Documents with service.name=foo contain only the destination_ip-field and documents with service.name=bar contain the destination_ip-field as well as the destination_uri-field.

Requirement:

I need to implement a watcher that gets me all destination_ip-values that match certain criteria and then use these values in a second search-input to get me all the corresponding destination_uri-values.

I was able to implement a watcher with chain-input like so:

"input" : {
  "chain" : {
    "inputs" : [ 
      {
        "first" : {
          "search":{
            "request":{
              "search_type": "query_then_fetch",
              "indices":[
                "<filebeat-7*-{now/d}>",
                "<filebeat-7*-{now/d-1d}>"
              ],
              "rest_total_hits_as_int": true,
              "body":{
                "query":{
                  "bool":{
                    "must":[
                      {
                        "term":{
                          "service.name": "foo"
                        }
                      }
                    ],
                    "must_not":[
                      {
                        ...
                      }
                    ]
                  }
                },
                "aggs":{
                  "destination_ips":{
                    "terms":{
                      "field": "destination_ip",
                      "size": 10
                    }
                  }
                }
              }
            }
          }
        }
      },
      {
        "second" : {
          "search":{
            "request":{
              "search_type": "query_then_fetch",
              "indices":[
                "<filebeat-7*-{now/d}>",
                "<filebeat-7*-{now/d-1d}>"
              ],
              "rest_total_hits_as_int": true,
              "body":{
                "query":{
                  "bool":{
                    "must":[
                      {
                        "term":{
                          "service.name": "bar"
                        }
                      },
                      {
                        "terms":{
                          "ip":[
                            "{{#ctx.payload.first.aggregations.destination_ips.buckets}}{{key}}",
                            "{{/ctx.payload.first.aggregations.destination_ips.buckets}}"
                          ]
                        }
                      }
                    ],
                    "must_not":[
                      {
                        ...
                      }
                    ]
                  }
                },
                "aggs":{
                  "destination_uris":{
                    "terms":{
                      "field": "destination_uri",
                      "size": 10
                    }
                  }
                }
              }
            }
          }
        }
      }
    ]
  }
}

The first search-input works fine and the aggregation gives me all ip-values.

The second search-input works also quite fine but with one error:

The resolved values of {{#ctx.payload.first.aggregations.destination_ips.buckets}} are matching the results in the aggregation of the first search-input but contain an additional whitespace at the end.

So the terms query in the second search-input looks like this:

{
  "terms":{
    "ip":[
       "192.168.6.37",
       "192.168.6.38",
       "192.168.6.39",
       "192.168.6.40",
       ""
    ]
  }
}

As a result, the watcher fails because it states (correctly) that "" (the blank string) is not an IP string literal.

There are no blank-values that are coming from the aggregation in the first search-input. I verified that with the following adjustments:

{
   "terms":{
     "ip":[
        "{{#ctx.payload.first.aggregations.destination_ips.buckets}}{{key}}+foo",
        "{{/ctx.payload.first.aggregations.destination_ips.buckets}}+bar"
     ]
   }
}

This gives me the following resolved values:

{
  "terms":{
    "ip":[
       "192.168.6.37+foo",
       "192.168.6.38+foo",
       "192.168.6.39+foo",
       "192.168.6.40+foo",
       "+bar"
    ]
  }
}

I am not sure if the mustache-syntax is correct.

Any help to get this working is much appreciated. The structure of the documents can not be changed.

Elasticsearch-Version: 7.2.0

[apt-get_install_skil]

Can anyone help me with this?