Problem in setting up a Watcher

Elastic Stack Elasticsearch
1

I've been trying to setup a Watcher that could print the following fields:

  • monitor.name
  • monitor.type
  • url.full
  • monitor.status

So far this was my configuration:

{
  "trigger": {
    "schedule": {
      "interval": "5m"
    }
  },
  "input": {
    "search": {
      "request": {
        "indices": ["heartbeat-*"],
        "body": {
          "size": 10,
          "_source": ["monitor.name", "monitor.type", "url.full", "monitor.status"],
          "query": {
            "range": {
              "@timestamp": {
                "gte": "now-5m"
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gte": 1
      }
    }
  },
  "actions": {
    "index_action": {
      "index": {
        "index": "watcher_ti",
        "execution_time_field": "timestamp",
        "doc": {
          "monitor_name": "{{ctx.payload.hits.hits.0._source.monitor.name}}",
          "monitor_type": "{{ctx.payload.hits.hits.0._source.monitor.type}}",
          "url_full": "{{ctx.payload.hits.hits.0._source.url.full}}",
          "monitor_status": "{{ctx.payload.hits.hits.0._source.monitor.status}}",
          "timestamp": "{{ctx.execution_time}}"
        }
      }
    }
  }
}

The error returned so far is:
could not parse [index] action [0bbde42b-e4a9-48cd-835b-09ea8dd2f093/index_action]. unexpected token [START_OBJECT]

Which led me to belive I'm making a mistake when addressing the:

        "doc": {
          "monitor_name": "{{ctx.payload.hits.hits.0._source.monitor.name}}",
          "monitor_type": "{{ctx.payload.hits.hits.0._source.monitor.type}}",
          "url_full": "{{ctx.payload.hits.hits.0._source.url.full}}",
          "monitor_status": "{{ctx.payload.hits.hits.0._source.monitor.status}}",
          "timestamp": "{{ctx.execution_time}}" 
        }

But I haven't beem able to find another solution. Any light here would be wonderful news.

Best regards!

2

Thanks for reaching out, @SamuelSMendes. This guide might be a good starting point here.

1 Like
3

Thank you for replying. I tried looking into this but still couldn't find a way to parse my Watcher event into the respective fields.
For now I've evolved to the point I can make the Watcher to gather the documents and trigger events into the desired index. But I still can't format the documents so it goes into the right fields.

My current Watcher configuration is this:

{
    "trigger": {
      "schedule": {
        "interval": "1m"
      }
    },
    "input": {
      "search": {
        "request": {
          "search_type": "query_then_fetch",
          "indices": [
            "heartbeat-*"
          ],
          "rest_total_hits_as_int": true,
          "body": {
            "size": 10,
            "_source": [
              "monitor.name",
              "monitor.type",
              "url.full",
              "monitor.status"
            ],
            "query": {
              "bool": {
                "must": [
                  {
                    "range": {
                      "@timestamp": {
                        "gte": "now-30m"
                      }
                    }
                  },
                  {
                    "match": {
                      "monitor.status": {
                        "query": "down"
                      }
                    }
                  }
                ]
              }
            }
          }
        }
      }
    },
    "condition": {
      "compare": {
        "ctx.payload.hits.total": {
          "gte": 0
        }
      }
    },
    "actions": {
      "index_action": {
        "index": {
          "index": "watcher_ti"
        }
      }
    }
}

And the returned documents comes as this:

{
  "_index": "watcher_ti",
  "_id": "9utADZABVTr_EGDCNTvM",
  "_version": 1,
  "_score": 0,
  "_source": {
    "_shards": {
      "total": 400,
      "failed": 0,
      "successful": 400,
      "skipped": 399
    },
    "hits": {
      "hits": [
        {
          "_index": ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
          "_source": {
            "monitor": {
              "name": "Web Backend CLIENT",
              "type": "http",
              "status": "down"
            },
            "url": {
              "full": "http://X.X.X.9:8082"
            }
          },
          "_id": "89A_DZABnV4Un6b4uJuA",
          "_score": 4.7359004
        },
        {
          "_index": ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
          "_source": {
            "monitor": {
              "name": "Web Backend CLIENT",
              "type": "http",
              "status": "down"
            },
            "url": {
              "full": "http://X.X.X.9:8082"
            }
          },
          "_id": "i9A-DZABnV4Un6b4zkch",
          "_score": 4.7359004
        },
        {
          "_index": ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
          "_source": {
            "monitor": {
              "name": "Web Backend CLIENT",
              "type": "http",
              "status": "down"
            },
            "url": {
              "full": "http://X.X.X.9:8082"
            }
          },
          "_id": "5889DZABnV4Un6b44_6_",
          "_score": 4.7359004
        },
        {
          "_index": ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
          "_source": {
            "monitor": {
              "name": "Web Backend CLIENT",
              "type": "http",
              "status": "down"
            },
            "url": {
              "full": "http://X.X.X.9:8082"
            }
          },
          "_id": "Xs88DZABnV4Un6b4-chf",
          "_score": 4.7359004
        },
        {
          "_index": ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
          "_source": {
            "monitor": {
              "name": "Web Backend CLIENT",
              "type": "http",
              "status": "down"
            },
            "url": {
              "full": "http://X.X.X.9:8082"
            }
          },
          "_id": "vM88DZABnV4Un6b4Dpf_",
          "_score": 4.7359004
        },
        {
          "_index": ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
          "_source": {
            "monitor": {
              "name": "Web Backend CLIENT",
              "type": "http",
              "status": "down"
            },
            "url": {
              "full": "http://X.X.X.9:8082"
            }
          },
          "_id": "Xc87DZABnV4Un6b4JGmf",
          "_score": 4.7359004
        },
        {
          "_index": ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
          "_source": {
            "monitor": {
              "name": "Web Backend CLIENT",
              "type": "http",
              "status": "down"
            },
            "url": {
              "full": "http://X.X.X.9:8082"
            }
          },
          "_id": "xc86DZABnV4Un6b4Ojw9",
          "_score": 4.7359004
        },
        {
          "_index": ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
          "_source": {
            "monitor": {
              "name": "Web Backend CLIENT",
              "type": "http",
              "status": "down"
            },
            "url": {
              "full": "http://X.X.X.9:8082"
            }
          },
          "_id": "m845DZABnV4Un6b4Uek7",
          "_score": 4.7359004
        },
        {
          "_index": ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
          "_source": {
            "monitor": {
              "name": "Application yyyy 1 CLIENT",
              "type": "http",
              "status": "down"
            },
            "url": {
              "full": "https://xxxCLIENT.tontonton.com/yyyy.html"
            }
          },
          "_id": "NsosDZABnV4Un6b4PkPv",
          "_score": 4.7359004
        },
        {
          "_index": ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
          "_source": {
            "monitor": {
              "name": "Application yyyy 2 CLIENT",
              "type": "http",
              "status": "down"
            },
            "url": {
              "full": "https://xxxCLIENT2.tontonton.com/yyyy.html"
            }
          },
          "_id": "N8osDZABnV4Un6b4PkPv",
          "_score": 4.7359004
        }
      ],
      "total": 136,
      "max_score": 4.7359004
    },
    "took": 41,
    "timed_out": false
  },
  "fields": {
    "_shards.successful": [
      400
    ],
    "hits.hits._source.monitor.status.keyword": [
      "down",
      "down",
      "down",
      "down",
      "down",
      "down",
      "down",
      "down",
      "down",
      "down"
    ],
    "took": [
      41
    ],
    "hits.hits._score": [
      4.7359004,
      4.7359004,
      4.7359004,
      4.7359004,
      4.7359004,
      4.7359004,
      4.7359004,
      4.7359004,
      4.7359004,
      4.7359004
    ],
    "hits.total": [
      136
    ],
    "_shards.skipped": [
      399
    ],
    "hits.hits._source.monitor.status": [
      "down",
      "down",
      "down",
      "down",
      "down",
      "down",
      "down",
      "down",
      "down",
      "down"
    ],
    "hits.hits._id.keyword": [
      "89A_DZABnV4Un6b4uJuA",
      "i9A-DZABnV4Un6b4zkch",
      "5889DZABnV4Un6b44_6_",
      "Xs88DZABnV4Un6b4-chf",
      "vM88DZABnV4Un6b4Dpf_",
      "Xc87DZABnV4Un6b4JGmf",
      "xc86DZABnV4Un6b4Ojw9",
      "m845DZABnV4Un6b4Uek7",
      "NsosDZABnV4Un6b4PkPv",
      "N8osDZABnV4Un6b4PkPv"
    ],
    "hits.hits._index.keyword": [
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001"
    ],
    "hits.hits._source.monitor.type": [
      "http",
      "http",
      "http",
      "http",
      "http",
      "http",
      "http",
      "http",
      "http",
      "http"
    ],
    "timed_out": [
      false
    ],
    "hits.hits._source.url.full": [
      "http://X.X.X.9:8082",
      "http://X.X.X.9:8082",
      "http://X.X.X.9:8082",
      "http://X.X.X.9:8082",
      "http://X.X.X.9:8082",
      "http://X.X.X.9:8082",
      "http://X.X.X.9:8082",
      "http://X.X.X.9:8082",
      "https://xxxCLIENT.tontonton.com/yyyy.html",
      "https://xxxCLIENT2.tontonton.com/yyyy.html"
    ],
    "hits.max_score": [
      4.7359004
    ],
    "_shards.total": [
      400
    ],
    "hits.hits._source.url.full.keyword": [
      "http://X.X.X.9:8082",
      "http://X.X.X.9:8082",
      "http://X.X.X.9:8082",
      "http://X.X.X.9:8082",
      "http://X.X.X.9:8082",
      "http://X.X.X.9:8082",
      "http://X.X.X.9:8082",
      "http://X.X.X.9:8082",
      "https://xxxCLIENT.tontonton.com/yyyy.html",
      "https://xxxCLIENT2.tontonton.com/yyyy.html"
    ],
    "hits.hits._source.monitor.type.keyword": [
      "http",
      "http",
      "http",
      "http",
      "http",
      "http",
      "http",
      "http",
      "http",
      "http"
    ],
    "hits.hits._source.monitor.name.keyword": [
      "Web Backend CLIENT",
      "Web Backend CLIENT",
      "Web Backend CLIENT",
      "Web Backend CLIENT",
      "Web Backend CLIENT",
      "Web Backend CLIENT",
      "Web Backend CLIENT",
      "Web Backend CLIENT",
      "Application yyyy 1 CLIENT",
      "Application yyyy 2 CLIENT"
    ],
    "hits.hits._source.monitor.name": [
      "Web Backend CLIENT",
      "Web Backend CLIENT",
      "Web Backend CLIENT",
      "Web Backend CLIENT",
      "Web Backend CLIENT",
      "Web Backend CLIENT",
      "Web Backend CLIENT",
      "Web Backend CLIENT",
      "Application yyyy 1 CLIENT",
      "Application yyyy 2 CLIENT"
    ],
    "hits.hits._index": [
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001",
      ".ds-heartbeat-8.11.3-2024.06.12-2024.06.12-000001"
    ],
    "hits.hits._id": [
      "89A_DZABnV4Un6b4uJuA",
      "i9A-DZABnV4Un6b4zkch",
      "5889DZABnV4Un6b44_6_",
      "Xs88DZABnV4Un6b4-chf",
      "vM88DZABnV4Un6b4Dpf_",
      "Xc87DZABnV4Un6b4JGmf",
      "xc86DZABnV4Un6b4Ojw9",
      "m845DZABnV4Un6b4Uek7",
      "NsosDZABnV4Un6b4PkPv",
      "N8osDZABnV4Un6b4PkPv"
    ],
    "_shards.failed": [
      0
    ]
  }
}

Now the issue I'm having is to set these documents into their fields instead of the giant array of Hits, which gather all documents inside a single one.

4

Thanks for following up, @SamuelSMendes. I'm glad to hear you are having some progress. There are a few older Discuss and StackOverflow posts that may be useful to take a look at:

This guide has also been helpful for me to take a look at in the past as well.