Watcher that counts the documents that arrive to an index in kibana

found it!!! You beat me... that is because the aggregation is not inside the "hits" we need to be explicit with the field name

In the condition.

  "condition": {
    "compare": {
      "ctx.payload.aggregations.heartbeat_count.value": {
        "gte": 1000
      }
    }
  }

And Action should be

  "actions": {
    "notify-slack": {
      "slack": {
        "message": {
          "to": [
            "#stephenb-es-integration"
          ],
          "text": "Encountered  {{ctx.payload.aggregations.heartbeat_count.value}} heartbeats in the last 1 day (facepalm)"
        }
      }
    }
  }
}

I will fix above!

Results

{
  "watch_id": "_inlined_",
  "node": "6FBy_sIiSGONVnQZ5O8GkA",
  "state": "executed",
  "user": "elastic",
  "status": {
    "state": {
      "active": true,
      "timestamp": "2021-04-23T23:35:37.938Z"
    },
    "last_checked": "2021-04-23T23:35:37.938Z",
    "last_met_condition": "2021-04-23T23:35:37.938Z",
    "actions": {
      "notify-slack": {
        "ack": {
          "timestamp": "2021-04-23T23:35:37.938Z",
          "state": "ackable"
        },
        "last_execution": {
          "timestamp": "2021-04-23T23:35:37.938Z",
          "successful": true
        },
        "last_successful_execution": {
          "timestamp": "2021-04-23T23:35:37.938Z",
          "successful": true
        }
      }
    },
    "execution_state": "executed",
    "version": -1
  },
  "trigger_event": {
    "type": "manual",
    "triggered_time": "2021-04-23T23:35:37.938Z",
    "manual": {
      "schedule": {
        "scheduled_time": "2021-04-23T23:35:37.938Z"
      }
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "heartbeat-*"
        ],
        "rest_total_hits_as_int": true,
        "body": {
          "size": 0,
          "query": {
            "bool": {
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-1d/d",
                      "lt": "now/d"
                    }
                  }
                }
              ]
            }
          },
          "aggs": {
            "heartbeat_count": {
              "value_count": {
                "field": "_index"
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.aggregations.heartbeat_count.value": {
        "gte": 1000
      }
    }
  },
  "metadata": {
    "name": "test-heartbeat-watcher",
    "xpack": {
      "type": "json"
    }
  },
  "result": {
    "execution_time": "2021-04-23T23:35:37.938Z",
    "execution_duration": 262,
    "input": {
      "type": "search",
      "status": "success",
      "payload": {
        "_shards": {
          "total": 11,
          "failed": 0,
          "successful": 11,
          "skipped": 0
        },
        "hits": {
          "hits": [],
          "total": 10000,
          "max_score": null
        },
        "took": 14,
        "timed_out": false,
        "aggregations": {
          "heartbeat_count": {
            "value": 60474
          }
        }
      },
      "search": {
        "request": {
          "search_type": "query_then_fetch",
          "indices": [
            "heartbeat-*"
          ],
          "rest_total_hits_as_int": true,
          "body": {
            "size": 0,
            "query": {
              "bool": {
                "filter": [
                  {
                    "range": {
                      "@timestamp": {
                        "gte": "now-1d/d",
                        "lt": "now/d"
                      }
                    }
                  }
                ]
              }
            },
            "aggs": {
              "heartbeat_count": {
                "value_count": {
                  "field": "_index"
                }
              }
            }
          }
        }
      }
    },
    "condition": {
      "type": "compare",
      "status": "success",
      "met": true,
      "compare": {
        "resolved_values": {
          "ctx.payload.aggregations.heartbeat_count.value": 60474
        }
      }
    },
    "actions": [
      {
        "id": "notify-slack",
        "type": "slack",
        "status": "success",
        "slack": {
          "account": "monitoring",
          "sent_messages": [
            {
              "status": "success",
              "to": "#stephenb-es-integration",
              "message": {
                "from": "x-pack",
                "icon": "http://example.com/images/watcher-icon.jpg",
                "text": "Encountered  60474 heartbeats in the last 1 day (facepalm)"
              }
            }
          ]
        }
      }
    ]
  },
  "messages": []
}

but it doesn't show up in the mail action
:frowning:

It does... go back and go slow... mine just showed up in slack.

Do the simulate and force the action...

Show me your email action again.

"actions": [
      {
        "id": "email_1",
        "type": "email",
        "status": "simulated",
        "email": {
          "message": {
            "id": "email_1__inlined__64096482-0d81-43e6-abba-947a6aa2cc67-2021-04-23T23:36:17.648901Z_26879",
            "sent_date": "2021-04-23T23:36:17.658797Z",
            "to": [
              "victor.vera@megadvantage.com",
              "juan.jaramillo@megadvantage.com",
              "andres.molinac.pr@etb.com.co",
              "juan.ariasp1.pr@etb.com.co"
            ],
            "bcc": [
              "juancho.jaramillo16@gmail.com"
            ],
            "subject": "Estado herramienta monitoreo",
            "body": {
              "text": "Buen dia, \n\n āš La herramienta de monitoreo a funcionado correctamente en las ultimas  24 horas  \n \n Estado: UPšŸŸ¢ \n\n šŸ•§Hora: 2021-04-23 18:36:17  \n\n Ɓrea: Networking \n \nšŸ””Mensaje Alerta: Por favor no responder a este mensaje \n\n "
            }
          }
        }
      }
    ]
  },
  "messages": []
}

this is the action:

"actions": {
    "email_1": {
      "email": {
        "profile": "standard",
        "attach_data": {
          "format": "yaml"
        },
        "to": [
          "victor.vera@megadvantage.com",
          "juan.jaramillo@megadvantage.com",
          "andres.molinac.pr@etb.com.co",
          "juan.ariasp1.pr@etb.com.co"
        ],
        "bcc": [
          "juancho.jaramillo16@gmail.com"
        ],
        "subject": "{{ctx.metadata.name}}",
        "body": {
          "text": """Buen dia, 

 āš La herramienta de monitoreo a funcionado correctamente en las ultimas  24 horas  {{ctx.payload.aggregations.heartbeat_count.value}}
 
 Estado: UPšŸŸ¢ 

 šŸ•§Hora: {{ctx.payload.time_triggered}}  {{ctx.result.execution_time}}

 Ɓrea: Networking 
 
šŸ””Mensaje Alerta: Por favor no responder a este mensaje 

 """
        }
      }
    }
  },

does not show it in the mail:

 "actions": [
      {
        "id": "email_1",
        "type": "email",
        "status": "simulated",
        "email": {
          "message": {
            "id": "email_1__inlined__48c2b07b-4996-4d22-9cfe-34bac4cd8ac3-2021-04-23T23:38:30.195799Z_26880",
            "sent_date": "2021-04-23T23:38:30.205315Z",
            "to": [
              "victor.vera@megadvantage.com",
              "juan.jaramillo@megadvantage.com",
              "andres.molinac.pr@etb.com.co",
              "juan.ariasp1.pr@etb.com.co"
            ],
            "bcc": [
              "juancho.jaramillo16@gmail.com"
            ],
            "subject": "Estado herramienta monitoreo",
            "body": {
              "text": "Buen dia, \n\n āš La herramienta de monitoreo a funcionado correctamente en las ultimas  24 horas  \n \n Estado: UPšŸŸ¢ \n\n šŸ•§Hora: 2021-04-23 18:38:30  \n\n Ɓrea: Networking \n \nšŸ””Mensaje Alerta: Por favor no responder a este mensaje \n\n "
            }
          }
        }
      }
    ]
  },
  "messages": []
}

can you test a simple body without all the punctuation first like

"text": "Encountered {{ctx.payload.aggregations.heartbeat_count.value}} heartbeats in the last 1 day"

Like this?

 "actions": {
    "email_1": {
      "email": {
        "profile": "standard",
        "attach_data": {
          "format": "yaml"
        },
        "to": [
          "victor.vera@megadvantage.com",
          "juan.jaramillo@megadvantage.com",
          "andres.molinac.pr@etb.com.co",
          "juan.ariasp1.pr@etb.com.co"
        ],
        "bcc": [
          "juancho.jaramillo16@gmail.com"
        ],
        "subject": "{{ctx.metadata.name}}",
        "body": {
          "text":  "Encountered {{ctx.payload.aggregations.heartbeat_count.value}} heartbeats in the last 1 day"
        }
      }
    }
  },

I just sent your text to slack looks like this and I see the number

Buen dia,
:warning:La herramienta de monitoreo a funcionado correctamente en las ultimas 24 horas 60474

Estado: UP:large_green_circle:
:clock1230:Hora:
Ɓrea: Networking

:bell:Mensaje Alerta: Por favor no responder a este mensaje

no good, nothing appears

 "actions": [
      {
        "id": "email_1",
        "type": "email",
        "status": "simulated",
        "email": {
          "message": {
            "id": "email_1__inlined__86e47b0a-e497-41e0-83bf-ff852ce4346b-2021-04-23T23:41:10.833199Z_26881",
            "sent_date": "2021-04-23T23:41:10.845017Z",
            "to": [
              "victor.vera@megadvantage.com",
              "juan.jaramillo@megadvantage.com",
              "andres.molinac.pr@etb.com.co",
              "juan.ariasp1.pr@etb.com.co"
            ],
            "bcc": [
              "juancho.jaramillo16@gmail.com"
            ],
            "subject": "Estado herramienta monitoreo",
            "body": {
              "text": "Encountered  heartbeats in the last 1 day"
            }
          }
        }
      }
    ]
  },
  "messages": []
}

what error is it? it doesn't work for me :frowning:

Something else is going on let me look close

My Slack :slight_smile:

I am sending you the whole code for your review:

{
  "trigger": {
    "schedule": {
      "daily": {
        "at": [
          "14:00"
        ]
      }
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "heartbeat*"
        ],
        "rest_total_hits_as_int": true,
        "body": {
          "size": 0,
          "query": {
            "bool": {
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-1d/d",
                      "lt": "now/d"
                    }
                  }
                }
              ]
            }
          },
          "aggs": {
            "heartbeat_count": {
              "value_count": {
                "field": "_index"
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.aggregations.heartbeat_count.value": {
        "gte": 1000
      }
    }
  },
  "actions": {
    "email_1": {
      "email": {
        "profile": "standard",
        "attach_data": {
          "format": "yaml"
        },
        "to": [
          "victor.vera@megadvantage.com",
          "juan.jaramillo@megadvantage.com",
          "andres.molinac.pr@etb.com.co",
          "juan.ariasp1.pr@etb.com.co"
        ],
        "bcc": [
          "juancho.jaramillo16@gmail.com"
        ],
        "subject": "{{ctx.metadata.name}}",
        "body": {
          "text":  "Encountered {{ctx.payload.aggregations.heartbeat_count.value}} heartbeats in the last 1 day"
        }
      }
    }
  },
  "transform": {
    "script": {
      "source": "return [ 'time_triggered': Instant.ofEpochMilli(ctx.trigger.triggered_time.getMillis()).atZone(ZoneId.of('America/Bogota')).format(DateTimeFormatter.ofPattern('YYYY-MM-dd HH:mm:ss')) ];",
      "lang": "painless"
    }
  }
}

To Test

  "trigger": {
    "schedule": {
      "interval": "15s"
    }
  },

To test take out that transform script at the bottom can try that later.

,
  "transform": {
    "script": {
      "source": "return [ 'time_triggered': Instant.ofEpochMilli(ctx.trigger.triggered_time.getMillis()).atZone(ZoneId.of('America/Bogota')).format(DateTimeFormatter.ofPattern('YYYY-MM-dd HH:mm:ss')) ];",
      "lang": "painless"
    }
  }

so just try this (put the simple email back in).
Email just to you (are you spamming your friends :slight_smile: )

 {
  "trigger": {
    "schedule": {
      "interval": "15s"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "heartbeat-*"
        ],
        "rest_total_hits_as_int": true,
        "body": {
          "size": 0,
          "query": {
            "bool": {
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-1d/d",
                      "lt": "now/d"
                    }
                  }
                }
              ]
            }
          },
          "aggs": {
            "heartbeat_count": {
              "value_count": {
                "field": "_index"
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.aggregations.heartbeat_count.value": {
        "gte": 1000
      }
    }
  },
  "actions": {
    "notify-slack": {
      "slack": {
        "message": {
          "to": [
            "#stephenb-es-integration"
          ],
          "text": "{{ctx.metadata.name}} : Encountered {{ctx.payload.aggregations.heartbeat_count.value}} heartbeats in the last 1 day"
        }
      }
    }
  }
}

My output

[4:58]
test-heartbeat-watcher : Encountered 60474 heartbeats in the last 1 day

[4:58]
test-heartbeat-watcher : Encountered 60474 heartbeats in the last 1 day

[4:58]
test-heartbeat-watcher : Encountered 60474 heartbeats in the last 1 day

[4:59]
test-heartbeat-watcher : Encountered 60474 heartbeats in the last 1 day

@Juan_David_Jaramillo I need to step out for a bit... you are close....

Thank you, it was indeed because of the script that I did not get it.
But now I have a problem, I can't use the script? :frowning:

"actions": [
      {
        "id": "email_1",
        "type": "email",
        "status": "simulated",
        "email": {
          "message": {
            "id": "email_1__inlined__bf91ec99-e03d-4917-8f25-483335fc9da2-2021-04-24T00:04:05.187357Z_26886",
            "sent_date": "2021-04-24T00:04:05.201541Z",
            "to": [
              "victor.vera@megadvantage.com",
              "juan.jaramillo@megadvantage.com"
            ],
            "bcc": [
              "juancho.jaramillo16@gmail.com"
            ],
            "subject": "Estado herramienta monitoreo",
            "body": {
              "text": "Buen dia, \n\n āš La herramienta de monitoreo a funcionado correctamente en las ultimas  24 horas  \n \n there are 82076 documents in your index. Threshold is 1000.\n \n Estado: UPšŸŸ¢ \n\n šŸ•§Hora:   \n\n Ɓrea: Networking \n \nšŸ””Mensaje Alerta: Por favor no responder a este mensaje \n\n "
            }
          }
        }
      }
    ]
  },

Yay ! :slight_smile:

The Script... Perhaps.... Probably and error in the script.... Perhaps I can take a look later.

What are you actually trying to do with the script?

Are You just trying to display the triggered time in the correct time zone?

yes,I need to display the actual time in my time zone.

Yes OK... I am sure we can figure that out.

Someone with painless might be faster... so perhaps a new thread..

Or I will take a look later...

ok, thanks, if it is important too, I need the real time because elasticsearch gives the wrong time, that's why I need the script.

Understood... it will be easy once we know the "Encantamiento mƔgico"