Current Date in watcher

Elastic Stack Elasticsearch
1

Hi,

I have below index which will create everyday -

POST /test_index-2024-04-17/_doc
{
"field_to_check": "old_value",
"field_to_update": "old_value"
}

POST /test_index-2024-04-18/_doc
{
"field_to_check": "old_value",
"field_to_update": "old_value"
}

....

Now I want to create watcher on top of this index for some background work. I created watcher something like below but not able to change index based on current date.

PUT /_watcher/watch/test_watch
{
"trigger": {
"schedule": {
"interval": "10s"
}
},
"input": {
"search": {
"request": {
"indices": [
"test_index-2024-04-17"
],
"body": {
"query": {
"match": {
"field_to_check": "old_value"
}
}
}
}
}
},
"condition": {
"compare": {
"ctx.payload.hits.total": {
"gt": 0
}
}
},
"actions": {
"log_hits": {
"logging" : {
"text" : "Watcher triggered !!!"
}
}
}
}

Please help me in solving this issue.

2

Hello, I didn't quite understand. What do you want to do with your Watcher? Does it have to work for all your indexes? One watcher per index?

3

I want one watcher only and whenever watcher will invoke, it will take that day index.
Hope it help.

Something like "test_index-{now/d}"

4

I don't know if it's possible to do this in the "index" section, but you could create a data view that groups all your indexes together, for example "test-index*". Then you can use the query system and the condition to filter the right index.

5

can you please help me via some example?

6

There an example in the documentation : Script query | Elasticsearch Guide [8.13] | Elastic

GET /_search
{
"runtime_mappings": {
"amount.signed": {
"type": "double",
"script": """
double amount = doc['amount'].value;
if (doc['type'].value == 'expense') {
amount *= -1;
}
emit(amount);
"""
}
},
"query": {
"bool": {
"filter": {
"range": {
"amount.signed": { "lt": 10 }
}
}
}
},
"fields": [{"field": "amount.signed"}]
}

In this example, the query uses a script as a filter.

The first part corresponds to a runtime field (a calculated field). You can modify this script to extract the date contained in the name of your index (the _index field).
The second part corresponds to the filter, in this case based on the value returned by the runtime field

I think this query could be simplified, by creating a runtime field directly in your dataview. This runtime field would return the date contained in your index name. Then, you could use this runtime field to filter your Watcher query.

7

So :

  • create a data view that includes all your indexes (test-index*)
  • inside this dataview, create un runtime field that extracts the date contained in the index name, thanks to the _index field
  • then, run a query in your watcher based on this runtime field

I don't know if there are other solutions, but this is personally how I'd do it.