Execute enrich policy automatically

Elastic Stack Elasticsearch
1

Hi,

I am having below enrichment policy and want to execute automatically.
Is there anyway to achieve it and also I want to execute it every second is there any impact overall if I execute every second?

POST /_enrich/policy/transaction-policy/_execute?wait_for_completion=false

2

There is no way to automate the execution of enrich policies, there is a github issue asking for this, you can check it here.

Yes, the create of the enrich index after you run execute could take a few seconds, so running an execute every seconds it is not ideal, in fact, you should not use the the enrich policy for real time data or data that changes frequently.

This is mentioned here in the documentation.

We do not recommend using the enrich processor to append real-time data. The enrich processor works best with reference data that doesn’t change frequently.

What is your requirement to need to run the enrich policy every second? Maybe it is not the best way to enrich your data.

1 Like
3

Hi,

probably, you can try with a watcher that execute a webhook.

Something like this:

PUT _watcher/watch/enrich_policy_execute
{
"trigger": {
"schedule": {
"interval": "1s"
}
},
"actions": {
"execute_enrich_policy": {
"webhook": {
"method": "POST",
"host": "localhost",
"port": 9200,
"path": "/_enrich/policy/transaction-policy/_execute?wait_for_completion=false"
}
}
}
}

However, as Leandro said, please note that executing an enrich policy every second could have a significant impact on your Elasticsearch cluster's performance.

Regards