We are trying to create an enrich-processor to add more data to our event documents, based on an existing reference data that may contain ~10k documents.
I've read the example in:
However, in my case the reference data changes, at least once a day. Following the execute enrich policy api , I understand that every time the reference data changes I should re-execute the enrich-policy to recreate the enrich-index.
Is this a scalable solution? Will re-executing the enrich-policy 10-20 times a day be too costly? Should I use this feature for this purpose or should we enrich the data in our own platform?