Run arbitrary code at ingest that is too big for Painless script

Elastic Stack Elasticsearch
1

Hello,

I have a simple algorithm I want to use as part of an ingest pipeline to derive a metric from a string field. For example: "this_is_my_string" and from that we derive a new field for insertion into the index which is called 'score' or similar.

The code and the constants I use as part of the calculation are 1MB and thus cannot be implemented in Painless which I think is limited to 64kB. I've experimented with the ML capability such as the BERT models etc but it seems like overkill for what is essentially 50 lines of code and 1MB of constants I need to derive the score.

Is there a feature in the Elastic ingest pipe where I can run my code?

Thank you

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.