I use enrich pipeline for matching docs from "source" index to "destination" index. For my data there are "many" docs from "source" index matched into "single" doc from "destination" index. Often, number of "source" docs is bigger than 128. Sometimes it can reach ~5k. So, I want to set "max_matches" to set 8196 in my enrich processor.
I've already tried to set enrich.coordinator_proxy.max_lookups_per_request to 8196. Found it here: Node Settings
But it did not take effect on "max_matches" option for enrich processor.
Eleasticsearch still returns error: "reason": "[max_matches] should be between 1 and 128",
Actually, I don't need to store "source" docs inside my "destination" doc. I need to do some calculations with list of "source" docs and save result into "destination" doc.
My ingest pipeline looks like:
- Enrich processor - match docs for "source" index to "destination" index by "internal_id" and save matched docs into "enrich_data" field of "destination" doc.
- Script processor - do some calculations with "enrich_data" field, save result of my calculation under some "score" field.
- Remove field processor - remove "enrich_data" field from "destination" doc.
How can I increase max allowed value for "max_matches" option for enrich processor?