Can I do query to retrieve some doc in ingest processor

Elastic Stack Elasticsearch
1

Hi
Does anyone have experience that writes your own ingest processor?
Is this possible to make a ingest processor that can query ES to retrieve a doc and that fill to the ingest doc before actual index?

By directly using Java API to query from ES is okay?

Please input your valuable advice. Thanks~

2

Ingest processors are limited to working with the current document, and should not call out. What you are looking to do should however be possible though Logstash and the use of the elasticsearch filter.

3

Hi @Christian_Dahlqvist
Thanks for your quick reply.
I'm limited to use elasticsearch only in this case. Basically what I need to do is to add a field to this ingest doc and that value is query from another index(in the same elasticsearch cluster). The easiest way to achieve to this is constructing doc in application side but unfortunately, application can't. So, I come to ingest pipeline to seek solution.

4

In case it helps:

5

Thank you @dadoonet
So this is not officailly supported by Elasticsearch right? Where can I have this plug-in though?

6

It’s not available. You just have some code source available within the PR and that’s all.

It was a WIP though.

7

Thanks for your guidance. Let me think how to start with it.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.