For some reason I will not mention here, I need to do document /_ingestion in an Elasticsearch cluster (only the data transformation) and index the transformed document in another cluster. Is it possible to tell an ingest node to index the document in another cluster that the one that it's part of?
No, as far as I know it is not.
Thanks for your answer. Workaround possible ? Like indexing the event on the ingest node then replicate (realtime) the document on the distant cluster (and the destroy the one on the ingest node)? I thinking of some "reindex" usage.
Why not use an ingest node on the other cluster or perhaps Logstash?
we are forbidden to use the /_ingest of the other cluster. However we already have code for /_ingest pipelines (written in painless if I'm not mistaken) on our dev environment. Is logstash directly compatible with the developed pipelines and can be used as a drop in replacement for the ingest functions of ES?
I completely forgot that version 6.5 actually introduced Cross Cluster Replication, which does exactly what you describe.
Logstash can do what ingest node can, but you can not directly use the same config. There is however a conversion utility available that can handle a lot of the available processors, so might be an option.
Our target cluster is a 5.5.0, however the ingest one is not installed and could be a 6.5, would this working?
However what you said about logstash maybe a good workaround!
Cross cluster replication require all clusters to be version 6.5 or higher.
Thanks a lot for those information.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.