Ingest Node - Stored pipelines

[gourmax-geo (Maxime Gourlay)]

Hi,

I'm trying to make some test with ingest node and coordinator node.
The settings between both is pretty easy, but I was wondering if there is any way to be sure that my pipeline is really "stored" in the ingest node.

What I have done:

• create my pipeline from the coordinator node IP
• check the cluster state to find information about the location storage of my pipeline
  But I can't find this particular information.

I have an elasticsearch cluster and I want to have use coordinator node as "load balancing", does coordinator node will load balance write operation directly to ingest node ? Or I need to make write operation directly in the ingest node ?

Thanks.

[jakelandis]

but I was wondering if there is any way to be sure that my pipeline is really "stored" in the ingest node

The pipelines are stored in cluster state, it doesn't matter which node or node type (as long it is part of the cluster) you are pointed to when creating the pipeline. You should be able to find the pipeline definitions with this request

GET _cluster/state/metadata/?filter_path=metadata.ingest.pipeline

I have an elasticsearch cluster and I want to have use coordinator node as "load balancing", does coordinator node will load balance write operation directly to ingest node ? Or I need to make write operation directly in the ingest node ?

If you send an request to get processed by an ingest node (e.g. ?pipeline=mypipeline) and that node is not an ingest node (e.g. node.ingest: false) the request will be forwarded round-robbin style to ingest capable nodes. So, yes a coordinating node will load balance (assuming it is node.ingest: false), and you do not need to send requests directly to the ingest node.

You should be able to see the requests getting balanced by looking at the node stats seeing which nodes are processing your pipelines

GET _nodes/stats/ingest?filter_path=nodes.*.ingest