Ingest node doesn't process log

[UweW (Uwe)]

Hi,
logs per second are growing and I want to move the ingest load from the data nodes to ingest only nodes. I have added an additional Elasticsearch ingest only node but I can see in the ingest stats, that the processed counter stay 0 from the ingest only node. All logs are still processed by the ingest roles on the data nodes.
At least I suggest that some traffic/load will be handed by the new ingest node.
Any thing that I missed ?

ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master
xx.xx.94.10 39 73 15 3.83 3.25 3.02 cdhimrsw -
xx.xx.94.12 60 73 44 3.98 3.03 2.76 cdhimrsw *
xx.xx.90.26 4 63 0 0.00 0.01 0.00 i -
xx.xx.94.11 43 69 14 2.24 2.16 2.33 cdhimrsw -

4 node cluster, one of them Ingest only
2 filebeat server
~7000Events/sec
1 Kibana
Version 7.16.2

best regards
Uwe

[stephenb (Stephen Brown)]

You have to remove the ingest role from your data nodes... it is still on them that is why the ingests pipelines are still running on them...

xx.xx.94.10 39 73 15 3.83 3.25 3.02 cdhimrsw that i is ingest... you then will need to specifically set the node roles

See Here

Then you will need to point your ingest client (filebeat / logstash etc) to that ingest node...

[UweW (Uwe)]

last hint brings me to the right direction.
I have to add Ingest node to the output.Elasticsearch.hosts array and remove tha data nodes.
Now it works, thanks a lot

best regards
Uwe