Do i need to use the ingest node instead of hot nodes in logstash output

Elastic Stack Elasticsearch
1

Hello i have dedicated nodes of ml,coordinate, and ml and hot

currently i am using the hot nodes in the out put of the logstash but i want to know shouldn't i remove the ingest role from hot and use ingest node in out of logstash, as ingest is meant to be for ingestion or i should coordinate nodes?

i have ingest pipelines for apm customise, and others but for logstash do you think approach is correct

2

Hi @kkumar123

What version of the stack?

What is the current architecture of the stack?

You do not need dedicated ingest / coordinator nodes unless you have a specific issues you are trying to solve.

Do you have a specific issue you are trying to solve? Are your hot node CPU bound?

There are specific use cases for ingest nodes but they can also cause complexity and bottleneck if not deployed correctly.

3

Hello , i am using the elk version v 9.3.4.

  • 3 master dedicated,
  • 2 coordinating nodes
  • 3 ingest dedicated
  • 6 Hot nodes (ingest, data_content, data_hot)
  • 2 frozen dedicated (search snapshot attached)
  • 2 ml (transform,)

image
image276×399 28.4 KB

i have huge apm data every day like 5 TB , i want to reduce the hot nodes load so here my question ?

what should be in output of fleet server settings output as the apm server setup and managed via fleet server policy .

image
image1549×328 30.6 KB

and currently i have added ingest node ip's in the Fleet > Settings > Output with port IP:9200

My observation:

When i removed the ingest role from the each hot node and restarted.

the apm data dropped to 0 .

why is that happening i am not able to figure-out can anyone let me know what should i do? should i coordinate nodes in output or keep the ingest nodes in Fleet > Output or Hot nodes and remove the ingest role from there.

can any one guide.