Ingest Role - purpose and is it required?

Elastic Stack Elasticsearch
1

I have upgraded my elasticsearch from version 7 to 8 (installed on kubernetes with bitnami helm chart) and now my fluentbit has started throwing exceptions of "illegal_state_exception","There are no ingest nodes in this cluster.."

Looking over the documentation again: Ingest pipelines | Elasticsearch Guide [8.14] | Elastic
From what I gather ingest pipelines are optional. If I have no pre-indexing processing of my own, I can and should disable the role- which I did.
However it seems there are some pre-defined ingest pipeline defined (which I cannot remove) and that error that is thrown from fluentbit- so I am left quite confused.

Are ingest nodes required and if they are why is it even an option to disable them?

2

How are you sending your data? Can you share the configuration you are using to send your data including the index name? Also, share the full error you are getting.

Ingest nodes are required when you use ingest pipelines, but there are too many internal things in Elasticsearch now that uses ingest pipelines that I think that Elastic should change the documentation to make it a requirement.

In your case, using fluentbit I would not expect it to call any ingest pipeline, unless your index name is matching one of the default built-in templates.