I'm currently facing an issue with Elasticsearch 8.7, specifically with the integration of Nginx logs and custom ingest pipelines. I have successfully installed Fleet Server with Elastic Agent, along with the Nginx integration. The default ingest pipeline provided by Nginx integration processes the default access.log and nginx.log files. However, my requirement is to use custom Nginx logs that contain additional custom fields.
To address this requirement, I have created a custom ingest pipeline from the advanced settings of the Nginx integration. Despite creating the custom pipeline, it seems that the integration still uses the default Nginx ingest pipeline, neglecting the one I have created.
I kindly request your assistance in resolving this issue. Specifically, I need guidance on how to remove the default Nginx ingest pipeline from the ingestion process, ensuring that only my custom pipeline is used.
Any insights or suggestions would be greatly appreciated. Thank you in advance for your help!