DLQ setup issue for specific application in Logstash pipeline

Elastic Stack Logstash
1

We have a single Logstash pipeline that listens on port 5044 and receives data from multiple Beats sources. Filebeat is installed on one of the source servers, and we want to configure a Dead Letter Queue (DLQ) only for one specific application’s logs, not for all incoming data.

The issue is that enabling DLQ at the pipeline level causes data from all Beats sources to be captured, whereas our requirement is to isolate DLQ for just one application. Creating separate pipelines is not feasible in our environment, as that would require additional ports and firewall connectivity requests.

We need guidance on how to configure DLQ selectively for a single application within the existing pipeline setup.

2

This is not possible, once enabled for a pipeline, DLQ will work for any event.

What you may try is to use Pipeline-to-pipeline communicaton to split the pipeline into multiple pipelines and enable DLQ only on the pipeline processing the application you want.

Your main pipeline would still listen on 5044, but you would change it to distribute the events between two internal pipelines.

And in the one that would process the desired application, you could enable the DLQ.