# Logstash pipeline to pipeline losing events when queue is full

**URL:** <https://discuss.elastic.co/t/logstash-pipeline-to-pipeline-losing-events-when-queue-is-full/238545>\
**Category:** Logstash\
**Created:** [June 24, 2020, 6:34pm UTC](https://discuss.elastic.co/t/logstash-pipeline-to-pipeline-losing-events-when-queue-is-full/238545 "2020-06-24T18:34:06Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![wizardofc0dz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wizardofc0dz/32/64254_2.png) [@wizardofc0dz](https://discuss.elastic.co/u/wizardofc0dz)\
**Post date:** [June 24, 2020, 6:34pm UTC](https://discuss.elastic.co/t/logstash-pipeline-to-pipeline-losing-events-when-queue-is-full/238545/1 "2020-06-24T18:34:06Z")

</div>

Hello,

I have configured pipeline-to-pipeline with output isolator configuration in Logstash version 7.4.1 to send logs to local and remote Elasticsearch. Both pipelines, local and remote output, have been configured with persistent queues.

The configuration works fine. If the connection to remote fails, the events are being queued and the local events are being sent to local elasticsearch but the problem is when the remote queue is full, Logstash starts droping the events on both pipelines.

Here is an example of the pipelines configuration

**pipeline esLocal**

```auto
    input {
    pipeline {
        address => "esLocal" 
    }
    }
    filter {
    }
    output {
    elasticsearch {
    		hosts => ["localhost:9200"]
    		index => "%{[@metadata][indexName]}-%{+yyyy.ww}"
        user => "elastic"
        password => "xxxxxxxxx"
    	}
    }

```

**pipeline esRemote**

```auto
    input {
        pipeline {
            address => "esRemote" 
        }
    }
    filter {
    }
    output {

    	elasticsearch {
    		hosts => ["192.168.100.100:9200"]
    		index => "%{[@metadata][indexName]}-%{+yyyy.ww}"
            user => "elastic"
            password => "xxxxxxxxx"
    	}
    }

```

**input pipeline**

```auto
    input{

    }
    output {
    	pipeline { send_to => [esLocal,esRemote] }
    }

```

I have also test sending the output like this:

```auto
    output {
    	pipeline { send_to => esLocal }
        pipeline { send_to => esRemote}
    }

```

but still experiencing the same problem.

Should i have to use the clone filter on the esRemote to prevent the events being dropped on the local pipeline or any other suggestion to solve this problem?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 22, 2020, 6:37pm UTC](https://discuss.elastic.co/t/logstash-pipeline-to-pipeline-losing-events-when-queue-is-full/238545/2 "2020-07-22T18:37:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.