# Timeout in Logstash Elasticsearch Filter and Elasticsearch Output

**URL:** <https://discuss.elastic.co/t/timeout-in-logstash-elasticsearch-filter-and-elasticsearch-output/304253>\
**Category:** Logstash\
**Created:** [May 9, 2022, 12:46pm UTC](https://discuss.elastic.co/t/timeout-in-logstash-elasticsearch-filter-and-elasticsearch-output/304253 "2022-05-09T12:46:00Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![sagarpatel](https://avatars.discourse-cdn.com/v4/letter/s/ebca7d/32.png) [@sagarpatel](https://discuss.elastic.co/u/sagarpatel)\
**Post date:** [May 9, 2022, 12:46pm UTC](https://discuss.elastic.co/t/timeout-in-logstash-elasticsearch-filter-and-elasticsearch-output/304253/1 "2022-05-09T12:46:00Z")

</div>

Elasticsearch version: **7.16** (running as elastic cloud)  
Logstash version: **7.16**

I am using `http_poller` input plugin which is scheduled every 15 mins. Based on the http\_poller API response I need to execute Elasticsearch query.

For executing Elasticsearch query, I am using Elasticsearch Filter plugins and it is executed the first time without issue, but after second run it is throwing below error:

```auto
[2022-05-09T11:34:46,738][WARN][logstash.filters.elasticsearch][logs][9c5fb8a0078cad1be396fedd387eb8680d72086b85be9efe15e6893ce2e73332] Failed to query elasticsearch for previous event {:index=>"logs-xx-prod_xx", :error=>"Read timed out"}

```

Aslo, it is throwing below error for Elasticsearch Output filter from onwards second run:

```auto
[2022-05-09T11:35:17,236][WARN][logstash.outputs.elasticsearch][logs][8850a096b09c55eca7744c74cb4821d3f6e42a3e87a464228013b22ea1f0d576] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [https://elastic:xxxxxx@test.westeurope.azure.elastic-cloud.com:9243/][Manticore::SocketException] Connection reset by peer: socket write error {:url=>https://elastic:xxxxxx@test.westeurope.azure.elastic-cloud.com:9243/, :error_message=>"Elasticsearch Unreachable: [https://elastic:xxxxxx@test.westeurope.azure.elastic-cloud.com:9243/][Manticore::SocketException] Connection reset by peer: socket write error", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}
[2022-05-09T11:35:17,236][ERROR][logstash.outputs.elasticsearch][logs][8850a096b09c55eca7744c74cb4821d3f6e42a3e87a464228013b22ea1f0d576] Attempted to send a bulk request but Elasticsearch appears to be unreachable or down {:message=>"Elasticsearch Unreachable: [https://elastic:xxxxxx@test.westeurope.azure.elastic-cloud.com:9243/][Manticore::SocketException] Connection reset by peer: socket write error", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :will_retry_in_seconds=>2}
[2022-05-09T11:35:19,236][ERROR][logstash.outputs.elasticsearch][logs][8850a096b09c55eca7744c74cb4821d3f6e42a3e87a464228013b22ea1f0d576] Attempted to send a bulk request but there are no living connections in the pool (perhaps Elasticsearch is unreachable or down?) {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError, :will_retry_in_seconds=>4}
[2022-05-09T11:35:19,377][WARN][logstash.outputs.elasticsearch][logs] Restored connection to ES instance {:url=>"https://elastic:xxxxxx@test.westeurope.azure.elastic-cloud.com:9243/"}

```

I have configured Logstash pipeline from Kibana as using the centralized pipeline management of the ES 7.16 version.

I have tried below configuration, but seems like not a single configuration is working.

- Changed `Pipeline batch size` value to `100` then `50` then `25`

- `pipeline workers` is set to 1

- set `validate_after_inactivity` to `0` and try diffrent value as well in Elasticsearch output plugin.

- tried various `timeout` value like `100`, `180`, `200`, `600` etc.

- Previously i was setting custom document id using `document_id` param that also disable now.

One of the strange behavior I have noticed is that, document count are increased in ES index even after above error.

Also, there is no option to set `timeout` in the Elasticsearch filter plugin. Because when I tried to set `timeout` it throws error that "timeout param is not supported".

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 6, 2022, 12:46pm UTC](https://discuss.elastic.co/t/timeout-in-logstash-elasticsearch-filter-and-elasticsearch-output/304253/2 "2022-06-06T12:46:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.