ELASTICSEARCH_17 - Could not index '1' records: listener timeout after waiting for [30000] ms

[senix]

We're using streamsets to send messages to Elasticsearch and are consistently getting the following error:

ELASTICSEARCH_17 - Could not index '1' records: listener timeout after waiting for [30000] ms

We've tried various approaches in streamsets and elasticsearch, such as:

• Adding some additional http parameters(timeout)

• Added Additional properties(Connection timeout, socket timeout, max retry timeout)

• Deleting and recreating index

• Increasing number of shards in Elasticsearch index

Anyone have an idea on how to remedy this?

[leandrojmp (Leandro Pereira)]

Hello and welcome!

You need to provide more context about your issue.

First, what is streamsets? How are you sending your data to Elasticsearch?

[senix]

Thank you! Streamsets is a tool used to interact with Kafka to create data pipelines. So in this case, we're consuming from a Kafka topic and writing to Elasticsearch in a stage where we specify the URI, index, etc.

[Christian_Dahlqvist]

Which version of Elasticsearch are you indexing into?

How is the cluster configured, e.g. with respect to security?

How are you configuring the connection?

[senix]

Version 7.17.3.

We're accessing Elasticsearch via basic auth

[Christian_Dahlqvist]

Can you connect to Elasticsearch via curl from the machine where Streamset runs? If you are, I suspect the issue is with Streamset, which we can not help with here. If you are not able to connect it could be a network or Elasticsearch issue.