Twitter Logstash pipeline stops after a certain period

[dihiaselma (Dihia LANASRI)]

Hello;

I have created a pipeline logstash to retrieve data from twitter, then insert these data into Elasticsearch.
It works fine for a certain period then, it stops without any error in logs.

I have just to restart it, and it works . then it stops again after a certain period....

Please, do you have any idea about this problem and how to fix it?

[Deh_Man]

I have a similar issue.

We are using Logstash (on Docker) to fetch some Twitter data which we'll then insert into OpenSearch. Everything was working fine until we noticed that the pipeline was not generating any new indices in OpenSearch. We checked the logs and we found 0 Errors.

Ironically, a temporary fix was to restart the Logstash Docker container which would automatically get things working again, until Logstash randomly stops fetching data and we'd have to restart it again, ...

I'm interested in knowing the source of the problem and how to fix the issue, rather than bypassing it by restarting the Logstash instance that we're running every time the problem occurs.

PS:

Logstash Docker Image opensearchproject/logstash-oss-with-opensearch-output-plugin:7.13.4

OpenSearch Docker Image opensearchproject/opensearch:1.2.0

[system]

OpenSearch/OpenDistro are AWS run products and differ from the original Elasticsearch and Kibana products that Elastic builds and maintains. You may need to contact them directly for further assistance.

(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns )