# Logstash stops sending docs to elasticsearch, starts over after restart of process

**URL:** <https://discuss.elastic.co/t/logstash-stops-sending-docs-to-elasticsearch-starts-over-after-restart-of-process/174659>\
**Category:** Logstash\
**Created:** [March 31, 2019, 5:15am UTC](https://discuss.elastic.co/t/logstash-stops-sending-docs-to-elasticsearch-starts-over-after-restart-of-process/174659 "2019-03-31T05:15:29Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![yossib](https://avatars.discourse-cdn.com/v4/letter/y/a6a055/32.png) [@yossib](https://discuss.elastic.co/u/yossib)\
**Post date:** [March 31, 2019, 5:15am UTC](https://discuss.elastic.co/t/logstash-stops-sending-docs-to-elasticsearch-starts-over-after-restart-of-process/174659/1 "2019-03-31T05:15:30Z")

</div>

elasticsearch is running on a 3 node cluster, logstash and filebeat are running on a single node  
all version are: 6.5.1

logs are being sent to elasticsearch for short periods (~2 hours), then they stop... only after restart logstash they resume

---

<div class="post-metadata">

**Author:** ![sagar\_arora1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sagar_arora1/32/43155_2.png) [@sagar\_arora1](https://discuss.elastic.co/u/sagar_arora1)\
**Post date:** [March 31, 2019, 9:53am UTC](https://discuss.elastic.co/t/logstash-stops-sending-docs-to-elasticsearch-starts-over-after-restart-of-process/174659/2 "2019-03-31T09:53:18Z")

</div>

Can you share logstash logs , are there any event rejections in logs

---

<div class="post-metadata">

**Author:** ![yossib](https://avatars.discourse-cdn.com/v4/letter/y/a6a055/32.png) [@yossib](https://discuss.elastic.co/u/yossib)\
**Post date:** [March 31, 2019, 10:16am UTC](https://discuss.elastic.co/t/logstash-stops-sending-docs-to-elasticsearch-starts-over-after-restart-of-process/174659/3 "2019-03-31T10:16:59Z")

</div>

**lines of log since restart:**

[2019-03-31T09:55:18,788][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600}  
[2019-03-31T09:55:19,236][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2049, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,202][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,313][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,313][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,312][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,313][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,313][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,313][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,333][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,333][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,333][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,333][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,333][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,334][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,345][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,345][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,345][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,346][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,346][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,346][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
[2019-03-31T09:55:20,350][WARN][logstash.codecs.netflow] Can't (yet) decode flowset id 260 from source id 2065, because no template to decode it with has been received. This message will usually go away after 1 minute.  
...  
[2019-03-31T10:03:12,881][INFO][logstash.outputs.file] Opening file {:path=\>"/logstash/parse\_failure/trustwave\_waf.log"}  
[2019-03-31T10:04:52,067][INFO][logstash.outputs.file] Closing file /logstash/parse\_failure/trustwave\_waf.log  
...

**I can move into debug mode...**

---

<div class="post-metadata">

**Author:** ![yossib](https://avatars.discourse-cdn.com/v4/letter/y/a6a055/32.png) [@yossib](https://discuss.elastic.co/u/yossib)\
**Post date:** [April 2, 2019, 8:14am UTC](https://discuss.elastic.co/t/logstash-stops-sending-docs-to-elasticsearch-starts-over-after-restart-of-process/174659/4 "2019-04-02T08:14:48Z")

</div>

No need to move into debug, issue with filebeat that caused logstash to get strangled with worker threads...

filebeat was including logs for .log.\* so, every time a new GZIP'd file was created it opened a new pipeline on logstash... logstash top was extremely high due to that

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 30, 2019, 8:14am UTC](https://discuss.elastic.co/t/logstash-stops-sending-docs-to-elasticsearch-starts-over-after-restart-of-process/174659/5 "2019-04-30T08:14:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
