# Logstash exception in pipeline worker

**URL:** <https://discuss.elastic.co/t/logstash-exception-in-pipeline-worker/87004>\
**Category:** Logstash\
**Created:** [May 24, 2017, 3:36pm UTC](https://discuss.elastic.co/t/logstash-exception-in-pipeline-worker/87004 "2017-05-24T15:36:41Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Singard123](https://avatars.discourse-cdn.com/v4/letter/s/df788c/32.png) [@Singard123](https://discuss.elastic.co/u/Singard123)\
**Post date:** [May 24, 2017, 3:36pm UTC](https://discuss.elastic.co/t/logstash-exception-in-pipeline-worker/87004/1 "2017-05-24T15:36:41Z")

</div>

I'm getting this error when I publish the events in filebeat:

> 2017/05/24 15:20:00.468812 sync.go:85: ERR Failed to publish events caused by: read tcp 10.0.3.150:56617-\>167.114.251.27:19130: i/o timeout  
> 2017/05/24 15:20:00.468844 single.go:91: INFO Error publishing events (retrying): read tcp 10.0.3.150:56617-\>167.114.251.27:19130: i/o timeout  
> 2017/05/24 15:20:17.127813 metrics.go:39: INFO Non-zero metrics in the last 30s: libbeat.logstash.publish.read\_errors=1 libbeat.logstash.publish.write\_errors=1 libbeat.logstash.published\_but\_not\_acked\_events=4

Causing Kibana wouldn't be able to get the logs. Here is error message from logstash:

> [[main]\>worker11] ERROR logstash.pipeline - Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {"exception"=\>"-1", "backtrace"=\>["java.util.ArrayList.elementData(ArrayList.java:418)", "java.util.ArrayList.remove(ArrayList.java:495)", "org.logstash.FieldReference.parse(FieldReference.java:37)", "org.logstash.PathCache.cache(PathCache.java:37)", "org.logstash.PathCache.isTimestamp(PathCache.java:30)", "org.logstash.ext.JrubyEventExtLibrary$RubyEvent.ruby\_set\_field(JrubyEventExtLibrary.java:122)",

We just upgraded it the new version of ELK stack, and the pipeline has never broken before in the old version. Is this a known bug or is there something I need to change with my filter in order to work with ELK 5?

logstash filter (as you can see it's really simple. Therefore, I don't understand why it breaks:

```auto
filter {
  kv { 
    trim_key => "<>\[\],`\."
    remove_field => ["\\%{some_field}", "{%{some_field}"]
    include_brackets => false
  }

  #prune {
  # blacklist_names => ["%{[^}]+}."]
  #}

  #ruby {
  # code => "
  # hashes_to_remove = []
  # event_hash = event.to_hash
  # event_hash.keys.each { |k| event_hash[k.gsub('.', '_')] = event_hash[k]; hashes_to_remove << k if k.include?('.') }
  # hashes_to_remove.each { |field| event_hash.delete(field) }
  # event.overwrite(LogStash::Event.new(event_hash))
  # "
  #}
}

```

---

<div class="post-metadata">

**Author:** ![orhiee](https://avatars.discourse-cdn.com/v4/letter/o/eb9ed0/32.png) [@orhiee](https://discuss.elastic.co/u/orhiee)\
**Post date:** [May 25, 2017, 4:23pm UTC](https://discuss.elastic.co/t/logstash-exception-in-pipeline-worker/87004/2 "2017-05-25T16:23:03Z")

</div>

assuming you have multiple configs can you run them one by one in debug mode and look at the outputs and paste it here

ps. my guess is you are missing a plugin (you have to reinstall after upgrade)

---

<div class="post-metadata">

**Author:** ![Singard123](https://avatars.discourse-cdn.com/v4/letter/s/df788c/32.png) [@Singard123](https://discuss.elastic.co/u/Singard123)\
**Post date:** [May 25, 2017, 4:31pm UTC](https://discuss.elastic.co/t/logstash-exception-in-pipeline-worker/87004/3 "2017-05-25T16:31:53Z")

</div>

what configs? logstash or filebeat?

---

<div class="post-metadata">

**Author:** ![orhiee](https://avatars.discourse-cdn.com/v4/letter/o/eb9ed0/32.png) [@orhiee](https://discuss.elastic.co/u/orhiee)\
**Post date:** [May 25, 2017, 7:19pm UTC](https://discuss.elastic.co/t/logstash-exception-in-pipeline-worker/87004/4 "2017-05-25T19:19:33Z")

</div>

let me explain a bit more, to make it easier to debug&update (my opinion) i use multiple logstash configs (windows, syslog, firewall ....).

if you are also using multiple logstash configs run them one by one in debug mode, and find the broken one 🙂 and share the error 🙂 and config

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 22, 2017, 7:19pm UTC](https://discuss.elastic.co/t/logstash-exception-in-pipeline-worker/87004/5 "2017-06-22T19:19:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.