# Broken after upgrade

**URL:** <https://discuss.elastic.co/t/broken-after-upgrade/37634>\
**Category:** Logstash\
**Created:** [December 20, 2015, 12:59am UTC](https://discuss.elastic.co/t/broken-after-upgrade/37634 "2015-12-20T00:59:35Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![DigiAngel](https://avatars.discourse-cdn.com/v4/letter/d/977dab/32.png) [@DigiAngel](https://discuss.elastic.co/u/DigiAngel)\
**Post date:** [December 20, 2015, 12:59am UTC](https://discuss.elastic.co/t/broken-after-upgrade/37634/1 "2015-12-20T00:59:35Z")

</div>

Topic says it....I upgraded to the latest version in the 1.5.6 from the ppa and now I am seeing this:

`{:timestamp=\>"2015-12-19T17:28:27.323000-0700", :message=\>"Got error to send bulk of actions: Cannot find Serializer for class: org.jruby.RubyObject", :level=\>:error}

{:timestamp=\>"2015-12-19T17:28:27.323000-0700", :message=\>"Failed to flush outgoing items", :outgoing\_count=\>1, :exception=\>"JrJackson::ParseError", :backtrace=\>["com/jrjackson/JrJacksonBase.java:78:in `generate'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/jrjackson-0.3.7/lib/jrjackson/jrjackson.rb:59:in`dump'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/multi\_json-1.11.2/lib/multi\_json/adapters/jr\_jackson.rb:20:in `dump'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/multi_json-1.11.2/lib/multi_json/adapter.rb:25:in`dump'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/multi\_json-1.11.2/lib/multi\_json.rb:136:in `dump'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-api-1.0.15/lib/elasticsearch/api/utils.rb:102:in`\_\_bulkify'", "org/jruby/RubyArray.java:2414:in `map'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-api-1.0.15/lib/elasticsearch/api/utils.rb:102:in`\_\_bulkify'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-api-1.0.15/lib/elasticsearch/api/actions/bulk.rb:82:in `bulk'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.1.0-java/lib/logstash/outputs/elasticsearch/protocol.rb:105:in`bulk'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.1.0-java/lib/logstash/outputs/elasticsearch.rb:548:in `submit'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.1.0-java/lib/logstash/outputs/elasticsearch.rb:547:in`submit'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.1.0-java/lib/logstash/outputs/elasticsearch.rb:572:in `flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.1.0-java/lib/logstash/outputs/elasticsearch.rb:571:in`flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.21/lib/stud/buffer.rb:219:in `buffer_flush'", "org/jruby/RubyHash.java:1342:in`each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.21/lib/stud/buffer.rb:216:in `buffer_flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.21/lib/stud/buffer.rb:193:in`buffer\_flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.21/lib/stud/buffer.rb:159:in `buffer_receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.1.0-java/lib/logstash/outputs/elasticsearch.rb:537:in`receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.6-java/lib/logstash/outputs/base.rb:88:in `handle'", "(eval):283:in`output\_func'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.6-java/lib/logstash/pipeline.rb:244:in `outputworker'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.6-java/lib/logstash/pipeline.rb:166:in`start\_outputs'"], :level=\>:warn}`

At this point logstash just fills up the log with this, stops receiving any input, and needs a kill -s 9 to get it to stop. Any hints at all would be lovely. Thank you.

---

<div class="post-metadata">

**Author:** ![german23](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/german23/32/11052_2.png) [@german23](https://discuss.elastic.co/u/german23)\
**Post date:** [December 21, 2015, 8:21am UTC](https://discuss.elastic.co/t/broken-after-upgrade/37634/2 "2015-12-21T08:21:44Z")

</div>

Hi,  
try add

--verbose

as startup option, than logstash should log the relevant logs that causes the "failed to flush" exceptions

---

<div class="post-metadata">

**Author:** ![DigiAngel](https://avatars.discourse-cdn.com/v4/letter/d/977dab/32.png) [@DigiAngel](https://discuss.elastic.co/u/DigiAngel)\
**Post date:** [December 21, 2015, 3:37pm UTC](https://discuss.elastic.co/t/broken-after-upgrade/37634/3 "2015-12-21T15:37:50Z")

</div>

Thanks....after packet capturing somewhere one of these strings causes this:

`<190>Dec 21 00:00:51 ids bro_conn #separator \x09 <190>Dec 21 00:00:51 ids bro_conn #set_separator , <190>Dec 21 00:00:51 ids bro_conn #empty_field (empty) <190>Dec 21 00:00:51 ids bro_conn #unset_field - <190>Dec 21 00:00:51 ids bro_conn #path conn <190>Dec 21 00:00:51 ids bro_conn #open 2015-12-21-00-00-51 <190>Dec 21 00:00:51 ids bro_conn #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig local_resp missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents <190>Dec 21 00:00:51 ids bro_conn #types time string addr port addr port enum string interval count count string bool bool count string count count count count set[string]`

This happens once a day when these logs rotate. We did not have this issue with 1.5.4, but not that we are on 1.5.6, this is an issue.

---

<div class="post-metadata">

**Author:** ![DigiAngel](https://avatars.discourse-cdn.com/v4/letter/d/977dab/32.png) [@DigiAngel](https://discuss.elastic.co/u/DigiAngel)\
**Post date:** [December 22, 2015, 4:06pm UTC](https://discuss.elastic.co/t/broken-after-upgrade/37634/4 "2015-12-22T16:06:07Z")

</div>

So after rolling back to 1.5.4 here's what I see in the logs:  
`{:timestamp=>"2015-12-21T16:58:50.564000-0700", :message=>"IP Field contained invalid IP address or hostname", :field=>"src_ip", :event=>#<LogStash::Event:0x1d3c8326 @metadata_accessors=#<LogStash::Util::Accessors:0xb86a4db @store={}, @lut={}>, @cancelled=false, @data={"message"=>"<190>Dec 21 23:58:25 dcids bro_conn #fields\tts\tuid\tid.orig_h\tid.orig_p\tid.resp_h\tid.resp_p\tproto\tservice\tduration\torig_bytes\tresp_bytes\tconn_state\tlocal_orig\tlocal_resp\tmissed_bytes\thistory\torig_pkts\torig_ip_bytes\tresp_pkts\tresp_ip_bytes\ttunnel_parents", "@version"=>"1", "@timestamp"=>"2015-12-21T23:58:25.480Z", "type"=>"Conn", "host"=>"x.x.x.x", "unixtime"=>"<190>Dec 21 23:58:25 dcids bro_conn #fields", "uid"=>"ts", "src_ip"=>"uid", "src_port"=>"id.orig_h", "dst_ip"=>"id.orig_p", "dst_port"=>"id.resp_h", "proto"=>"id.resp_p", "service"=>"proto", "duration"=>"service", "orig_bytes"=>"duration", "resp_bytes"=>"orig_bytes", "conn_state"=>"resp_bytes", "local_orig"=>"conn_state", "local_resp"=>"local_orig", "missed_bytes"=>"local_resp", "history"=>"missed_bytes", "orig_packts"=>"history", "orig_ip_bytes"=>"orig_pkts", "resp_packts"=>"orig_ip_bytes", "resp_ip_bytes"=>"resp_pkts", "t<snipped>`

Looks like indeed the #fields line caused this....interesting that 1.5.4 could handle the issue, but 1.5.6 could not.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:17am UTC](https://discuss.elastic.co/t/broken-after-upgrade/37634/5 "2017-07-06T05:17:32Z")

</div>


