# Keep Logstash from Crashing **URL:** \ **Category:** Logstash\ **Created:** [June 22, 2017, 5:45am UTC](https://discuss.elastic.co/t/keep-logstash-from-crashing/90392 "2017-06-22T05:45:37Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![allenfin](https://avatars.discourse-cdn.com/v4/letter/a/9d8465/32.png) [@allenfin](https://discuss.elastic.co/u/allenfin)\ **Post date:** [June 22, 2017, 5:45am UTC](https://discuss.elastic.co/t/keep-logstash-from-crashing/90392/1 "2017-06-22T05:45:37Z")
Been having a hell of a time with logstash. Right now I setup a few different nginx instances to pump access and error logs into logstash to funnel into elasticsearch, but now I'm getting crashes and I have no idea how to fix it: ``` [2017-06-22T05:37:41,258][FATAL][logstash.runner] An unexpected error occurred! {:error=>#, :backtrace=>["org/logstash/ext/JrubyEventExtLibrary.java:202:in `sprintf'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.3.0-java/lib/logstash/outputs/elasticsearch/common.rb:153:in `event_action_params'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.3.0-java/lib/logstash/outputs/elasticsearch/common.rb:40:in `event_action_tuple'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.3.0-java/lib/logstash/outputs/elasticsearch/common.rb:34:in `multi_receive'", "org/jruby/RubyArray.java:2414:in `map'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.3.0-java/lib/logstash/outputs/elasticsearch/common.rb:34:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator_strategies/shared.rb:13:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator.rb:47:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:407:in `output_batch'", "org/jruby/RubyHash.java:1342:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:406:in `output_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:352:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:317:in `start_workers'"]} ``` Is there a way for logstash to just flatout ignore anything that doesn't have a timestamp? I used the NGXINX templates from here: [https://www.elastic.co/guide/en/logstash/5.4/logstash-config-for-filebeat-modules.html#\_nginx\_error\_logs](https://www.elastic.co/guide/en/logstash/5.4/logstash-config-for-filebeat-modules.html#_nginx_error_logs) ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 22, 2017, 5:49am UTC](https://discuss.elastic.co/t/keep-logstash-from-crashing/90392/2 "2017-06-22T05:49:56Z")
Your log is mangled and doesn't show the full error message. What comes after "An unexpected error occurred! {:error=\>#"? Also, what does your Logstash configuration look like? ---
**Author:** ![allenfin](https://avatars.discourse-cdn.com/v4/letter/a/9d8465/32.png) [@allenfin](https://discuss.elastic.co/u/allenfin)\ **Post date:** [June 22, 2017, 6:33am UTC](https://discuss.elastic.co/t/keep-logstash-from-crashing/90392/3 "2017-06-22T06:33:29Z")
fixed up the logged out error. The link i posted above contains both setups for nginx access and error logs. The only things I changed were the elasticsearch and port properties. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 22, 2017, 8:53am UTC](https://discuss.elastic.co/t/keep-logstash-from-crashing/90392/4 "2017-06-22T08:53:30Z")
The problem is that you're using the default `index` value for your elasticsearch output, "logstash-%{+YYYY.MM.dd}", which requires that the `@timestamp` field exists. Can you temporarily replace your elasticsearch output with a `stdout { codec => rubydebug }` output so we can see exactly what the events look like? The error indicates that the event that causes the crash lacks a `@timestamp` field but the date filter should make sure that field exists. ---
**Author:** ![allenfin](https://avatars.discourse-cdn.com/v4/letter/a/9d8465/32.png) [@allenfin](https://discuss.elastic.co/u/allenfin)\ **Post date:** [June 28, 2017, 8:24am UTC](https://discuss.elastic.co/t/keep-logstash-from-crashing/90392/5 "2017-06-28T08:24:22Z")
ah... very nice 🙂 thanks a lot for that! i just removed the timestamp. You would figure that the timestamp would have been caught or ignored or something. Just changing the name of teh index seemed to have worked well. next problem is disk space! 🙂 ---
**Author:** ![dedemorton](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dedemorton/32/84409_2.png) [@dedemorton](https://discuss.elastic.co/u/dedemorton)\ **Post date:** [July 12, 2017, 8:03pm UTC](https://discuss.elastic.co/t/keep-logstash-from-crashing/90392/6 "2017-07-12T20:03:42Z")
Sounds like you've resolved this issue. But for other users who encounter this problem, another work around is to use `add_field` instead of `rename` in your `mutate` filter. So use this: ```auto mutate { add_field => { "read_timestamp" => "%{@timestamp}" } } ``` Instead of: ```auto mutate { rename => { "@timestamp" => "read_timestamp" } } ``` The config in the docs should work with this change. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 9, 2017, 8:03pm UTC](https://discuss.elastic.co/t/keep-logstash-from-crashing/90392/7 "2017-08-09T20:03:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.