Logstash v2.1.3: A plugin had an unrecoverable error

Elastic Stack Logstash
1

Hi,
I push windows eventlog using winlogbeat-dev(Version: 5.0.0-nightly2bfc1c4) to logstash v2.1.3,the logstash configuration is

input{
beats {
add_field =>["itcategory","it_security_windowslog"]
port => 5150
}
}
(no filter)

output{
elasticsearch {
hosts => "localhost"
index => "it_security_windowslog-%{+YYYY.MM.dd}"
document_type => "windowslog"
}
}
}

but sometimes ,logstash crashed, logstash.err shows :
(I have increased the logstash memory from 500MB to 2048MB,no effect)

Error: Your application used more memory than the safety cap of 2048M.
Specify -J-Xmx####m to increase it (#### = cap size in MB).
Specify -w for full OutOfMemoryError stack trace

logstash.log shows:
{:timestamp=>"2016-06-22T07:35:04.563000+0800", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Beats add_field=>{"itcategory"=>"it_security_windowslog"}, port=>5150, codec=><LogStash::Codecs::Plain charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, congestion_threshold=>5, target_field_for_codec=>"message">\n Error: unable to create new native thread", :level=>:error}
{:timestamp=>"2016-06-22T07:35:05.566000+0800", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Beats add_field=>{"itcategory"=>"it_security_windowslog"}, port=>5150, codec=><LogStash::Codecs::Plain charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, congestion_threshold=>5, target_field_for_codec=>"message">\n Error: unable to create new native thread", :level=>:error}
{:timestamp=>"2016-06-22T07:35:06.568000+0800", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Beats add_field=>{"itcategory"=>"it_security_windowslog"}, port=>5150, codec=><LogStash::Codecs::Plain charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, congestion_threshold=>5, target_field_for_codec=>"message">\n Error: unable to create new native thread", :level=>:error}
{:timestamp=>"2016-06-22T07:35:07.571000+0800", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Beats add_field=>{"itcategory"=>"it_security_windowslog"}, port=>5150, codec=><LogStash::Codecs::Plain charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, congestion_threshold=>5, target_field_for_codec=>"message">\n Error: unable to create new native thread", :level=>:error}

anybody could help me? thanks.

2

Append:
there are other warning log in logsash before crash(A plugin had an unrecoverable error):

{:timestamp=>"2016-06-23T06:32:03.049000+0800", :message=>"Beats Input: Remote connection closed", :peer=>"10.42.31.8:23357", :exception=>#<Lumberjack::Beats::Connection::ConnectionClosed: Lumberjack::Beats::Connection::ConnectionClosed wrapping: Errno::ECONNRESET, Connection reset by peer - Connection reset by peer>, :level=>:warn}
{:timestamp=>"2016-06-23T06:43:36.955000+0800", :message=>"Beats Input: Remote connection closed", :peer=>"10.42.31.8:40070", :exception=>#<Lumberjack::Beats::Connection::ConnectionClosed: Lumberjack::Beats::Connection::ConnectionClosed wrapping: Errno::ECONNRESET, Connection reset by peer - Connection reset by peer>, :level=>:warn}
{:timestamp=>"2016-06-23T06:49:30.620000+0800", :message=>"Beats Input: Remote connection closed", :peer=>"10.42.31.8:61223", :exception=>#<Lumberjack::Beats::Connection::ConnectionClosed: Lumberjack::Beats::Connection::ConnectionClosed wrapping: Errno::ECONNRESET, Connection reset by peer - Connection reset by peer>, :level=>:warn}
{:timestamp=>"2016-06-23T06:55:22.942000+0800", :message=>"Beats Input: Remote connection closed", :peer=>"10.42.31.8:6350", :exception=>#<Lumberjack::Beats::Connection::ConnectionClosed: Lumberjack::Beats::Connection::ConnectionClosed wrapping: Errno::ECONNRESET, Connection reset by peer - Connection reset by peer>, :level=>:warn}
{:timestamp=>"2016-06-23T06:57:53.266000+0800", :message=>"Beats Input: Remote connection closed", :peer=>"10.42.31.8:50935", :exception=>#<Lumberjack::Beats::Connection::ConnectionClosed: Lumberjack::Beats::Connection::ConnectionClosed wrapping: Errno::ECONNRESET, Connection reset by peer - Connection reset by peer>, :level=>:warn}
{:timestamp=>"2016-06-23T06:58:13.399000+0800", :message=>"Beats Input: Remote connection closed", :peer=>"10.42.31.8:5069", :exception=>#<Lumberjack::Beats::Connection::ConnectionClosed: Lumberjack::Beats::Connection::ConnectionClosed wrapping: Errno::ECONNRESET, Connection reset by peer - Connection reset by peer>, :level=>:warn}
{:timestamp=>"2016-06-23T07:00:32.039000+0800", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Beats add_field=>{"itcategory"=>"it_p3sfcs_windowslog"}, port=>5159, codec=><LogStash::Codecs::Plain charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, congestion_threshold=>5, target_field_for_codec=>"message">\n Error: unable to create new native thread", :level=>:error}
{:timestamp=>"2016-06-23T07:00:33.042000+0800", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Beats add_field=>{"itcategory"=>"it_p3sfcs_windowslog"}, port=>5159, codec=><LogStash::Codecs::Plain charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, congestion_threshold=>5, target_field_for_codec=>"message">\n Error: unable to create new native thread", :level=>:error}
{:timestamp=>"2016-06-23T07:00:34.045000+0800", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Beats add_field=>{"itcategory"=>"it_p3sfcs_windowslog"}, port=>5159, codec=><LogStash::Codecs::Plain charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, congestion_threshold=>5, target_field_for_codec=>"message">\n Error: unable to create new native thread", :level=>:error}
{:timestamp=>"2016-06-23T07:00:35.048000+0800", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Beats add_field=>{"itcategory"=>"it_p3sfcs_windowslog"}, port=>5159, codec=><LogStash::Codecs::Plain charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, congestion_threshold=>5, target_field_for_codec=>"message">\n Error: unable to create new native thread", :level=>:error}