Logstash failing to connect with Elasticsearch and file input plugin crashes

Elastic Stack Logstash
1

I am trying to read a .txt file into Logstash and then grok parse it into ES.

I am encountering this error:

{:timestamp=>"2015-11-24T20:46:44.957000+0000", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::File type=>"dailyreport", path=>["/usr/local/sparkexec/report.txt"], start_position=>"end", delimiter=>"\n">\n Error: undefined method `+' for nil:NilClass", :level=>:error}

Logstash config:

input {
file{
type => "dailyreport"
path => ["/usr/local/sparkexec/report.txt"]
start_position => "beginning"
sincedb_path => "/var/log/logstash/.sincedb"
}
}

filter{
if [type]=="dailyreport"{
grok{
match => { "message" => "%{DATA:lineItem}:%{NUMBER:value}"
}
}
mutate{
strip => ["lineItem"]
}
}
if "_grokparsefailure" in [tags]{
drop{}
}
}

output {
elasticsearch {
host => "10.2.1.179"
protocol => "http"
index => "report-%{+YYYY.MM.dd}"
}
}

Appreciate any guidance

2

Do you have a complete stacktrace or is that all you get? Which version of Logstash?

start_position=>"end"

Huh, but your configuration says start_position => "beginning"? Are you sure you're getting the right configuration (starting Logstash with --debug will tell you exactly what Logstash gets from the configuration files).

3

The version of Logstash is 1.5.1
I get the same failure even when I change the start_position => "beginning"

I don't have a complete stack trace as of now.

4

Here is the stack trace -

5

Here we go:

{:timestamp=>"2015-11-24T22:34:50.766000+0000", :message=>"A plugin had an unrecoverable error. Will restart this plugin.
  Plugin: <LogStash::Inputs::File type=>\"dailyreport\", path=>[\"/usr/local/sparkexec/report.txt\"], start_position=>\"beginning\", delimiter=>\"\
\">
  Error: undefined method `+' for nil:NilClass
  Exception: NoMethodError
  Stack: /opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.3/lib/filewatch/tail.rb:183:in `_read_file'
org/jruby/RubyArray.java:1613:in `each'
/opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.3/lib/filewatch/tail.rb:181:in `_read_file'
org/jruby/RubyKernel.java:1511:in `loop'
/opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.3/lib/filewatch/tail.rb:177:in `_read_file'
/opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.3/lib/filewatch/tail.rb:84:in `subscribe'
/opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.3/lib/filewatch/watch.rb:90:in `each'
org/jruby/RubyArray.java:1613:in `each'
/opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.3/lib/filewatch/watch.rb:75:in `each'
/opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.3/lib/filewatch/watch.rb:117:in `subscribe'
/opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.3/lib/filewatch/tail.rb:75:in `subscribe'
/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-file-0.1.10/lib/logstash/inputs/file.rb:136:in `run'
/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.1-java/lib/logstash/pipeline.rb:176:in `inputworker'
/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.1-java/lib/logstash/pipeline.rb:170:in `start_input'", :level=>:error, :file=>"logstash/pipeline.rb", :line=>"181", :method=>"inputworker"}

This was fixed in Logstash 1.5.2. See https://github.com/elastic/logstash/issues/3473.