Logstash 2.3.0 Startup Error

Elastic Stack Logstash
#1

Hello!

I had been using this configuration for Logstash 2.2.0 until this morning, which worked perfectly. But after an upgrade to 2.3.0, I get the following error message on the console -

An unexpected error occurred! {:error=>#<NoMethodError: undefined method config_valid?' for LogStash::Pipeline:Class>, :class=>"NoMethodError", :backtrace=>["/home/bits/logstash-2.3.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.0-java/lib/logstash/agent.rb:169:inexecute'", "/home/bits/logstash-2.3.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.0-java/lib/logstash/runner.rb:94:in run'", "org/jruby/RubyProc.java:281:incall'", "/home/bits/logstash-2.3.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.0-java/lib/logstash/runner.rb:99:in run'", "org/jruby/RubyProc.java:281:incall'", "/home/bits/logstash-2.3.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/task.rb:24:in `initialize'"], :level=>:warn}

Here's my config file -

input {
file {
path => "/home/bits/Server_Code/TempLight.log"
type => "sample"
start_position => "beginning"

}
}

filter {
grok { match => { "message" => "%{DAY:day}\s%{MONTH:month}\s%{MONTHDAY:monthday}\s%{YEAR:year}\s%{TIME:time}\sGMT(?[+-]\d\d\d\d)\s([^)]+)\s%{NUMBER:temp}\s%{NUMBER:light}\s%{GREEDYDATA:room}"} }

}

output {

elasticsearch {
index => "logstash-templightlogs"

}

stdout{}

}

#2

Is that all the error was?

#3

Yes, that was all.

#4

I tried the same config on Logstash 2.2.3 as opposed to Logstash 2.3.0 and here's the error I'm seeing on the console.

Screenshot.png1364×768 115 KB

#5

Update : I hosted Elasticsearch publicly on '172.25.3.31' and port '9200'. So I updated my Logstash config to reflect this. Now I see Logstash Startup Completed with no errors but I don't see any of the logs in the console, which means they aren't going through to the ES server. Here's the config file -

input {
file {
path => "/home/bits/Server_Code/TempLight.log"
type => "sample"
start_position => "beginning"

}
}

filter {
grok { match => { "message" => "%{DAY:day}\s%{MONTH:month}\s%{MONTHDAY:monthday}\s%{YEAR:year}\s%{TIME:time}\sGMT(?[+-]\d\d\d\d)\s([^)]+)\s%{NUMBER:temp}\s%{NUMBER:light}\s%{GREEDYDATA:room}"} }

}

output {

elasticsearch {
index => "logstash-templightlogs"
hosts => "172.25.3.31:9200"

}

stdout{}

}

#6

I think this might be a known issue we are working on fixing. Try downgrading to 2.2.X and see if you get it.

#7

I tried this on Logstash 2.2.3 as well. Same issue. It says "Logstash startup completed" but nothing after that. Anything I can change in my config?

#8

Yes, I have tried this on Logstash 2.2.3 as well. Same error. Anything I
can change in my config?

#9

Ah see I didn't open the image because they are usually too hard to read. It'd better if you post the text of console output like that, not the picture :slight_smile:

What you are seeing is normal, it means LS has started and is waiting on new data in the file you specified in the input.
This is not an error.

#10

"Connection refused {:class=>"Manticore::SocketException", :level=>:error}" is what the console says for Logstash-2.3.0. The same line is repeated 8 times for Logstash-2.2.3.

My config file is as follows -

input {
file {
path => "/home/bits/Server_Code/TempLight.log"
type => "sample"
start_position => "beginning"
sincedb_path => "/home/bits/logstash-2.3.0/file"

}
}

filter {
grok { match => { "message" => "%{DAY:day}\s%{MONTH:month}\s%{MONTHDAY:monthday}\s%{YEAR:year}\s%{TIME:time}\sGMT(?[+-]\d\d\d\d)\s([^)]+)\s%{NUMBER:temp}\s%{NUMBER:light}\s%{GREEDYDATA:room}"} }

}

output {

elasticsearch {
index => "logstash-templightlogs"
hosts => "172.25.3.31:9200"

}

stdout{}

}

#11

Hosts is an array, so I'd fix that.
Make sure you can connect to ES from the LS host too.

#12

Right, so I've made hosts an array like you said. I must say here that I did not need to mention the hosts parameter when I was using Logstash-2.2.0 until two days ago. I'm surprised it is needed here.

I see the following on my console -

Settings: Default pipeline workers: 8
Pipeline main started

However, none of the logs are actually getting stashed after this as is supposed to happen. Pasting my config file again here below -

input {
file {
path => "/home/bits/Server_Code/TempLight.log"
type => "sample"
start_position => "beginning"
sincedb_path => "/home/bits/logstash-2.3.0/files"

}
}

filter {
grok { match => { "message" => "%{DAY:day}\s%{MONTH:month}\s%{MONTHDAY:monthday}\s%{YEAR:year}\s%{TIME:time}\sGMT(?[+-]\d\d\d\d)\s([^)]+)\s%{NUMBER:temp}\s%{NUMBER:light}\s%{GREEDYDATA:room}"} }

}

output {

elasticsearch {
index => "logstash-templightlogs"
hosts => ['172.25.3.31:9200']

}

stdout{}

}

#13

Then it's probably a sincedb issue, have you delete the old ones?

It's in the docs, has been for a while!

#14

Got it running with Logstash-2.2.2. No sweat. Even deletion of the old ones did not help with the newer versions of Logstash.

#15