Logstash 2.3.0 Startup Error

Elastic Stack Logstash
1

Hello!

I had been using this configuration for Logstash 2.2.0 until this morning, which worked perfectly. But after an upgrade to 2.3.0, I get the following error message on the console -

An unexpected error occurred! {:error=>#<NoMethodError: undefined method config_valid?' for LogStash::Pipeline:Class>, :class=>"NoMethodError", :backtrace=>["/home/bits/logstash-2.3.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.0-java/lib/logstash/agent.rb:169:inexecute'", "/home/bits/logstash-2.3.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.0-java/lib/logstash/runner.rb:94:in run'", "org/jruby/RubyProc.java:281:incall'", "/home/bits/logstash-2.3.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.0-java/lib/logstash/runner.rb:99:in run'", "org/jruby/RubyProc.java:281:incall'", "/home/bits/logstash-2.3.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/task.rb:24:in `initialize'"], :level=>:warn}

Here's my config file -

input {
file {
path => "/home/bits/Server_Code/TempLight.log"
type => "sample"
start_position => "beginning"

}
}

filter {
grok { match => { "message" => "%{DAY:day}\s%{MONTH:month}\s%{MONTHDAY:monthday}\s%{YEAR:year}\s%{TIME:time}\sGMT(?[+-]\d\d\d\d)\s([^)]+)\s%{NUMBER:temp}\s%{NUMBER:light}\s%{GREEDYDATA:room}"} }

}

output {

elasticsearch {
index => "logstash-templightlogs"

}

stdout{}

}

2

Is that all the error was?

3

Yes, that was all.

4

I tried the same config on Logstash 2.2.3 as opposed to Logstash 2.3.0 and here's the error I'm seeing on the console.

Screenshot.png1364×768 115 KB

5

Update : I hosted Elasticsearch publicly on '172.25.3.31' and port '9200'. So I updated my Logstash config to reflect this. Now I see Logstash Startup Completed with no errors but I don't see any of the logs in the console, which means they aren't going through to the ES server. Here's the config file -

input {
file {
path => "/home/bits/Server_Code/TempLight.log"
type => "sample"
start_position => "beginning"

}
}

filter {
grok { match => { "message" => "%{DAY:day}\s%{MONTH:month}\s%{MONTHDAY:monthday}\s%{YEAR:year}\s%{TIME:time}\sGMT(?[+-]\d\d\d\d)\s([^)]+)\s%{NUMBER:temp}\s%{NUMBER:light}\s%{GREEDYDATA:room}"} }

}

output {

elasticsearch {
index => "logstash-templightlogs"
hosts => "172.25.3.31:9200"

}

stdout{}

}

6

I think this might be a known issue we are working on fixing. Try downgrading to 2.2.X and see if you get it.

7

I tried this on Logstash 2.2.3 as well. Same issue. It says "Logstash startup completed" but nothing after that. Anything I can change in my config?

8

Yes, I have tried this on Logstash 2.2.3 as well. Same error. Anything I
can change in my config?

9

Ah see I didn't open the image because they are usually too hard to read. It'd better if you post the text of console output like that, not the picture :slight_smile:

What you are seeing is normal, it means LS has started and is waiting on new data in the file you specified in the input.
This is not an error.

10

"Connection refused {:class=>"Manticore::SocketException", :level=>:error}" is what the console says for Logstash-2.3.0. The same line is repeated 8 times for Logstash-2.2.3.

My config file is as follows -

input {
file {
path => "/home/bits/Server_Code/TempLight.log"
type => "sample"
start_position => "beginning"
sincedb_path => "/home/bits/logstash-2.3.0/file"

}
}

filter {
grok { match => { "message" => "%{DAY:day}\s%{MONTH:month}\s%{MONTHDAY:monthday}\s%{YEAR:year}\s%{TIME:time}\sGMT(?[+-]\d\d\d\d)\s([^)]+)\s%{NUMBER:temp}\s%{NUMBER:light}\s%{GREEDYDATA:room}"} }

}

output {

elasticsearch {
index => "logstash-templightlogs"
hosts => "172.25.3.31:9200"

}

stdout{}

}

11

Hosts is an array, so I'd fix that.
Make sure you can connect to ES from the LS host too.

12

Right, so I've made hosts an array like you said. I must say here that I did not need to mention the hosts parameter when I was using Logstash-2.2.0 until two days ago. I'm surprised it is needed here.

I see the following on my console -

Settings: Default pipeline workers: 8
Pipeline main started

However, none of the logs are actually getting stashed after this as is supposed to happen. Pasting my config file again here below -

input {
file {
path => "/home/bits/Server_Code/TempLight.log"
type => "sample"
start_position => "beginning"
sincedb_path => "/home/bits/logstash-2.3.0/files"

}
}

filter {
grok { match => { "message" => "%{DAY:day}\s%{MONTH:month}\s%{MONTHDAY:monthday}\s%{YEAR:year}\s%{TIME:time}\sGMT(?[+-]\d\d\d\d)\s([^)]+)\s%{NUMBER:temp}\s%{NUMBER:light}\s%{GREEDYDATA:room}"} }

}

output {

elasticsearch {
index => "logstash-templightlogs"
hosts => ['172.25.3.31:9200']

}

stdout{}

}

13

Then it's probably a sincedb issue, have you delete the old ones?

It's in the docs, has been for a while!

14

Got it running with Logstash-2.2.2. No sweat. Even deletion of the old ones did not help with the newer versions of Logstash.