I am tring to upgrade from Logstash 1.4.4 to Logstash 2.4.0
I have reused the same configuration file that we used before for parsing our application logs
I am running Logstash 2.4.0 in debug mode and getting that block that does not indicate that much
{:timestamp=>"2016-11-10T15:49:43.008000-0500", :message=>"Pipeline aborted due to error", :exception=>"LogStash::ConfigurationError", :backtrace=>["/cceapps/logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/config/mixin.rb:88:in `config_init'", "org/jruby/RubyHash.java:1342:in `each'", "/cceapps/logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/config/mixin.rb:72:in `config_init'", "/cceapps/logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/outputs/base.rb:79:in `initialize'", "/cceapps/logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/output_delegator.rb:74:in `register'", "/cceapps/logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:181:in `start_workers'", "org/jruby/RubyArray.java:1613:in `each'", "/cceapps/logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:181:in `start_workers'", "/cceapps/logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:136:in `run'", "/cceapps/logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/agent.rb:491:in `start_pipeline'"], :level=>:error, :file=>"logstash/agent.rb", :line=>"493", :method=>"start_pipeline"}
{:timestamp=>"2016-11-10T15:49:46.016000-0500", :message=>"stopping pipeline", :id=>"main", :file=>"logstash/agent.rb", :line=>"406", :method=>"shutdown_pipelines"}
{:timestamp=>"2016-11-10T15:49:46.019000-0500", :message=>"Closing inputs", :level=>:info, :file=>"logstash/pipeline.rb", :line=>"384", :method=>"shutdown"}
{:timestamp=>"2016-11-10T15:49:46.021000-0500", :message=>"stopping", :plugin=>"LogStash::Inputs::File", :level=>:debug, :file=>"logstash/inputs/base.rb", :line=>"81", :method=>"do_stop"}
{:timestamp=>"2016-11-10T15:49:46.023000-0500", :message=>"Closed inputs", :level=>:info, :file=>"logstash/pipeline.rb", :line=>"386", :method=>"shutdown"}
When i run the configtest parameter i get a Coonfiguration OK message
$ ./bin/logstash --configtest --config ./conf/app.conf
Configuration OK
My configuration file uses custom patterns created in files that exist in patterns_dir
Here is my configuration file
input {
file {
path => "/apps/wlserver/logs/pce.log"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
multiline {
# Lines starting with a date are new entries (ie. not multiline).
pattern => "^%{MYTIMESTAMP} "
negate => true
what => "previous"
}
if "multiline" in [tags] {
# Multiline messages contain an fullqualifiedname and stacktrace.
grok {
patterns_dir => "./patterns"
match => [ "message", "^%{MYTIMESTAMP:datetime}\s+\[%{GREEDYDATA:brackettext}\]\s+\{%{LOGLEVEL:severity}\}\s+%{GREEDYDATA:jobname}\s+\(%{JAVACLASS:javaclass}(:(?<linenumber>\d+))?\)\s+-\s+%{GREEDYDATA:messagetext}\n(?m)(?<stacktrace>(?<fullqualifiedname>([a-zA-Z0-9-]+\.)*(?<classname>[A-Za-z0-9$_]+))?%{GREEDYDATA})" ]
}
} else {
# Single line messages don't contain an classname or stacktrace. Add empty fields.
grok {
patterns_dir => "./patterns"
match => [ "message", "^%{MYTIMESTAMP:datetime}\s+\[%{GREEDYDATA:brackettext}\]\s+\{%{LOGLEVEL:severity}\}\s+%{GREEDYDATA:jobname}\s+\(%{JAVACLASS:javaclass}(:(?<linenumber>\d+))?\)\s+-\s+%{GREEDYDATA:messagetext}$" ]
add_field => {
"fullqualifiedname" => ""
"classname" => ""
"stacktrace" => ""
}
}
}
# Drop entries based on severity
if [severity] != "ERROR" and [severity] != "WARN" {
drop { }
}
}
output {
stdout { codec => rubydebug }
elasticsearch {
cluster => "myculster"
flush_size => 500
host => "hostname"
idle_flush_time => 30
index => "logstash-%{+YYYY-MM-dd}"
node_name => "dev"
port => "19001"
protocol => "http"
workers => 1
}
}
sorry for the formatting i could not do better than that
Regards