So I fixed the error message regarding the match part. Now I keep getting a new error I don't' really understand. The error is as follows:
es_1 | log4j:WARN No appenders could be found for logger (bootstrap).
es_1 | log4j:WARN Please initialize the log4j system properly.
es_1 | log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
kibana_1 | {"@timestamp":"2015-07-07T07:54:08.962Z","level":"error","node_env":" production","error":"Request error, retrying -- connect ECONNREFUSED"}
kibana_1 | {"@timestamp":"2015-07-07T07:54:08.965Z","level":"warn","message":"Un able to revive connection: http://172.17.6.166:9200/","node_env":"production"}
kibana_1 | {"@timestamp":"2015-07-07T07:54:08.965Z","level":"warn","message":"No living connections","node_env":"production"}
kibana_1 | {"@timestamp":"2015-07-07T07:54:08.966Z","level":"info","message":"Un able to connect to elasticsearch at http://172.17.6.166:9200. Retrying in 2.5 se conds.","node_env":"production"}
kibana_1 | {"@timestamp":"2015-07-07T07:54:11.532Z","level":"info","message":"Fo und kibana index","node_env":"production"}
kibana_1 | {"@timestamp":"2015-07-07T07:54:11.696Z","level":"info","message":"Li stening on 0.0.0.0:5601","node_env":"production"}
stash_1 | {:timestamp=>"2015-07-07T07:54:12.689000+0000", :message=>"Using vers ion 0.1.x input plugin 'syslog'. This plugin isn't well supported by the communi ty and likely has no maintainer.", :level=>:info}
stash_1 | {:timestamp=>"2015-07-07T07:54:12.703000+0000", :message=>"Using vers ion 0.1.x codec plugin 'json'. This plugin isn't well supported by the community and likely has no maintainer.", :level=>:info}
stash_1 | {:timestamp=>"2015-07-07T07:54:12.713000+0000", :message=>"Using vers ion 0.1.x input plugin 'file'. This plugin isn't well supported by the community and likely has no maintainer.", :level=>:info}
stash_1 | {:timestamp=>"2015-07-07T07:54:12.721000+0000", :message=>"Using vers ion 0.1.x codec plugin 'plain'. This plugin isn't well supported by the communit y and likely has no maintainer.", :level=>:info}
stash_1 | {:timestamp=>"2015-07-07T07:54:12.724000+0000", :message=>"You are us ing a deprecated config setting "type" set in grok. Deprecated settings will c ontinue to work, but are scheduled for removal from logstash in the future. You can achieve this same behavior with the new conditionals, like: if [type] == \" sometype\" { grok { ... } }. If you have any questions about this, please visit the #logstash channel on freenode irc.", :name=>"type", :plugin=><LogStash::Fil ters::Grok --->, :level=>:warn}
stash_1 | {:timestamp=>"2015-07-07T07:54:12.815000+0000", :message=>"Using vers ion 0.1.x output plugin 'elasticsearch'. This plugin isn't well supported by the community and likely has no maintainer.", :level=>:info}
stash_1 | Configuration OK
I really don't know if I need the locale => "sv" command either. The reason I put it there is because I want it to take the appropriate time from the event in the old log itself and not the time I index the log which of course would be the current time.
I've been searching in the logstash config but I can't find any syntax errors. It seems to have something to do with my current output elasticsearch version. This is what my whole logstash config looks like right now.
input {
syslog {
port => 5514
codec => "json"
}
file {
path => "/var/externallogs_maven/oldlogs/request.log.2015-06-22"
type => "nexus-log"
start_position => "beginning"
}
}
filter {
grok {
type => "nexus-log"
patterns_dir => "./config-dir/patterns"
match => [
"message", "\b\w+\b\s/nexus/content/repositories/(?<repositories>[^/]+)",
"message", "(?<mytimestamp>%{MONTHDAY}/%{MONTH}/%{YEAR}:%{HOUR}:%{MINUTE}:%{SECOND})"
]
}
date {
match => ["mytimestamp", "dd/MMM/YYYY:HH:mm:ss +SSSS"]
locale => "sv"
}
}
output {
elasticsearch {
host => es
port => 9300
cluster => "elks"
protocol => "transport"
}
}