Hi,
I have two different weblogic log files, my conf file looks like below but it doesn't work:
input {
file {
path => "D:/<dir>/logs/ecomm_vws02_app"
start_position => beginning
ignore_older => 0
sincedb_path => "/dev/null"
type => "application"
}
file {
path => "D:/<dir>/logs/ecomm_vws02_access.201609121200"
start_position => beginning
ignore_older => 0
sincedb_path => "/dev/null"
type => "access"
}
}
filter {
if [type] == "application" {
grok {
patterns_dir => "D:/<dir>/logstash-all-plugins-2.4.0/logstash-2.4.0/patterns"
match => [ "message","%{WEBLOGIC_POC}|%{WEBLOGIC_POC1}"]
}
date {
match => ["timestamp","EEE MMM dd HH:mm:ss z yyyy"]
}
}
if [type] == "access" {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}"}
}
date {
match => ["timestamp","EEE MMM dd HH:mm:ss z yyyy"]
}
}
}
output {
if [type] == "application" {
elasticsearch {
hosts => [ "localhost:9200" ]
index => "prod_logs_data-%{+YYYY.MM.dd}"
}
}
if [type] == "access"{
elasticsearch {
hosts => [ "localhost:9200" ]
index => "prod_accesslogs_data-%{+YYYY.MM.dd}"
}
}
}`Preformatted text`Please format your code with the </> button, it’s very hard to read as it is.
Does it look better?
What isn't working?
Hi, Looks like its not able to understand the two file inputs, its reading the second file - ecomm_vws02_access.201609121200 alone and creating only-prod_logs_data-%{+YYYY.MM.dd} index, with gorkparse failure and not creating the second index -prod_accesslogs_data-*
Please advice, thanks
Is it reading the other file at all? I suggest you disable the (somewhat) working file input and focus debugging on what doesn't work at all and then fix one thing at a time. This will be easier to debug if you replace the elasticsearch outputs with a single stdout { codec => rubydebug } output.
Thanks Magnus - But the issue is, if I specify a single input location, it works perfectly well. But If i place both types of logs in the input directory, it reads the FIRST one ONLY. Second file isnt getting read at all.Anything wrong in my syntax? Please suggest, its little critical. Many Thanks.Preformatted text
I haven't heard of such behavior before. I'd kick the log level to max with --debug and use other debugging tools (lsof, strace, ...) if needed. Is it opening both files? Is it reading from both files? Et cetera.