Logstash will read 3days old file


(P Kumar) #1

Is it possible to logstash5 will read 3 days old SystemOut.log file ?


(Magnus Bäck) #2

If configured to do so Logstash will read files with any name and any age.


(P Kumar) #3

I configure 5 files in one logstash configration but it is sending only 3 files to elastic serch and rest are not sending but it is showing in stdout .so i thought it may be file age.how we will debug this issue and who is blocking rest 2 files to send elstic search


(Magnus Bäck) #4

Show us your configuration.


(P Kumar) #5

input {
file {
type => "nodeagent"
path => [ "/opt/WebSphere/AppServer/profiles/AppSrv01/logs/nodeagent/SystemOut.log" ]
ignore_older => "400000000"
start_position => "beginning"
sincedb_path => "/dev/null"

    }
    file {
            type => "websphere1"
            path => [ "/opt/WebSphere/AppServer/profiles/AppSrv01/logs/App-v01/SystemOut.log" ]
            ignore_older => 4
            start_position => "beginning"
            sincedb_path => "/dev/null"
    }
    file {
            type => "app1"
            path => [ "/opt/WebSphere/AppServer/profiles/AppSrv01/logs/App1-v01/SystemOut.log" ]
            ignore_older => 4
            start_position => "beginning"
            sincedb_path => "/dev/null"
    }
    file {
            type => "app"
            path => [ "/opt/WebSphere/AppServer/profiles/AppSrv01/logs/App12Server1/SystemOut.log" ]
            ignore_older => 4
            start_position => "beginning"
            sincedb_path => "/dev/null"
    }
    file{
            type => "dmgr"
            path => [ "/opt/WebSphere/AppServer/profiles/Dmgr01/logs/dmgr/SystemOut.log" ]
            start_position => "beginning"
            ignore_older => 4
            sincedb_path => "/dev/null"

    }

}

#filter {

grok {

match => [ "message", "%{COMBINEDAPACHELOG}" ]

}

#}

output {
if [type] == "nodeagent" {
elasticsearch{
hosts => ["x.x.x.x:9200"]
index => "x.x.x.x_nodeAgent_%{+YYYY.MM.dd}"
}
}
if [type] == "websphere1" {
elasticsearch{
hosts => ["x.x.x.x:9200"]
index => "x.x.x.x_App-v01_%{+YYYY.MM.dd}"
}
}
if [type] == "app1" {
elasticsearch{
hosts => ["x.x.x.x:9200"]
index => "x.x.x.x_App1-v01_%{+YYYY.MM.dd}"
}
}
if [type] == "app" {
elasticsearch{
hosts => ["x.x.x.x:9200"]
index => "x.x.x.x_App12Server1-v01_%{+YYYY.MM.dd}"
}
}
if [type] == "dmgr" {
elasticsearch{
hosts => ["x.x.x.x:9200"]
index => "x.x.x.x_dmgr_%{+YYYY.MM.dd}"
}
}
else {
stdout{
codec => rubydebug
}
}

}


(Magnus Bäck) #6

If you look in your Logstash logs I'm pretty sure you'll find lots of log entries telling you that it can't create the indexes because index names must be all lowercase.


(P Kumar) #7

Thanks it is working now.


(system) #8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.