Logstash not picking older logs

Elastic Stack Logstash
1

I have a scenario where I need to pick 4 to 5 years older data using Logstash. Below is the config file

input {
file {
path => [ "cricsheet-xml-master\0.7\5*.xml" ]
type => "CricketData"
start_position => "beginning"
}
}
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}"}
}
}
output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}

But when I run the logstash, it is just starting but no data is stashed in elasticsearch.

2

That path doesn't look valid?

Anyway, take a look at https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-ignore_older

3

Apart from ignore_older, I'm pretty sure the file input's path patterns need to be absolute paths.

4

The path is valid... I only pasted a part of it because it includes domain of organization... You can assume that as "C:/Users/elk/...../cricsheet-xml-master\0.7\5*.xml"

5

Path of the input file is abslolute only... as per the protocol of the organization am not allowed to give the full path. So i did write only a part of it

6

If you're going to obfuscate your configuration or logs, don't change the structure of the original string. Just replace the substrings that are confidentials.

Read about sincedb in the file input documentation. Logstash is probably tailing the input file.

7

Sure Thanks...

About the "not picking up older logs", sometimes even the newer logs are not picking even.. I don't know whether logstash expects something which am not giving it to it.

Anyway, thank you for the replies.. I will update if i get to solve the problem. Or comeback if i don't..