# Logstash not picking older logs

**URL:** https://discuss.elastic.co/t/logstash-not-picking-older-logs/62944
**Category:** Logstash
**Created:** [October 13, 2016, 1:03pm UTC](https://discuss.elastic.co/t/logstash-not-picking-older-logs/62944 "2016-10-13T13:03:29Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![VijayKarthikeyan](https://avatars.discourse-cdn.com/v4/letter/v/8491ac/32.png) [@VijayKarthikeyan](https://discuss.elastic.co/u/VijayKarthikeyan)
#### Post date: [October 13, 2016, 1:03pm UTC](https://discuss.elastic.co/t/logstash-not-picking-older-logs/62944/1 "2016-10-13T13:03:29Z")

</div>

I have a scenario where I need to pick 4 to 5 years older data using Logstash. Below is the config file

input {  
file {  
path =\> ["cricsheet-xml-master\0.7\5\*.xml"]  
type =\> "CricketData"  
start\_position =\> "beginning"  
}  
}  
filter {  
grok {  
match =\> { "message" =\> "%{COMBINEDAPACHELOG}"}  
}  
}  
output {  
elasticsearch { hosts =\> ["localhost:9200"] }  
stdout { codec =\> rubydebug }  
}

But when I run the logstash, it is just starting but no data is stashed in elasticsearch.

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 15, 2016, 6:13am UTC](https://discuss.elastic.co/t/logstash-not-picking-older-logs/62944/2 "2016-10-15T06:13:44Z")

</div>

That path doesn't look valid?

Anyway, take a look at [https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-ignore\_older](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-ignore_older)

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [October 15, 2016, 9:04am UTC](https://discuss.elastic.co/t/logstash-not-picking-older-logs/62944/3 "2016-10-15T09:04:23Z")

</div>

Apart from `ignore_older`, I'm pretty sure the file input's path patterns need to be absolute paths.

---

<div class="post-metadata">

### Author: ![VijayKarthikeyan](https://avatars.discourse-cdn.com/v4/letter/v/8491ac/32.png) [@VijayKarthikeyan](https://discuss.elastic.co/u/VijayKarthikeyan)
#### Post date: [October 19, 2016, 7:08am UTC](https://discuss.elastic.co/t/logstash-not-picking-older-logs/62944/4 "2016-10-19T07:08:55Z")

</div>

The path is valid... I only pasted a part of it because it includes domain of organization... You can assume that as "C:/Users/elk/...../cricsheet-xml-master\0.7\5\*.xml"

---

<div class="post-metadata">

### Author: ![VijayKarthikeyan](https://avatars.discourse-cdn.com/v4/letter/v/8491ac/32.png) [@VijayKarthikeyan](https://discuss.elastic.co/u/VijayKarthikeyan)
#### Post date: [October 19, 2016, 7:10am UTC](https://discuss.elastic.co/t/logstash-not-picking-older-logs/62944/5 "2016-10-19T07:10:08Z")

</div>

Path of the input file is abslolute only... as per the protocol of the organization am not allowed to give the full path. So i did write only a part of it

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [October 19, 2016, 7:13am UTC](https://discuss.elastic.co/t/logstash-not-picking-older-logs/62944/6 "2016-10-19T07:13:36Z")

</div>

If you're going to obfuscate your configuration or logs, don't change the structure of the original string. Just replace the substrings that are confidentials.

Read about sincedb in the file input documentation. Logstash is probably tailing the input file.

---

<div class="post-metadata">

### Author: ![VijayKarthikeyan](https://avatars.discourse-cdn.com/v4/letter/v/8491ac/32.png) [@VijayKarthikeyan](https://discuss.elastic.co/u/VijayKarthikeyan)
#### Post date: [October 19, 2016, 7:21am UTC](https://discuss.elastic.co/t/logstash-not-picking-older-logs/62944/7 "2016-10-19T07:21:10Z")

</div>

Sure Thanks...

About the "not picking up older logs", sometimes even the newer logs are not picking even.. I don't know whether logstash expects something which am not giving it to it.

Anyway, thank you for the replies.. I will update if i get to solve the problem. Or comeback if i don't..

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:33am UTC](https://discuss.elastic.co/t/logstash-not-picking-older-logs/62944/8 "2017-07-06T04:33:35Z")

</div>


