# Logstash is not reading all the logs

**URL:** https://discuss.elastic.co/t/logstash-is-not-reading-all-the-logs/61302
**Category:** Logstash
**Created:** [September 22, 2016, 6:36pm UTC](https://discuss.elastic.co/t/logstash-is-not-reading-all-the-logs/61302 "2016-09-22T18:36:23Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![vk126y](https://avatars.discourse-cdn.com/v4/letter/v/aeb1de/32.png) [@vk126y](https://discuss.elastic.co/u/vk126y)
#### Post date: [September 22, 2016, 6:36pm UTC](https://discuss.elastic.co/t/logstash-is-not-reading-all-the-logs/61302/1 "2016-09-22T18:36:23Z")

</div>

I want my logstash to read the following pattern of files from the logs directory

Current Date File -\> server.log  
Historical File Name -\> server.log.yyyy-mm-dd

In my system I have 22 files in my logs folder

server.log  
server.log.2016-09-22  
server.log.2016-09-21  
server.log.2016-09-20  
server.log.2016-09-19  
server.log.2016-09-18  
server.log.2016-09-17

and so on in this way everyday files are renamed with date appended to name and current date file is named as server.log

my conf file is as follows

> input {
> 
> file {  
> type =\> "serverlog"  
> path =\> ["E:/Code\_Athon/test/test1/_server.log_"]  
> start\_position =\> "beginning"
> 
> }
> 
> }

`I found the above configuration does not work for me and is reading only server.log file. No past files` ☹

I also tried the below configurations

> input {
> 
> file {  
> type =\> "serverlog"  
> path =\> ["E:/test/test1/server.log","E:/test/test1/server.log\*"]  
> start\_position =\> "beginning"
> 
> }
> 
> }

In the above configuration i observed that logstash picked up server.log and server.log.2016-09-21 and it does not go to other  
files in past , may be because of performance issues or worker thread , but I did not get any error on console. ☹

> input {
> 
> file {  
> type =\> "serverlog"  
> path =\> ["E:/test/test1/server.log","E:/test/test1/server.log\*"]  
> path =\> ["E:/test/test1/server.log._-_-\*"]  
> start\_position =\> "beginning"
> 
> }
> 
> }

`Even this configuration does not solve my issue.`

> Can anyone suggest what should be my configuration so that my lohgstash can read the current file , past file and daily rollover file  
> named as server.log

**Am i missing any configurations here ?**

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [September 22, 2016, 6:43pm UTC](https://discuss.elastic.co/t/logstash-is-not-reading-all-the-logs/61302/2 "2016-09-22T18:43:56Z")

</div>

You may need to change the [ignore\_older](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-ignore_older) setting for the file input plugin in order to make Logstash process older files.

---

<div class="post-metadata">

### Author: ![vk126y](https://avatars.discourse-cdn.com/v4/letter/v/aeb1de/32.png) [@vk126y](https://discuss.elastic.co/u/vk126y)
#### Post date: [September 23, 2016, 6:36am UTC](https://discuss.elastic.co/t/logstash-is-not-reading-all-the-logs/61302/3 "2016-09-23T06:36:13Z")

</div>

Thanks Chris.

It worked out after applying the suggested settings.

Shall I now remove the path filter to skip reading the old files as I have no backlog of logs ?

Regards  
Vinay Kumar

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:37am UTC](https://discuss.elastic.co/t/logstash-is-not-reading-all-the-logs/61302/4 "2017-07-06T04:37:14Z")

</div>


