# Reading file from beginning using file input

**URL:** <https://discuss.elastic.co/t/reading-file-from-beginning-using-file-input/168253>\
**Category:** Logstash\
**Created:** [February 13, 2019, 3:51pm UTC](https://discuss.elastic.co/t/reading-file-from-beginning-using-file-input/168253 "2019-02-13T15:51:58Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [February 13, 2019, 3:51pm UTC](https://discuss.elastic.co/t/reading-file-from-beginning-using-file-input/168253/1 "2019-02-13T15:51:58Z")

</div>

I have this setting, but it is not reading file again and again.  
I want to read this file from beginning when it is changed.  
I am getting this file copy from another system every one hour.

It reads once when I starts logstash from command line. Then as test when I transfer file or update that file it is not re-reading at all.

input {  
file {  
path =\> "/elkdata01/a\_size\_final\_log"  
sincedb\_path =\> "/dev/null"  
start\_position =\> "beginning"  
}  
}

If I touch/edit this log file manually on /elkdata01/ then it reads it. but if I scp that file from different system it does not. Why? what do I have to do to make it work

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 13, 2019, 4:03pm UTC](https://discuss.elastic.co/t/reading-file-from-beginning-using-file-input/168253/2 "2019-02-13T16:03:25Z")

</div>

Even if you set sincedb\_path =\> "/dev/null" the file input still uses an in-memory sincedb. It just does not persist it to disk. It will only re-read a file with that name from the beginning if the inode number changes. So instead of overwriting it, you would need to move it aside and write a new file.

---

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [February 13, 2019, 4:25pm UTC](https://discuss.elastic.co/t/reading-file-from-beginning-using-file-input/168253/3 "2019-02-13T16:25:47Z")

</div>

ok. I deleted file. and then scp new file (same\_name) but it didn't read it.

Then I did vi of that file on logstash server and it read it right away.

---

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [February 13, 2019, 4:27pm UTC](https://discuss.elastic.co/t/reading-file-from-beginning-using-file-input/168253/4 "2019-02-13T16:27:52Z")

</div>

even if touch the file on logstash server it does not read it.  
only way right now it re-reads file is if I do vi, change value in it and save it.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 13, 2019, 4:45pm UTC](https://discuss.elastic.co/t/reading-file-from-beginning-using-file-input/168253/5 "2019-02-13T16:45:07Z")

</div>

If you delete a file and create a new one, then on some filesystems the inode number is re-used. That will prevent the file input from seeing it as a new file. For example

```
echo foo > foo ; ls -li foo ; rm foo 
echo foo > foo ; ls -li foo ; rm foo 

```

gets me

```
4272123 -rw-r--r--. 1 user user 4 Feb 13 11:41 foo
4272123 -rw-r--r--. 1 user user 4 Feb 13 11:41 foo

```

It re-uses inode 4272123. Ugh!

---

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [February 13, 2019, 5:17pm UTC](https://discuss.elastic.co/t/reading-file-from-beginning-using-file-input/168253/6 "2019-02-13T17:17:47Z")

</div>

Alright make sense now. this is what I am gone a do.  
create a file with day of week somewhere.

delete all log file once a week. and rotate number

foo.1 foo.2 foo3 ...foo.7  
and rotate.

I tested this method with some dummy file name and it worked.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 13, 2019, 5:17pm UTC](https://discuss.elastic.co/t/reading-file-from-beginning-using-file-input/168253/7 "2019-03-13T17:17:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
