# How Logstash reads the same file name but with different content(keeps updating)? **URL:** \ **Category:** Logstash\ **Created:** [June 19, 2017, 6:42pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984 "2017-06-19T18:42:53Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)\ **Post date:** [June 19, 2017, 6:42pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/1 "2017-06-19T18:42:53Z")
Hi, I am using logstash to read a log file. The file name reamins same but the content keeps changing for 1 hour(wipes off the previous logs and inserts new logs). But logstash is just reading last line of the file. Here is my config file, ``` input{ file { path=>"C:logs/report.log" start_position => "beginning" type=>"ex" } } output{ if [type] == "ex"{ stdout { codec => rubydebug } } ``` Please help me on this issue. Thanks in advance. ---
**Author:** ![infamous](https://avatars.discourse-cdn.com/v4/letter/i/df705f/32.png) [@infamous](https://discuss.elastic.co/u/infamous)\ **Post date:** [June 19, 2017, 8:28pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/2 "2017-06-19T20:28:14Z")
It might be a sincedb issue? Logstash has logs to remember where it left off when parsing a certain file. Try going to your Logstash folder and clearing the sincedb folder (should be in "data"). Then do: `file { path=>"C:logs/report.log" start_position => "beginning" type=>"ex" sincedb_path => "NUL" }` Then try again. Hopefully that helps! ---
**Author:** ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)\ **Post date:** [June 20, 2017, 3:25pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/3 "2017-06-20T15:25:32Z")
Hi, thanks for your reply. Will try that. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 21, 2017, 7:17am UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/4 "2017-06-21T07:17:43Z")
> wipes off the previous logs and inserts new logs _Exactly_ how is this done? Is the original file overwritten or is the new data written to a new file that's renamed to replace the original file? ---
**Author:** ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)\ **Post date:** [June 23, 2017, 1:31am UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/5 "2017-06-23T01:31:01Z")
Hi Magnus, there is a cron job which replaces the content in the file every 1 hour, file name remains same. I've deleted sincedb folder and changed the config file but it still reads just the last line. Here is my config, input{ file { path=\>"C:logs/report.log" start\_position =\> "beginning" type=\>"ex" sincedb\_path =\> "NUL" } } output{ if [type] == "ex"{ stdout { codec =\> rubydebug } } ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 26, 2017, 9:13pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/6 "2017-06-26T21:13:19Z")
Really, it's reading only the last line of the file even with `sincedb_path => "NUL"`? ---
**Author:** ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)\ **Post date:** [June 27, 2017, 5:10pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/7 "2017-06-27T17:10:31Z")
I've seen some instances where it only reads the last event. But now it's not reading at all. When I manually delete the content in file and add some content, logstash is reading the content but it's not doing automatically. Also I am running logstash.conf from a different path(not from logstash bin), will this be causing any issue? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 27, 2017, 5:30pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/8 "2017-06-27T17:30:04Z")
> I've seen some instances where it only reads the last event. But now it's not reading at all. When I manually delete the content in file and add some content, logstash is reading the content but it's not doing automatically. Bumping up the log level to debug and looking for everything having to do with the reading of the file (including sincedb) might give useful clues. > Also I am running logstash.conf from a different path(not from logstash bin), will this be causing any issue? No. ---
**Author:** ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)\ **Post date:** [June 27, 2017, 5:48pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/9 "2017-06-27T17:48:00Z")
Magnus, Can you please be more specific? i don't understand "Bumping up the log level to debug". Thank you. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 27, 2017, 5:50pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/10 "2017-06-27T17:50:25Z")
Increase Logstash's log level, e.g. by starting it with `--log.level debug`. ---
**Author:** ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)\ **Post date:** [June 27, 2017, 6:03pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/11 "2017-06-27T18:03:14Z")
I've started it with log level, but can you please say what extra information I can get from that? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 27, 2017, 6:05pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/12 "2017-06-27T18:05:14Z")
Clues about what's going in inside Logstash, like what sincedb file (if any) it's reading, the current read position in a file, and so on.. I don't know what's wrong with your setup so I can't tell you exactly what to look for. ---
**Author:** ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)\ **Post date:** [June 27, 2017, 6:33pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/13 "2017-06-27T18:33:17Z")
But I don't see any extra information like current read position, sincedb file. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 27, 2017, 6:35pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/14 "2017-06-27T18:35:44Z")
Yes, but maybe someone else can spot the clues if you post the log to e.g. pastebin. ---
**Author:** ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)\ **Post date:** [June 27, 2017, 6:52pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/15 "2017-06-27T18:52:24Z")
Magnus, I appreciate your patience. Please look at the logs below and let me know if those make sense, [2017-06-27T14:46:22,523][DEBUG][logstash.inputs.file] each: file grew: C:logs/another\_report.log: old size 5924, new size 5931 [2017-06-27T14:46:23,517][DEBUG][logstash.pipeline] Pushing flush onto p ipeline [2017-06-27T14:46:23,525][DEBUG][logstash.inputs.file] each: file grew: C:logs/another\_report.log: old size 5924, new size 5931 [2017-06-27T14:46:24,526][DEBUG][logstash.inputs.file] each: file grew: C:logs/another\_report.log: old size 5924, new size 5931 [2017-06-27T14:46:24,527][DEBUG][logstash.agent] Reading config file {:config\_file=\>"C:logs/log.conf"} [2017-06-27T14:46:24,528][DEBUG][logstash.agent] no configuration cha nge for pipeline {:pipeline=\>"main", :config=\>"input{\n\tfile {\n\t\tpath=\>"C:/ ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 27, 2017, 6:54pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/16 "2017-06-27T18:54:34Z")
Let's see all the logs. ---
**Author:** ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)\ **Post date:** [June 27, 2017, 7:43pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/17 "2017-06-27T19:43:19Z")
It's running continuously, they are pretty much the same logs that are being repeated. When you see the above logs, each:file grew...is another file in config, not the actual file I am looking for. (In the config I have multiple files) ---
**Author:** ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)\ **Post date:** [July 10, 2017, 3:26pm UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/18 "2017-07-10T15:26:18Z")
Hi Magnus, Can you please help on this? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 12, 2017, 11:54am UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/19 "2017-07-12T11:54:40Z")
I asked for complete logs and you haven't provided them. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 9, 2017, 11:54am UTC](https://discuss.elastic.co/t/how-logstash-reads-the-same-file-name-but-with-different-content-keeps-updating/89984/20 "2017-08-09T11:54:41Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.