# Is it expected that filestream input picks up all documents again when file is updated again after close.on\_state\_change.inactive is exceeded? **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [March 15, 2024, 5:28pm UTC](https://discuss.elastic.co/t/is-it-expected-that-filestream-input-picks-up-all-documents-again-when-file-is-updated-again-after-close-on-state-change-inactive-is-exceeded/355514 "2024-03-15T17:28:25Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![jori-be](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jori-be/32/96028_2.png) [@jori-be](https://discuss.elastic.co/u/jori-be)\ **Post date:** [March 15, 2024, 5:28pm UTC](https://discuss.elastic.co/t/is-it-expected-that-filestream-input-picks-up-all-documents-again-when-file-is-updated-again-after-close-on-state-change-inactive-is-exceeded/355514/1 "2024-03-15T17:28:25Z")
Hey all, I'm using Filebeat 8.11.3 and I noticed that it sometimes resubmits documents that it already handled before. So I'm wondering what the expected behaviour is of Filebeat. There is a setting [filestream input | Filebeat Reference [8.11] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/8.11/filebeat-input-filestream.html#filebeat-input-filestream-close-inactive). How I think it works in this version is that the filestream input starts reading a file. In case it doesn't receive any new documents anymore in that file and the `close.on_state_change.inactive` time is exceeded, filebeat closes the handles for that file. That makes sense to me. But when there are new documents again after this time is exceeded, filebeat reprocesses all documents again. Is this expected behavior? If not, how is it supposed to work? Just for reference, this is input config I'm using: ```auto filebeat: inputs: - close.on_state_change.inactive: 10m enabled: true id: some_id_log paths: - /data/log/app/appparsed/*/app-*.log* prospector.scanner.exclude_files: - \.gz$ type: filestream ``` 10 minutes before the first log below, the logs were stopped and the handles was closed as expected. Filebeat logs: ```auto {"log.level":"info","@timestamp":"2024-03-14T08:52:55.387+0100","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":336},"message":"Reader was closed. Closing. Path='/data/log/app/appparsed/TEST-8-2-B-FGPVRPP-P-NI/app-TEST-8-2-B-FGPVRPP-P-NI.log'","service.name":"filebeat","id":"some_id_log","source_file":"filestream::some_id_log::native::536879026-64770","path":"/data/log/app/appparsed/TEST-8-2-B-FGPVRPP-P-NI/app-TEST-8-2-B-FGPVRPP-P-NI.log","state-id":"native::536879026-64770","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2024-03-14T08:53:20.378+0100","log.logger":"input.filestream","log.origin":{"file.name":"filestream/input.go","file.line":274},"message":"File was truncated. Reading file from offset 0. Path=/data/log/app/appparsed/TEST-8-2-B-FGPVRPP-P-NI/app-TEST-8-2-B-FGPVRPP-P-NI.log","service.name":"filebeat","id":"some_id_log","source_file":"filestream::some_id_log::native::536879026-64770","path":"/data/log/app/appparsed/TEST-8-2-B-FGPVRPP-P-NI/app-TEST-8-2-B-FGPVRPP-P-NI.log","state-id":"native::536879026-64770","ecs.version":"1.6.0"} ``` Thanks in advance! ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 15, 2024, 6:05pm UTC](https://discuss.elastic.co/t/is-it-expected-that-filestream-input-picks-up-all-documents-again-when-file-is-updated-again-after-close-on-state-change-inactive-is-exceeded/355514/2 "2024-03-15T18:05:53Z")
> [@jori-be](#): > > `/data/log/app/appparsed/` Is this a network share or a path in the disk? ---
**Author:** ![jori-be](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jori-be/32/96028_2.png) [@jori-be](https://discuss.elastic.co/u/jori-be)\ **Post date:** [March 19, 2024, 12:49pm UTC](https://discuss.elastic.co/t/is-it-expected-that-filestream-input-picks-up-all-documents-again-when-file-is-updated-again-after-close-on-state-change-inactive-is-exceeded/355514/4 "2024-03-19T12:49:03Z")
It is a local path, XFS filesystem ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 16, 2024, 2:49pm UTC](https://discuss.elastic.co/t/is-it-expected-that-filestream-input-picks-up-all-documents-again-when-file-is-updated-again-after-close-on-state-change-inactive-is-exceeded/355514/5 "2024-04-16T14:49:59Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.