# Logstash only ingesting data when shutdown/closing files **URL:** \ **Category:** Logstash\ **Created:** [September 18, 2019, 6:15pm UTC](https://discuss.elastic.co/t/logstash-only-ingesting-data-when-shutdown-closing-files/200067 "2019-09-18T18:15:43Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![edster](https://avatars.discourse-cdn.com/v4/letter/e/da6949/32.png) [@edster](https://discuss.elastic.co/u/edster)\ **Post date:** [September 18, 2019, 6:15pm UTC](https://discuss.elastic.co/t/logstash-only-ingesting-data-when-shutdown-closing-files/200067/1 "2019-09-18T18:15:43Z")
My logstash config file seems to running successfully, however, it doesn't pull in any data. It isn't until I manually shutdown the process using 'Ctrl+C', that it pulls in data. This doesn't seem to normal behavior. Is this so? And if not how can I correct this? P.S. I get no errors. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 18, 2019, 7:07pm UTC](https://discuss.elastic.co/t/logstash-only-ingesting-data-when-shutdown-closing-files/200067/2 "2019-09-18T19:07:56Z")
That would happen if the input expects a separator between events (e.g. a tcp input looking for newline) and it never sees the separator. Or a multiline codec that never sees a line that causes it to flush an event. What is your input configuration? ---
**Author:** ![edster](https://avatars.discourse-cdn.com/v4/letter/e/da6949/32.png) [@edster](https://discuss.elastic.co/u/edster)\ **Post date:** [September 18, 2019, 7:17pm UTC](https://discuss.elastic.co/t/logstash-only-ingesting-data-when-shutdown-closing-files/200067/3 "2019-09-18T19:17:24Z")
The input configuration is > input { > file { > path =\> "/data/xmlFiles/\*.xml" > sincedb\_path =\> "/dev/null" > start\_position =\> "beginning" > type =\> "xml" > codec =\> multiline { > pattern =\> "^()" > negate =\> true > what =\> previous > max\_lines =\> 105000 > } > } > } I assume I'm missing the line "auto\_flush\_interval =\> 1" in the codec from what it sounds like. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 18, 2019, 8:26pm UTC](https://discuss.elastic.co/t/logstash-only-ingesting-data-when-shutdown-closing-files/200067/4 "2019-09-18T20:26:16Z")
> [@edster](#): > > I assume I'm missing the line "auto\_flush\_interval =\> 1" in the codec Yes, if you are trying to consume an entire file as a single event you need that. ---
**Author:** ![edster](https://avatars.discourse-cdn.com/v4/letter/e/da6949/32.png) [@edster](https://discuss.elastic.co/u/edster)\ **Post date:** [September 19, 2019, 1:13pm UTC](https://discuss.elastic.co/t/logstash-only-ingesting-data-when-shutdown-closing-files/200067/5 "2019-09-19T13:13:04Z")
It's more like multiple files in a single event. The Auto\_flush didn't pull them in would it have to be a different value? ---
**Author:** ![edster](https://avatars.discourse-cdn.com/v4/letter/e/da6949/32.png) [@edster](https://discuss.elastic.co/u/edster)\ **Post date:** [September 19, 2019, 2:20pm UTC](https://discuss.elastic.co/t/logstash-only-ingesting-data-when-shutdown-closing-files/200067/6 "2019-09-19T14:20:02Z")
Sorry scratch that last comment. It did work there was a change that needed to be done elsewhere. A gsub to separate where the pattern was located, on the last line. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 17, 2019, 2:20pm UTC](https://discuss.elastic.co/t/logstash-only-ingesting-data-when-shutdown-closing-files/200067/7 "2019-10-17T14:20:06Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.