# Missing EOL character - Can't read line **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [April 11, 2018, 10:25am UTC](https://discuss.elastic.co/t/missing-eol-character-cant-read-line/127609 "2018-04-11T10:25:17Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![sancroth](https://avatars.discourse-cdn.com/v4/letter/s/838e76/32.png) [@sancroth](https://discuss.elastic.co/u/sancroth)\ **Post date:** [April 11, 2018, 10:25am UTC](https://discuss.elastic.co/t/missing-eol-character-cant-read-line/127609/1 "2018-04-11T10:25:17Z")
I am trying to capture magento's Report logs. Magento report are generated into a single file every time so you can have multiple reports at multiple files. I am harvesting them by using the path: `/path/to/magento/reports/*` I have tried capturing the log with **and** without multiline without any success. The log looks like this: `{"0":"Specified invalid parent id (MageGuide_Mgbase::MageGuide)","1":"#0 \/home\/pharma\/public_html\/vendor\/magento\/framework\/Interception\/Interceptor.php(58): Magento\\Backend\\Model\\Menu\\Builder->getResult(Object(Magento\\Backend\\Model\\Menu))\n#1 \/home\/pharma\/public_html\/vendor\/magento\/framework\/Interception\/Interceptor.php(138): Magento\\Backend\\Model\\Menu\\Builder\\Interceptor->___callParent('getResult', Array)\n#2 \/home\/pharma\/public_html\/vendor\/magento\/framework\/Interception\/Interceptor.php(153): Magento\\Backend\\Model\\Menu\\Builder\\Interceptor->Magento\\Framework\\Interception\\{closure}(Object(Magento\\Backend\\Model\\Menu))\n#3 \/home\/pharma\/public_html\/generated\/code\/Magento\/Backend\/Model\/Menu\/Builder\/Interceptor.php(39): Magento\\Backend\\Model\\Menu\\Builder\\Interceptor->___callPlugins('getResult', Array, Array)\n#4 \/home\/pharma\/public_html\/vendor\/magento\/module-backend\/Model\/Menu\/Config.php(148): Magento\\Backend\\Model\\Menu\\Builder\\Interceptor->getResult(Object(Magento\\Backend\\Model\\Menu))\n#5 \/home\/pharma\/public_html\/vendor\/magento\/module-backend\/Model\/Menu\/Config.php(111): Magento\\BackendMagento\\Backend\\Controller\\Adminhtml\\Index\\Index\\Interceptor->_processUrlKeys()\n#11 \/home\/pharma\/public_html\/vendor\/magento\/framework\/Interception\/Interceptor.php(58): Magento\\Backend\\App\\AbstractAction->dispatch(Object(Magento\\Framework\\App\\Request\\Http))\n#12 \/home\/pharma\/public_html\/vendor\/magento\/framework\/Interception\/Interceptor.php(138): Magento\\Backend\\Controller\\Adminhtml\\Index\\Index\\Interceptor->___callParent('dispatch', Array)\n#14 \/home\/pharma\/public_html\/vendor\/magento\/framework\/Interception\/Interceptor.php(135): Magento\\Backend\\App\\Action\\Plugin\\Authentication->aroundDispatch(Object(Magento\\Backend\\Controller\\Adminhtml\\Index\\Index\\Interceptor), Object(Closure), Object(Magento\\Framework\\App\\Request\\Http))\n#15 \/home\/pharma\/public_html\/vendor\/magento\/framework\/Interception\/Interceptor.php(153): Magento\\Backend\\Controller\\Adminhtml\\Index\\Index\\Interceptor->Magento\\Framework\\Interception\\{closure}(Object(Magento\\Framework\\App\\Request\\Http))\n#16 \/home\/pharma\/public_html\/generated\/code\/Magento\/Backend\/Controller\/Adminhtml\/Index\/Index\/Interceptor.php(39): Magento\\Backend\\Controller\\Adminhtml\\Index\\Index\\Interceptor->___callPlugins('dispatch', Array, NULL)\n#17 \/home\/pharma\/public_html\/vendor\/magento\/framework\/App\/FrontController.php(55): Magento\\Backend\\Controller\\Adminhtml\\Index\\Index\\Interceptor->dispatch(Object(Magento\\Framework\\App\\Request\\Http))\n#18 \/home\/pharma\/public_html\/vendor\/magento\/framework\/Interception\/Interceptor.php(58): Magento\\Framework\\App\\FrontController->dispatch(Object(Magento\\Framework\\App\\Request\\Http))\n#19 \/home\/pharma\/public_html\/vendor\/magento\/framework\/Interception\/Interceptor.php(138): Magento\\Framework\\App\\FrontController\\Interceptor->___callParent('dispatch', Array)\n#20 \/home\/pharma\/public_html\/vendor\/magento\/framework\/Interception\/Interceptor.php(153): Magento\\Framework\\App\\FrontController\\Interceptor->Magento\\Framework\\Interception\\{closure}(Object(Magento\\Framework\\App\\Request\\Http))\n#21 \/home\/pharma\/public_html\/generated\/code\/Magento\/Framework\/App\/FrontController\/Interceptor.php(26): Magento\\Framework\\App\\FrontController\\Interceptor->___callPlugins('dispatch', Array, Array)\n#22 \/home\/pharma\/public_html\/vendor\/magento\/framework\/App\/Http.php(135): Magento\\Framework\\App\\FrontController\\Interceptor->dispatch(Object(Magento\\Framework\\App\\Request\\Http))\n#23 \/home\/pharma\/public_html\/vendor\/magento\/framework\/App\/Bootstrap.php(256): Magento\\Framework\\App\\Http->launch()\n#24 \/home\/pharma\/public_html\/pub\/index.php(37): Magento\\Framework\\App\\Bootstrap->run(Object(Magento\\Framework\\App\\Http))\n#25 {main}","url":"\/backpharm\/admin\/index\/index\/key\/9044c99afcf4336da610dab700bc8027979e52169bdae38fbe0d3cd0adb259c1\/","script_name":"\/index.php"}` I have tried multilines like: ``` pattern: '"\}$' negate: true match: before pattern:'^{' negate:true match:after ``` And a few more but to no fruit. Doesn't capture the message or any line at all even with no multiline context. Message i get from debug is: ``` 2018-04-11T11:41:04.836+0200 DEBUG [prospector] log/prospector.go:499 Harvester for file is still running: /home/pharma/public_html/var/report/123123123123123 2018-04-11T11:41:04.836+0200 DEBUG [prospector] log/prospector.go:361 Check file for harvesting: /home/pharma/public_html/var/report/1378880469402 2018-04-11T11:41:04.836+0200 DEBUG [prospector] log/prospector.go:447 Update existing file for harvesting: /home/pharma/public_html/var/report/1378880469402, offset: 0 2018-04-11T11:41:04.836+0200 DEBUG [prospector] log/prospector.go:499 Harvester for file is still running: /home/pharma/public_html/var/report/1378880469402 2018-04-11T11:41:04.836+0200 DEBUG [prospector] log/prospector.go:361 Check file for harvesting: /home/pharma/public_html/var/report/222222222222 2018-04-11T11:41:04.836+0200 DEBUG [prospector] log/prospector.go:447 Update existing file for harvesting: /home/pharma/public_html/var/report/222222222222, ``` As well as a Back off message because it harvester reached the EOF. As i understand there is no matching happening with the patterns so the harvester is still looking for the matching pattern or found the matching pattern and looks for another that never comes(depending on the the before-after match value) or something is getting messed up with the characters. Anyone had any luck with this kind of messages? Also. I guess harvesters will keep monitoring these files as long as they exist. Is there a way to stop this from happening everytime they read the message(when that happens 😛 ). Found the close\_eof directives and some others but not sure if i should use them and how to use them properly yet. ---
**Author:** ![sancroth](https://avatars.discourse-cdn.com/v4/letter/s/838e76/32.png) [@sancroth](https://discuss.elastic.co/u/sancroth)\ **Post date:** [April 11, 2018, 11:10am UTC](https://discuss.elastic.co/t/missing-eol-character-cant-read-line/127609/2 "2018-04-11T11:10:13Z")
As i mentioned i tried getting the message without multiline and did not work. This was because i expected based on cat that the message is single line regardless of some posts that i read here and there. In the end i tried the obvious: > cat /home/pharma/public\_html/var/report/1111111111111 | wc -l Which returned the value **0**. If i am not mistaken `wc -l` counts the end of line characters. So if EOL is missing for some reason filebeat cannot really understand that this is the message and nothing more is coming, if i m guessing right. So how do we work around this if that is the case? Update: manually adding \r\n(or just an enter) to the end of the report file makes filebeat read and send the message. So how can we work around this if this is missing? Also what about stop polling the files after a message has been sent? ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [April 11, 2018, 2:20pm UTC](https://discuss.elastic.co/t/missing-eol-character-cant-read-line/127609/3 "2018-04-11T14:20:13Z")
@sancroth like you saw filebeat monitor file and watch for EOL (\n), without that it assume that the line is incomplete, Filebeat will automatically close an inactive [file after 5mins](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#close-inactive). You can also configure Filebeat to [ignore file older](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#ignore-older) than a certain time. ---
**Author:** ![sancroth](https://avatars.discourse-cdn.com/v4/letter/s/838e76/32.png) [@sancroth](https://discuss.elastic.co/u/sancroth)\ **Post date:** [April 11, 2018, 3:05pm UTC](https://discuss.elastic.co/t/missing-eol-character-cant-read-line/127609/4 "2018-04-11T15:05:32Z")
I ended up creating a bash script that looks for all ~/$projects/path/to/reports/ and appends an "" ,which creates a new line, to all files on that dir that were modified in the last minute(mmin -1). Then added the script on cron running every minute. If you can think of a better alternative let me know. Will add the ignore file and i think it should be fine. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 9, 2018, 3:05pm UTC](https://discuss.elastic.co/t/missing-eol-character-cant-read-line/127609/5 "2018-05-09T15:05:38Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.