# How to handle the log partialy updated **URL:** \ **Category:** Logstash\ **Created:** [September 7, 2017, 7:06pm UTC](https://discuss.elastic.co/t/how-to-handle-the-log-partialy-updated/99759 "2017-09-07T19:06:45Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![feijiangnan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/feijiangnan/32/20893_2.png) [@feijiangnan](https://discuss.elastic.co/u/feijiangnan)\ **Post date:** [September 7, 2017, 7:06pm UTC](https://discuss.elastic.co/t/how-to-handle-the-log-partialy-updated/99759/1 "2017-09-07T19:06:45Z")
Hi there, WebSphere Application Server has ffdc exception log, and it looks like below, when the ffdc event happened, it search the log first, if it find same exception, it updates count only, otherwise append the log to the end. Now, my problem is: Everytime, there is a new line appended, Filebeat resends all lines, to logstash, and in pie chart, I sum the the counts by exception, and the sum become extremely high eventually. My question: is Logstash/Filebeat capable to process this kind of logs? Index Count Time of first Occurrence Time of last Occurrence Exception SourceId ProbeId ------+------+---------------------------+---------------------------+--------------------------- 0 44 9/7/17 9:17:59:696 EDT 9/7/17 9:18:04:130 EDT com.ibm.CSIv2Security.CSIv2RequirementsNotSatisfied com.ibm.ISecurityLocalObjectCSIv2UtilityImpl.CSIv2EffectivePerformPolicy.getInstance 2106 /devZone/logs/wsqa/wasJVM/ffdc/server1\_591f46d8\_17.09.07\_09.17.59.7334786773500846563884.txt 1 11 9/7/17 9:17:59:827 EDT 9/7/17 9:18:04:132 EDT java.lang.reflect.InvocationTargetException com.ibm.ejs.oa.LocationService.runAsSystem 465 /devZone/logs/wsqa/wasJVM/ffdc/server1\_591f46d8\_17.09.07\_09.17.59.8308500032144821282524.txt 2 195 9/7/17 9:23:33:886 EDT 9/7/17 14:43:35:167 EDT javax.transaction.RollbackException com.ibm.tx.jta.impl.TransactionImpl.prepareResources 1505 /devZone/logs/wsqa/wasJVM/ffdc/server1\_17f46a58\_17.09.07\_09.23.33.8914122574254552181999.txt ------+------+---------------------------+---------------------------+--------------------------- Thanks and Regards, Fei ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 11, 2017, 8:48pm UTC](https://discuss.elastic.co/t/how-to-handle-the-log-partialy-updated/99759/2 "2017-09-11T20:48:27Z")
> Everytime, there is a new line appended, Filebeat resends all lines, to logstash Exactly how are lines appended? Are they literally appended or is the old and new data written to a new file that's renamed to replace the old file? That's very atypical for log files but would match the behavior you describe. ---
**Author:** ![feijiangnan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/feijiangnan/32/20893_2.png) [@feijiangnan](https://discuss.elastic.co/u/feijiangnan)\ **Post date:** [September 14, 2017, 6:27pm UTC](https://discuss.elastic.co/t/how-to-handle-the-log-partialy-updated/99759/3 "2017-09-14T18:27:41Z")
Thanks, Magnus Good question! I don't have the details how the file updated. But if you have something I can test, I would like to figure it out. Regards, Fei ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 12, 2017, 6:27pm UTC](https://discuss.elastic.co/t/how-to-handle-the-log-partialy-updated/99759/4 "2017-10-12T18:27:41Z")
Closed on October 12, 2017, 6:27pm UTC This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.