# Handle multiple loglines as batch with condition **URL:** \ **Category:** Logstash\ **Created:** [May 19, 2017, 7:37am UTC](https://discuss.elastic.co/t/handle-multiple-loglines-as-batch-with-condition/86371 "2017-05-19T07:37:59Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![zorkin82](https://avatars.discourse-cdn.com/v4/letter/z/54ee81/32.png) [@zorkin82](https://discuss.elastic.co/u/zorkin82)\ **Post date:** [May 19, 2017, 7:37am UTC](https://discuss.elastic.co/t/handle-multiple-loglines-as-batch-with-condition/86371/1 "2017-05-19T07:37:59Z")
Hello together, I'm looking for a solution for the following problem: - I have a logfile tat looks something like this: > 2017-03-24T13:18:07,147 INFO [topology-scan\_Worker-1]: Executing [PersistTopologyToZipCommand] > 2017-03-24T13:18:07,152 ERROR [topology-scan\_Worker-1]: Error during command execution: Error during topology scan execution! (BasicCommandExecutorImpl) > java.lang.NullPointerException > at java.util.Hashtable.put(Unknown Source) ~[?:1.8.0\_121] > 2017-03-24T13:18:07,159 INFO [topology-scan\_Worker-1]: Executing [2] fallback commands... > 2017-03-24T13:18:07,159 INFO [topology-scan\_Worker-1]: Executing [MovePersistedXmlTopologyFileCommand] > 2017-03-24T13:18:07,159 INFO [topology-scan\_Worker-1]: Moving topology file [topology-data-170324131807.tmp]... > 2017-03-24T13:18:07,160 INFO [topology-scan\_Worker-1]: Execution of command > 2017-03-24T13:18:07,160 INFO [topology-scan\_Worker-1]: Executing > 2017-03-24T13:18:07,165 INFO [topology-scan\_Worker-1]: Execution of command > 2017-03-24T13:18:07,165 INFO [topology-scan\_Worker-1]: Destroying SSHCommandExecutionContext... > 2017-03-24T13:18:32,751 INFO [topology-scan\_Worker-2]: Closing all SSH related artifacts if required... > 2017-03-24T13:18:32,751 INFO [topology-scan\_Worker-2]: Closing session output reader... > 2017-03-24T13:18:32,751 INFO [topology-scan\_Worker-2]: Closing ssh session... > 2017-03-24T13:18:32,760 INFO [topology-scan\_Worker-2]: Closing ssh client... > 2017-03-24T13:18:32,761 INFO [topology-scan\_Worker-2]: Finished topology scan execution > 2017-03-24T13:18:57,210 INFO [topology-scan\_Worker-3]: Starting topology scan execution... > 2017-03-24T13:18:57,210 INFO [topology-scan\_Worker-3]: Setting up SSHCommandExecutionContext... > 2017-03-24T13:18:57,211 INFO [topology-scan\_Worker-3]: Identifying object to scan... > 2017-03-24T13:18:57,211 INFO [topology-scan\_Worker-3]: Closing all SSH related artifacts if required... > 2017-03-24T13:18:57,573 INFO [topology-scan\_Worker-3]: Trying SSH connect: ( > 2017-03-24T13:18:58,454 INFO [topology-scan\_Worker-3]: Authentication result: COMPLETE > 2017-03-24T13:19:01,709 INFO [topology-scan\_Worker-3]: Destroying SSHCommandExecutionContext... > 2017-03-24T13:19:01,710 INFO [topology-scan\_Worker-3]: Closing all SSH related artifacts if required... > 2017-03-24T13:19:01,710 INFO [topology-scan\_Worker-3]: Closing session output reader... > 2017-03-24T13:19:01,710 INFO [topology-scan\_Worker-3]: Closing ssh session... > 2017-03-24T13:19:01,712 INFO [topology-scan\_Worker-3]: Closing ssh client... > 2017-03-24T13:19:01,712 INFO [topology-scan\_Worker-3]: Finished topology scan execution - the log ist grouped by "batches" that are identified by a worker ( [topology-scan\_Worker-3] ) - if an error occurs in one worker process ("batch") I need to handle the complete log batch in a separate way - the logs are stored in elasticsearch So based on the given log example I want to do: 1. All Log lines with the identifier **[topology-scan\_Worker-3]** should be send to a error index in elasticsearch because there was a **ERROR** inside the batch 2. All other lines should be send to another index in elasticsearch Are there any suggestions how to solve this with logstash or if it is unpossible. Best regards Dirk ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [May 19, 2017, 10:32am UTC](https://discuss.elastic.co/t/handle-multiple-loglines-as-batch-with-condition/86371/2 "2017-05-19T10:32:04Z")
Logstash as it is now is not able to do this. We treat every event as if we have never seen any other events before - it is stateless. In terms of your data - is the above section a complete log or part of a stream of events? I can't see any relationship between `topology-scan_Worker-1`, `topology-scan_Worker-2` and `topology-scan_Worker-3` in the data. As a human I can but as a machine I could not. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 16, 2017, 10:32am UTC](https://discuss.elastic.co/t/handle-multiple-loglines-as-batch-with-condition/86371/3 "2017-06-16T10:32:25Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.