Filtering rows from a log based in their last status

[ManuelChaverra]

Hello everyone, this is my first time using Elastic Stack so I'm trying to learn a lot of things that I'd like to implement in my work.

I'm using Filebeat to send logs files to logstash, then I'm filtering information and sending it to elasticsearch to finally be able to review it in Kibana (So Filebeat > Logstash > Elasticsearch > Kibana)

But I'm have the following problem:
My log file looks like this:

I need to extract the rows corresponding to the last status of each process, for example, if I have the following rows:

05/15/2020 09:10:41|Maquina_18|RPA202_Elastic|ElasticLogGenerator.atmx|https://3gjn1nu1.ce.automationanywhere.digital|Info|0 - Proceso finalizado correctamente
05/15/2020 10:15:27|Maquina_18|RPA202_Elastic|ElasticLogGenerator.atmx|https://3gjn1nu1.ce.automationanywhere.digital|Error|574 - Error crítico no controlado
05/15/2020 09:47:47|Maquina_5|RPA313_Elastic|ElasticLogGenerator.atmx|https://3gjn1nu1.ce.automationanywhere.digital|Warning|100 - Error controlado
05/15/2020 10:07:33|Maquina_5|RPA313_Elastic|ElasticLogGenerator.atmx|https://3gjn1nu1.ce.automationanywhere.digital|Info|0 - Proceso finalizado correctamente

I need to return the following since they contain the last status (based on the date) of the process (RPA202 and RPA313 respectively) and ran in the same machine (Maquina_18 and Maquina_5 respectively):

05/15/2020 10:15:27|Maquina_18|RPA202_Elastic|ElasticLogGenerator.atmx|https://3gjn1nu1.ce.automationanywhere.digital|Error|574 - Error crítico no controlado
05/15/2020 10:07:33|Maquina_5|RPA313_Elastic|ElasticLogGenerator.atmx|https://3gjn1nu1.ce.automationanywhere.digital|Info|0 - Proceso finalizado correctamente

So, to achieve this, I tried a dissect filter in the logstash config file to indicate the pipes as separators and then use Kibana filters to get the information as I need it, but I haven't achieve it.

My configuration file looks like this:

I would appreciate a lot any advice or hint to achieve it

Thanks in advance! I hope I make myself clear because English is not my first language, so I'm sorry If I had any mistake writing this.

[Badger]

If you want to do it in logstash (rather than by using an elasticsearch query) then you could use an aggregate filter to determine the last message. See example 3.

Make sure you set pipeline.workers to 1 and disable java_execution, otherwise events get processed out of order and last is no longer last.