# Method to handle with false positive files

**URL:** <https://discuss.elastic.co/t/method-to-handle-with-false-positive-files/136248>\
**Category:** Logstash\
**Created:** [June 18, 2018, 7:16am UTC](https://discuss.elastic.co/t/method-to-handle-with-false-positive-files/136248 "2018-06-18T07:16:03Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![dyl](https://avatars.discourse-cdn.com/v4/letter/d/5fc32e/32.png) [@dyl](https://discuss.elastic.co/u/dyl)\
**Post date:** [June 18, 2018, 7:16am UTC](https://discuss.elastic.co/t/method-to-handle-with-false-positive-files/136248/1 "2018-06-18T07:16:03Z")

</div>

Hello ELK community.

I would like to know the way to handle with false positive files. Let me explain.

I use filebeat to listen a directory which contains several files. On these files, some of them are "false positive" files. Let me explain again ^^ :

On my log file, if the "CODE RETOUR" is -4 and the first column is only "INFO", this file is a false positive file. For exemple :

```
INFO;0000;000001; ******************************************************************************************************************
INFO;0000;000002;* LOG D'EXECUTION *
INFO;0000;000003;* /data/EDT/batchs/files/logs/FAC028MT-20180612-20180612-2108.log *
INFO;0000;000004;* 2018-06-12 21:08:28:348 *
INFO;0000;000005;* FAC028MT - Planification des moments de facturation *
INFO;0000;000006; ******************************************************************************************************************
INFO;0000;000007;
INFO;0000;000008; les paramètres du batch sont :
INFO;0000;000009; l'utilisateur (code : USER) : B-GF-UEM
INFO;0000;000010; le champ "FETCH_SIZE_VALUE" n'existe pas dans le fichier de propriétes du batch. utilisati
INFO;0000;000011; on de la valeur par défaut.
INFO;0000;000012;
INFO;0000;000013; Version de l'application : 13.13.100 (build : af9c96c, date : 09-05-2018 17:21:19)
INFO;0000;000014; Version de l'architecture : 4.143.500 (build : 879ab1c, date : 30-04-2018 09:42:03)
INFO;0000;000015; Version du framework : 4.143.500 (build : 879ab1c, date : 30-04-2018 09:42:03)
INFO;0000;000016; Version EDK : 4.131.500 (build : 1561e01, date : 02-05-2018 14:58:47)
INFO;0000;000017; Version ecore : 4.140.500 (build : 3eef259, date : 03-05-2018 15:49:45)
INFO;0000;000018; Utilisateur Oracle : HERMES_USER
INFO;0000;000019; info BDD : 13.13.100 / UEM / METZ
INFO;0000;000020;
INFO;0000;000021; paramètres d'exécution ajustables:
INFO;0000;000022; nombre de threads max : 8
INFO;0000;000023; taille des lots techniques : 10
INFO;0000;000024; commit actif : true
INFO;0000;000035; Contrat n°365722
INFO;0000;000036; Contrat n°379571
INFO;0000;000037; Contrat n°397833
INFO;0000;000038; Contrat n°404790
INFO;0000;000039; Contrat n°405373
INFO;0000;000040; Contrat n°405377
INFO;0000;000041; Contrat n°405380
INFO;0000;000060; statistiques job PlanificationMomentFacturationJob_2_21 :
INFO;0000;000061; nb de lots techniques traités : 111
INFO;0000;000062; durées - total : 3 s 95 ms , moyenne par lot : 0 s, max : 0 s [1], min : 0 s
INFO;0000;000063; statistiques job PlanificationMomentFacturationJob_7_24 :
INFO;0000;000140;
INFO;0000;000141; TEMPS D'EXECUTION : 0h 0' 13" 477ms
INFO;0000;000142;
INFO;0000;000143;CODE RETOUR : -4
INFO;0000;000144;
INFO;0000;000145; ******************************************************************************************************************

```

Only "INFO" in the first column and "CODE RETOUR : -4" at line 143. =\> false positive file.

Now this is a good file which interest me :

```
INFO;0000;000001; ******************************************************************************************************************
INFO;0000;000002;* LOG D'EXECUTION *
INFO;0000;000003;* /data/EDT/batchs/files/logs/MNS014MT-20180612-20180613-0105.log *
INFO;0000;000004;* 2018-06-13 01:05:43:448 *
INFO;0000;000005;* MNS014MT - Prélèvement et validation d'échéancier suite à saisie de RIB *
INFO;0000;000006; ******************************************************************************************************************
INFO;0000;000007;
INFO;0000;000016; Version EDK : 4.131.500 (build : 1561e01, date : 02-05-2018 14:58:47)
INFO;0000;000017; Version ecore : 4.140.500 (build : 3eef259, date : 03-05-2018 15:49:45)
INFO;0000;000018; Utilisateur Oracle : HERMES_USER
INFO;0000;000019; info BDD : 13.13.100 / UEM / METZ
INFO;0000;000020;
INFO;0000;000021; paramètres d'exécution ajustables:
INFO;0000;000034;
INFO;0000;000035; Traitement de l'offre 'fixo gaz uem' (id : G$3XVK)
MNOR;3011;000036; Erreur de traitement d'un élément
MNOR;3012;000037; Erreur lors de la mise en transaction
MNOR;4009;000038; aucune ligne de compte à mettre en transaction (compte:625316)
INFO;0000;000039;   
INFO;0000;000082; exécution batch de la requête 'Insertion des taxes' : 0h 0' 0" 113ms
INFO;0000;000083; exécution batch de la requête 'Mise à jour de l'échéancier de mensu' : 0h 0' 0" 18ms
INFO;0000;000084; exécution batch de la requête 'Insertion des lignes de compte' : 0h 0' 0" 240ms
INFO;0000;000093;
INFO;0000;000094; TEMPS D'EXECUTION : 0h 1' 48" 216ms
INFO;0000;000095;
INFO;0000;000096;CODE RETOUR : -4
INFO;0000;000097;
INFO;0000;000098; ******************************************************************************************************************

```

As you can see, "CODE retour: -4" but I have "MNOR" information in the first column. =\> good file.

So how to ask to logstash to wait to see the "CODE RETOUR" value and then see if it is only "INFO" value in the first column ?

For the moment I parse correctly my logs and drop undesirable lines.

Thx u 🙂

---

<div class="post-metadata">

**Author:** ![dyl](https://avatars.discourse-cdn.com/v4/letter/d/5fc32e/32.png) [@dyl](https://discuss.elastic.co/u/dyl)\
**Post date:** [June 18, 2018, 7:18pm UTC](https://discuss.elastic.co/t/method-to-handle-with-false-positive-files/136248/2 "2018-06-18T19:18:53Z")

</div>

Can somebody help me ? 🙂

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [June 18, 2018, 10:39pm UTC](https://discuss.elastic.co/t/method-to-handle-with-false-positive-files/136248/3 "2018-06-18T22:39:48Z")

</div>

You might have better luck getting a helpful answer in the [Beats Forum](https://discuss.elastic.co/c/beats).

When configured using the Beats Input plugin, Logstash receives a series of individual _events_ from a beat (in this case: Filebeat, which is looking for new _lines_ to be appended to files matching the configured pattern); it does not have the entire file in context.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 16, 2018, 10:40pm UTC](https://discuss.elastic.co/t/method-to-handle-with-false-positive-files/136248/4 "2018-07-16T22:40:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.