# Lancer un script python depuis une config Logstash **URL:** \ **Category:** Discussions en français\ **Tags:** elastic-stack-monitoring\ **Created:** [March 13, 2019, 10:24am UTC](https://discuss.elastic.co/t/lancer-un-script-python-depuis-une-config-logstash/172119 "2019-03-13T10:24:09Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Radic4l](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radic4l/32/37335_2.png) [@Radic4l](https://discuss.elastic.co/u/Radic4l)\ **Post date:** [March 13, 2019, 10:24am UTC](https://discuss.elastic.co/t/lancer-un-script-python-depuis-une-config-logstash/172119/1 "2019-03-13T10:24:09Z")
Salut la communauté ! Voilà j'ai un petit problème ... Encore 😃 J'ai créer un script python pour supprimer les doublons de mon index, dans le but de comparer des IP (par doublons) si l'IP existe dans deux bases de données différentes alors ils apparaissent en double dans mon index. il me suffit ensuite de supprimer tous les doublons pour obtenir une liste des IPs n'apparaissant que dans une seul liste. Voici le script: ```auto from elasticsearch import Elasticsearch # Settings es = Elasticsearch('http://localhost:9000/') def search_datas(): # Corp de la requête pour rechercher les doublons search_query = { "aggs": { "dup": { "terms": { "size": 20000, "field": "ip_commune.keyword", "min_doc_count": 2 } } } } # La requête qui va chercher tous les doublons dans l'index 'index-ip' search_result = es.search(index="index-ip", body=search_query, doc_type='doc', filter_path=['aggregations.dup.buckets.key']) new_datas = [] try: datas = search_result.get('aggregations').get('dup').get('buckets') for key in datas: value = key.get('key') new_datas.append(value) return new_datas except: pass def delete_datas(datas): # Corp de la requête pour effacer tous les doublons delete_query = { "query": { "terms": {"ip_commune.keyword": datas} } } if datas != None: # La requête qui va chercher tous les doublons dans l'index 'index-ip' es.delete_by_query(index="index-ip",body=delete_query, doc_type='doc') else: print('Pas de doublons') pass delete_datas(search_datas()) ``` J'aimerais pouvoir le lancer dès que ma config Logstash d'aggregations des IP est lancer. j'avais pensé à quelque chose du genre du filtre ruby: ```auto filter { ruby{ path => "/root/scripts/py_scripts/ip_compare.py" } } ``` mais pour python 😕 Deux point ce soulève alors, 1. est-ce possible de lancer ip\_compart.py depuis la config logstash genre : ```auto # Fichier pour comparaison d'adresse IP input { jdbc { jdbc_driver_library => "/usr/share/java/mysql-connector-java-8.0.15.jar" jdbc_driver_class => "com.mysql.jdbc.Driver" jdbc_connection_string => "jdbc:mysql://glpi_server:3306/my_db?useSSL=true&verifyServerCertificate=false&requireSSL=true" jdbc_user => "surcouche" jdbc_password_filepath => "/etc/mysql-ssl/.pcw" jdbc_validate_connection => "true" tracking_column_type => "timestamp" tracking_column => "date_mod" tags => ["ip-glpi"] # schedule => "* * * * *" statement => "SELECT DISTINCT name as ip_commune FROM glpi_ipaddresses where name not in ('','::','::1','::1/128','0.0.0.0','127.0.0.1')" clean_run => "true" } jdbc { jdbc_driver_library => "/usr/share/java/mysql-connector-java-8.0.15.jar" jdbc_driver_class => "com.mysql.jdbc.Driver" jdbc_connection_string => "jdbc:mysql://ipam_server:3306/my_db?useSSL=true&verifyServerCertificate=false&requireSSL=true" jdbc_user => "surcouche" jdbc_password_filepath => "/etc/mysql-ssl/.pcw" jdbc_validate_connection => "true" # schedule => "* * * * *" tags => ["ip-ipam"] statement => "SELECT DISTINCT ip_addr as ip_commune FROM devices" clean_run => "true" } } filter { ruby{ path => "/root/scripts/py_scripts/ip_compare.py" } } output { if "ip-glpi" in [tags] or "ip-ipam" in [tags] { elasticsearch { hosts => ["127.0.0.1:9000"] index => "index-ip" } } } ``` 1. et si j'utilise une technique similaire le script ne risque t'il pas de se lancer avant le stockage des données dans Elasticsearch ? 😕 Je me demande quelle serais la meilleurs solution ! si vous avez un avis, il est bienvenue ! =) ---
**Author:** ![gabriel\_tessier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gabriel_tessier/32/27911_2.png) [@gabriel\_tessier](https://discuss.elastic.co/u/gabriel_tessier)\ **Post date:** [March 17, 2019, 1:36am UTC](https://discuss.elastic.co/t/lancer-un-script-python-depuis-une-config-logstash/172119/2 "2019-03-17T01:36:20Z")
Bonjour, Y'a une discussion similaire ici juste au cas ou. > [@How to call python script from Logstash to process (partialy) logs](https://discuss.elastic.co/t/how-to-call-python-script-from-logstash-to-process-partialy-logs/40009): > > Hello, I have to process parts of the logging information using python script (python beautifulsoup and other useful stuff which I do not want to reimnplement in Ruby). What is the simplest way to do this? As of now I have found two ways: exec output plugin. In this case, my python script is launched after elasticsearch output and I can make additional processing and update event in Elasticsearch at the end (sending all the required ids needed to the script through cmdline) Create my own fi… ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 14, 2019, 1:49am UTC](https://discuss.elastic.co/t/lancer-un-script-python-depuis-une-config-logstash/172119/3 "2019-04-14T01:49:04Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.