I have to process parts of the logging information using python script (python beautifulsoup and other useful stuff which I do not want to reimnplement in Ruby).
What is the simplest way to do this?
As of now I have found two ways:
- exec output plugin. In this case, my python script is launched after elasticsearch output and I can make additional processing and update event in Elasticsearch at the end (sending all the required ids needed to the script through cmdline)
- Create my own filter plugin for logstash and call python script from that plugin
It looks like second option is better in terms of performance (no need to do insert/update to Elastic, I will insert data to the database only once) but a little bit more complicated than the first option.
Do you have any other ideas?
Thanks in advance!