# Nagios Performance data =\> Kafka Topic =\> Elastic search(Need to process Nagios performance data) **URL:** \ **Category:** Logstash\ **Created:** [March 14, 2022, 9:27am UTC](https://discuss.elastic.co/t/nagios-performance-data-kafka-topic-elastic-search-need-to-process-nagios-performance-data/299594 "2022-03-14T09:27:08Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![ranjithkodumbu](https://avatars.discourse-cdn.com/v4/letter/r/eb8c5e/32.png) [@ranjithkodumbu](https://discuss.elastic.co/u/ranjithkodumbu)\ **Post date:** [March 14, 2022, 9:27am UTC](https://discuss.elastic.co/t/nagios-performance-data-kafka-topic-elastic-search-need-to-process-nagios-performance-data/299594/1 "2022-03-14T09:27:08Z")
Hello team, I am trying to process Nagios performance data to ELK using logstash, I am receiving Nagios performance data format below, need to process and update it to the Elastic index. {"check\_type": "service", "check\_time": 1647249138.689, "hostname": "wins2016s", "state": "0", "servicename": "Swap Usage", "output": "OK: Swap usage was 53.80 % (Total: 8.93 GiB, Used: 4.80 GiB, Free: 4.13 GiB) | 'total'=8.93GiB;;; 'used'=4.80GiB;;; 'free'=4.13GiB;;;"} I am able to view data in elastic index and filter with check\_type, hostname etc. But I need to split performance data after the | ('total'=8.93GiB;;; 'used'=4.80GiB;;; 'free'=4.13GiB;;;") and write it into something like below Nagios performance data format is like this: current value with Unit;Warning value;Critical Value;Min value;Max value e.g. here. total= 8.93 is the current value and Unit is GiB and there is no warning and critical value here so, I need to process the all perf data and write it into the index like this. Trying something like this without calling perl script: [Convert Nagios perfdata string to JSON | Philippe Lewin](http://philippe.lewin.me/2014/09/29/convert-nagios-perfdata-to-json/). By doing this I will be able to write condition based on the value and alert it. ###################### [ { "value": 8.93, "min": "null", "label": "total", "uom": "GiB", "warning": null, "max": null, "critical": null }, { "value": 4.80, "min": "null", "label": "used", "uom": "GiB", "warning": null, "max": null, "critical": null }, { "value": 4.13, "min": "null", "label": "total", "uom": "GiB", "warning": null, "max": null, "critical": null } ] ##################### Thanks, Ranjith ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [March 14, 2022, 7:54pm UTC](https://discuss.elastic.co/t/nagios-performance-data-kafka-topic-elastic-search-need-to-process-nagios-performance-data/299594/2 "2022-03-14T19:54:08Z")
You could do something like ``` grok { break_on_match => false keep_empty_captures => true match => { "output" => [ "'total'=%{NUMBER:[total][value]:float}%{DATA:[total][uom]};(%{NUMBER:[total][warning]:float}|)?;(%{NUMBER:[total][critical]:float}|)?;((%{NUMBER:[total][min]:float};)?%{NUMBER:[total][max]:float};)?( |$)", "'used'=%{NUMBER:[used][value]:float}%{DATA:[used][uom]};(%{NUMBER:[used][warning]:float}|)?;(%{NUMBER:[used][critical]:float}|)?;((%{NUMBER:[used][min]:float};)?%{NUMBER:[used][max]:float};)?( |$)", "'free'=%{NUMBER:[free][value]:float}%{DATA:[free][uom]};(%{NUMBER:[free][warning]:float}|)?;(%{NUMBER:[free][critical]:float}|)?;((%{NUMBER:[free][min]:float};)?%{NUMBER:[free][max]:float};)?( |$)" ] } add_field => { "[total][label]" => "total" "[used][label]" => "used" "[free][label]" => "free" } } ruby { code => ' s = [] s << event.remove("total") s << event.remove("used") s << event.remove("free") event.set("someField", s) ' } ``` but I think that final ruby filter will make it far harder to process the data. ---
**Author:** ![ranjithkodumbu](https://avatars.discourse-cdn.com/v4/letter/r/eb8c5e/32.png) [@ranjithkodumbu](https://discuss.elastic.co/u/ranjithkodumbu)\ **Post date:** [March 15, 2022, 8:27pm UTC](https://discuss.elastic.co/t/nagios-performance-data-kafka-topic-elastic-search-need-to-process-nagios-performance-data/299594/3 "2022-03-15T20:27:11Z")
Hi Badger, thanks for the response. Data is dynamic, it will not be always the one I just gave an example. So, is there anyway to execute perl script in logstash and output should be written into index etc. I am looking for something in this blog [Convert Nagios perfdata string to JSON | Philippe Lewin](http://philippe.lewin.me/2014/09/29/convert-nagios-perfdata-to-json/) . ```auto e.g. perl perfdatatojson.pl "rta=0.038000ms;5000.000000;5000.000000;0.000000 pl=0%;100;100;0" so, if I can execute the perl script in logstash, I just need to pass performance data as a input to the script, which should write into a index. Note: Performance output will be like this format. current value with Unit;Warning value;Critical Value;Min value;Max value. In this format there may be even 10 items or more than that which I need to run it through iterations and format it. In this case perl script will take care of iterations, I just need to pass it as a input. Thanks, Ranjith. ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [March 15, 2022, 11:32pm UTC](https://discuss.elastic.co/t/nagios-performance-data-kafka-topic-elastic-search-need-to-process-nagios-performance-data/299594/4 "2022-03-15T23:32:47Z")
In a ruby filter you could fork and exec the perl script. It would probably be very expensive to do so. I suggest you take a look at wrapping that perl script in a REST API and then using an http filter to call it. ---
**Author:** ![ranjithkodumbu](https://avatars.discourse-cdn.com/v4/letter/r/eb8c5e/32.png) [@ranjithkodumbu](https://discuss.elastic.co/u/ranjithkodumbu)\ **Post date:** [March 16, 2022, 5:13am UTC](https://discuss.elastic.co/t/nagios-performance-data-kafka-topic-elastic-search-need-to-process-nagios-performance-data/299594/5 "2022-03-16T05:13:20Z")
Hi Badger, thanks for your suggestion. Is there any other way which we can achieve this without script. Thanks, Ranjith ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 13, 2022, 5:14am UTC](https://discuss.elastic.co/t/nagios-performance-data-kafka-topic-elastic-search-need-to-process-nagios-performance-data/299594/6 "2022-04-13T05:14:15Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.