Nagios Performance data => Kafka Topic => Elastic search(Need to process Nagios performance data)

[ranjithkodumbu (Ranjith Kumar)]

Hello team,

I am trying to process Nagios performance data to ELK using logstash, I am receiving Nagios performance data format below, need to process and update it to the Elastic index.

{"check_type": "service", "check_time": 1647249138.689, "hostname": "wins2016s", "state": "0", "servicename": "Swap Usage", "output": "OK: Swap usage was 53.80 % (Total: 8.93 GiB, Used: 4.80 GiB, Free: 4.13 GiB) | 'total'=8.93GiB;;; 'used'=4.80GiB;;; 'free'=4.13GiB;;;"}

I am able to view data in elastic index and filter with check_type, hostname etc. But I need to split performance data after the | ('total'=8.93GiB;;; 'used'=4.80GiB;;; 'free'=4.13GiB;;;") and write it into something like below
Nagios performance data format is like this:
current value with Unit;Warning value;Critical Value;Min value;Max value
e.g. here. total= 8.93 is the current value and Unit is GiB and there is no warning and critical value here
so, I need to process the all perf data and write it into the index like this.
Trying something like this without calling perl script: Convert Nagios perfdata string to JSON | Philippe Lewin. By doing this I will be able to write condition based on the value and alert it.
######################
[
{
"value": 8.93,
"min": "null",
"label": "total",
"uom": "GiB",
"warning": null,
"max": null,
"critical": null
},
{
"value": 4.80,
"min": "null",
"label": "used",
"uom": "GiB",
"warning": null,
"max": null,
"critical": null
},
{
"value": 4.13,
"min": "null",
"label": "total",
"uom": "GiB",
"warning": null,
"max": null,
"critical": null
}
]

#####################

Thanks,
Ranjith

[Badger]

You could do something like

    grok {
        break_on_match => false
        keep_empty_captures => true
        match => {
            "output" => [
                "'total'=%{NUMBER:[total][value]:float}%{DATA:[total][uom]};(%{NUMBER:[total][warning]:float}|)?;(%{NUMBER:[total][critical]:float}|)?;((%{NUMBER:[total][min]:float};)?%{NUMBER:[total][max]:float};)?( |$)",
                "'used'=%{NUMBER:[used][value]:float}%{DATA:[used][uom]};(%{NUMBER:[used][warning]:float}|)?;(%{NUMBER:[used][critical]:float}|)?;((%{NUMBER:[used][min]:float};)?%{NUMBER:[used][max]:float};)?( |$)",
                "'free'=%{NUMBER:[free][value]:float}%{DATA:[free][uom]};(%{NUMBER:[free][warning]:float}|)?;(%{NUMBER:[free][critical]:float}|)?;((%{NUMBER:[free][min]:float};)?%{NUMBER:[free][max]:float};)?( |$)"

            ]
        }
        add_field => {
            "[total][label]" => "total"
            "[used][label]" => "used"
            "[free][label]" => "free"
        }
    }
    ruby {
        code => '
            s = []
            s << event.remove("total")
            s << event.remove("used")
            s << event.remove("free")
            event.set("someField", s)
        '
    }

but I think that final ruby filter will make it far harder to process the data.

[ranjithkodumbu (Ranjith Kumar)]

Hi Badger, thanks for the response. Data is dynamic, it will not be always the one I just gave an example. So, is there anyway to execute perl script in logstash and output should be written into index etc. I am looking for something in this blog Convert Nagios perfdata string to JSON | Philippe Lewin .

e.g. perl perfdatatojson.pl "rta=0.038000ms;5000.000000;5000.000000;0.000000 pl=0%;100;100;0"


so, if I can execute the perl script in logstash, I just need to pass performance data as a input to the script, which should write into a index.

Note: Performance output will be like this format. current value with Unit;Warning value;Critical Value;Min value;Max value. In this format there may be even 10 items or more than that which I need to run it through iterations and format it. In this case perl script will take care of iterations, I just need to pass it as a input.

Thanks,

Ranjith.

[Badger]

In a ruby filter you could fork and exec the perl script. It would probably be very expensive to do so.

I suggest you take a look at wrapping that perl script in a REST API and then using an http filter to call it.

[ranjithkodumbu (Ranjith Kumar)]

Hi Badger, thanks for your suggestion. Is there any other way which we can achieve this without script.

Thanks,

Ranjith