Parse custom log file to get metrics

[Etienne.lv (Etienne)]

Hi !

I'm new on ELK environment, so I apologize for asking some "obvious" things.

To understand my problem, this is the context :

• I wrote an application and deploy it on 2 VMs "server". (in the future i would like to deploy it on multiple VMs, maybe 10/20/30).
  --> Each application populate two log files locally : application.log and application-admin.log

Each "server" VMs receive requests from different clients and populate application.log file with specific line to store "requests" action.

My idea is :

1. To set up an OS monitoring (on each VM)
2. To set up an Application Monitoring (on each VM and in the future : one by host)

I have built a VM with : E + L + K.

I deployed on my client : Metricbeat to perform OS monitoring and Filebeat to perform Application monitoring. (i send onyl the "application.log" file to logstash, the administrator one is useless). The Metricbeat is OK, but the application monitoring is much more complicated for me.

I would like to parse my application.log file to set up metric on two specific lines to count the number of times each event occurs and number of times each event occures per minute. (to display, at the end some graphs on kibana and to be able to monitor the number of request/action each server perform)

I don't know if logstash can do want i want to do and how to do it "simply".

My first idea was to create a filter in logstash configuration to match (with grok and a regex on "message" field) the 2 particular lines i want to monitor but i feel completly lost on how to do it and what to do after that.

If you have any idea to get me on track i would be grateful.

Thanks

Etienne

[Badger]

You can count events and get a per-minute rate using a metrics filter.

[Etienne.lv (Etienne)]

Thanks for you answer.

I had some configuration line in my filter:

if [message] =~ /.*Request.*/ {
        metrics {
                meter => "request"
                add_tag => "metric"
                }
} 

Do i have something to change in my output block ? Something to add in ElasticSearch or Kibana side to retrieve these stats in kibana and build graph ?

Thanks

Etienne

[Badger]

If you search for documents in Kibana you should get metric events that you can visualize.

[Etienne.lv (Etienne)]

Thanks for your help i found what i was lokking for.

But i've another issue/answer.

For each logline (filter with grok), i would like to send multiple metrics ?

To explain:

My log is a "request" --> i would like to monitor it with a "requestMetric".

But inside this request i've an array:

responses: [ { "name": "Input", type": "Good"}, { "name": "Output", "type":"Bad" } ]

I would like to also monitor name and type. (input and output for name and Good or Bad for type) in "nameMetric" and "inputMetric".

To be as precise as possible:
Name have 3 possible value.
Input have 4 possible value.

To be more complicated reponses is a dynamic array...

Thanks for your help

Etienne