I'm new on ELK environment, so I apologize for asking some "obvious" things.
To understand my problem, this is the context :
- I wrote an application and deploy it on 2 VMs "server". (in the future i would like to deploy it on multiple VMs, maybe 10/20/30).
--> Each application populate two log files locally : application.log and application-admin.log
Each "server" VMs receive requests from different clients and populate application.log file with specific line to store "requests" action.
My idea is :
- To set up an OS monitoring (on each VM)
- To set up an Application Monitoring (on each VM and in the future : one by host)
I have built a VM with : E + L + K.
I deployed on my client : Metricbeat to perform OS monitoring and Filebeat to perform Application monitoring. (i send onyl the "application.log" file to logstash, the administrator one is useless). The Metricbeat is OK, but the application monitoring is much more complicated for me.
I would like to parse my application.log file to set up metric on two specific lines to count the number of times each event occurs and number of times each event occures per minute. (to display, at the end some graphs on kibana and to be able to monitor the number of request/action each server perform)
I don't know if logstash can do want i want to do and how to do it "simply".
My first idea was to create a filter in logstash configuration to match (with grok and a regex on "message" field) the 2 particular lines i want to monitor but i feel completly lost on how to do it and what to do after that.
If you have any idea to get me on track i would be grateful.