Monitor state transistion of ongoing application

[schuessa]

I'm trying to set up the ability have some application log jobs such as

    2021-02-02T12:55:51-0500 [JOB:444] [START] My external job has started
    2021-02-02T12:55:51-0500 [JOB:444] [MY_EVENT] Some update to the job
    2021-02-02T12:55:51-0500 [JOB:444] [MY_EVENT] Some update to the job
    2021-02-02T12:55:51-0500 [JOB:444] [END] The job has finished successfully (or not)

And I want to be able to have logstash put these into ES such that via kibana I can see a dashboard of all the jobs (JOB:444, JOB:445 etc) that are running/finished/failed etc

I've looked at the aggregate plugin but it seems that will collapse all the events until the full START-END cycle is complete.

Is this a normal usage pattern for these tools (newbie here...)? To be able to look at the state of some application and update its status as incoming logs from the external application dictates?

[tatdat (Dat Pham Tat)]

I think you question is normal. You can done with grok, mutate filter and logstash condition.