Hi,
I am evaluating ELK stack to use for one of our projects. And also new in ELK stack.
Let me explain the use case for which I am looking for a reference for ELK.
We have multiple components, C1,C2,C3 etc in our application and each one contains different classes. All components talk to each other via ESB and Message Queues. One component, C3 also talk to 3rd party systems. We measure the execution time for each methods inside classes of each component while it is in a specific component(we call it Log Point -LP) and also in MQ or getting response from third parties. Please note that not all functionality executes each LP so each transactions are having different LP. Which gives us a behavior like
{TXId, LogPoint, Entry Timestamp}
{“1445“,”LP1“,”100015“}
{“1445“,”LP4“,”100025“}
{“1445“,”LP5“,”100026“}
{“1380“,”LP1“,”100012“}
{“1380“,”LP4“,”100013“}
{“1380“,”LP5“,”100020“}
{“1805“,”LP1“,”100009“}
{“1805“,”LP4“,”100014“}
{“1805“,”LP5“,”100022“}
{“1499“,”LP1“,”100017“}
{“1499“,”LP4“,”100022“}
{“1499“,”LP5“,”100028“}
{“1768“,”LP1“,”100016“}
{“1768“,”LP4“,”100023“}
This gives around a million of records in each day. We need to analyze with queries like
-
give us the the number of transactions which took 100 ms between LP5 and LP4
-
give us the the number of transactions which took 3 sec between LP125(i.e. end logpoint for a specific functionality) and LP1(Start logpoint) on peak load time for example 7 PM to 9 PM in the evening for last one week.
3)raise alarm if any transaction take more than 6 sec on peak load time
Most importantly,
4)Showing time difference between LP1 and LP2 for each transaction. Putting it on a graph where one axis is the difference, the other one is the time when LP1 occurred. we don't want to see the transactions that have LP1 but no LP2.
Can you please redirect me to any of document/link/ video which solves this sort of problem with a recommended datamodel?
Thanks in advance !