Query on multiple indexes and generate the dashboard in Kibana

Elastic Stack Kibana
1

My requirement is to build Dashboards to monitor the transaction flow between Two system with middleware as interface to connect both the systems. Each event is captured in elastic search indexes.
System 1
Interface (Middleware)
System 2

Sample Transaction flow as below:
Step1:
System 1 -> Interface -> System 2
Step2:
System 2 -> Interface -> System 1

Sample Log:
Step1:
System1 Log -> {"TxnRef": "12345","System1Status":"TxnPlaced","System1TimeStamp":"dd:mm:yyyy hh:mm:ss"}
Interface Log-> {"TxnRef":"12345","InterfaceStatus":"Interfaced2System2","dd:mm:yyyy hh:mm:ss"}
System 2 Log -> {"TxnRef": "12345","System2Status":"TxnProcessed","System2TimeStamp":"dd:mm:yyyy hh:mm:ss"}

Use Case1:
How do i build a query to get the latest status based on the status above example i need to retrieve the result as TxnProcessed for transaction ref 12345
Expected Results:
TxnRef: 12345
Status: TxnProcessed

Use Case2:
To get the processing time between each processing to identify the lagging processing

System1 to Interface timetaken = Interface Timestamp - System 1 TimeStamp
Interface to System2 processing = System 2 Timestamp - Interface Timestamp

Overall Processing Time = System2 Timestamp - System1 Timestamp

Kibana dashboard to be built based on the status
Example: Dashboard 1 Transaction with status as "TxnPlaced"
Dashboard 2 Transaction with status as "Interfaced2System2"
Dashboard 3 Transaction with status as "TxnProcessed"

I need your suggestion on how to achieve this case in elastic search. Hope i have explained to some extent on my requirement.

2

What you need to do is to create a transform job that is calculating your use cases and storing that result in a separate index.
Then you use that index to visualize the results.

3

Thanks for your reply Felix_Roessel. Since I’m pretty new to elasticsearch please give some more insights on transform of index option available in elastic search or need to create api outside to do transformation.

4

Here you find everything you need

Let me know if you have further questions

5

Thanks for your quick reply.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.