I have a continuous stream of logs being ingested in ELK stack and created visualizations in Kibana view for monitoring purposes. The logs are ingested from an server application which receives HTTP requests from client.
One of the new monitoring requirement is to find out requests failed at time T1 is succeeded at time T2 or not. In the logs, we have timestamp, request ID and request status.
As seen here, there was a retry for the same object 20 minutes later. Using Kibana visualization I want to know if there was an object retry from client performed or not. As seen in the below table, column A have failed object names, while column B have succeeded object names. File4 and File8 object were not retried by the client.
Using Kibana, how can I find out such difference?
Right now I'm using a script that queries the data node using the index.
Can it be achieved using a plugin?
Because you are doing matching across multiple documents, which is a type of join, you need to do some custom scripting. It could be done in a Kibana plugin because you can make multiple requests and join the results.
That gives us a confidence to start looking at the Kibana plugins.
As per my understanding, here could be the flow for me - develop Kibana plugin, deploy, and use it to visualize the data I want.
Wylie, I'm still a noob in ELK ecosystem, but can you tell me which one is easy and feasible solution to my question? - Kibana plugin or vega visualization?
I don't think the thing you are trying to do will be easy with either approach. You will need to decide whether you want to learn how to use Vega or learn the Kibana plugin system.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.