# How to find difference in ingested logs using Kibana visualization? **URL:** \ **Category:** Kibana\ **Created:** [April 20, 2020, 11:24pm UTC](https://discuss.elastic.co/t/how-to-find-difference-in-ingested-logs-using-kibana-visualization/228944 "2020-04-20T23:24:16Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![rajdevworks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajdevworks/32/66719_2.png) [@rajdevworks](https://discuss.elastic.co/u/rajdevworks)\ **Post date:** [April 20, 2020, 11:24pm UTC](https://discuss.elastic.co/t/how-to-find-difference-in-ingested-logs-using-kibana-visualization/228944/1 "2020-04-20T23:24:16Z")
I have a continuous stream of logs being ingested in ELK stack and created visualizations in Kibana view for monitoring purposes. The logs are ingested from an server application which receives HTTP requests from client. One of the new monitoring requirement is to find out requests failed at time T1 is succeeded at time T2 or not. In the logs, we have timestamp, request ID and request status. Example: Failure request log ```auto { "time_start": "10/Apr/2020:16:36:05 +0000", "status": 500, "request_id": "7974457c", "object_name": "fileA.txt", } ``` Success request log ```auto { "time_start": "10/Apr/2020:16:56:59 +0000", "status": 200, "request_id": "3e35bd25", } ``` As seen here, there was a retry for the same object 20 minutes later. Using Kibana visualization I want to know if there was an object retry from client performed or not. As seen in the below table, column A have failed object names, while column B have succeeded object names. File4 and File8 object were not retried by the client. [![enter image description here](https://us1.discourse-cdn.com/elastic/original/3X/d/b/db44f65384fe7e1c0dcbc687a7be84ed0dcaedac.png)](https://i.stack.imgur.com/gDn7P.png) Using Kibana, how can I find out such difference? Right now I'm using a script that queries the data node using the index. Can it be achieved using a plugin? ---
**Author:** ![wylie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wylie/32/81794_2.png) [@wylie](https://discuss.elastic.co/u/wylie)\ **Post date:** [April 22, 2020, 7:07pm UTC](https://discuss.elastic.co/t/how-to-find-difference-in-ingested-logs-using-kibana-visualization/228944/2 "2020-04-22T19:07:55Z")
Because you are doing matching across multiple documents, which is a type of join, you need to do some custom scripting. It could be done in a Kibana plugin because you can make multiple requests and join the results. ---
**Author:** ![rajdevworks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajdevworks/32/66719_2.png) [@rajdevworks](https://discuss.elastic.co/u/rajdevworks)\ **Post date:** [April 22, 2020, 8:15pm UTC](https://discuss.elastic.co/t/how-to-find-difference-in-ingested-logs-using-kibana-visualization/228944/3 "2020-04-22T20:15:55Z")
That gives us a confidence to start looking at the Kibana plugins. As per my understanding, here could be the flow for me - develop Kibana plugin, deploy, and use it to visualize the data I want. Correct me if I'm wrong or bless me. Thank you. ---
**Author:** ![wylie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wylie/32/81794_2.png) [@wylie](https://discuss.elastic.co/u/wylie)\ **Post date:** [April 22, 2020, 8:58pm UTC](https://discuss.elastic.co/t/how-to-find-difference-in-ingested-logs-using-kibana-visualization/228944/4 "2020-04-22T20:58:28Z")
The only other option that might exist for you is to build a custom Vega visualization: this is not easy to do, but theoretically possible. ---
**Author:** ![rajdevworks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajdevworks/32/66719_2.png) [@rajdevworks](https://discuss.elastic.co/u/rajdevworks)\ **Post date:** [April 22, 2020, 10:06pm UTC](https://discuss.elastic.co/t/how-to-find-difference-in-ingested-logs-using-kibana-visualization/228944/5 "2020-04-22T22:06:44Z")
Wylie, I'm still a noob in ELK ecosystem, but can you tell me which one is easy and feasible solution to my question? - Kibana plugin or vega visualization? ---
**Author:** ![wylie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wylie/32/81794_2.png) [@wylie](https://discuss.elastic.co/u/wylie)\ **Post date:** [April 22, 2020, 10:13pm UTC](https://discuss.elastic.co/t/how-to-find-difference-in-ingested-logs-using-kibana-visualization/228944/6 "2020-04-22T22:13:02Z")
I don't think the thing you are trying to do will be easy with either approach. You will need to decide whether you want to learn how to use Vega or learn the Kibana plugin system. ---
**Author:** ![rajdevworks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajdevworks/32/66719_2.png) [@rajdevworks](https://discuss.elastic.co/u/rajdevworks)\ **Post date:** [April 24, 2020, 4:03am UTC](https://discuss.elastic.co/t/how-to-find-difference-in-ingested-logs-using-kibana-visualization/228944/7 "2020-04-24T04:03:28Z")
Wylie, are there any examples to look at that might help me? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 22, 2020, 4:03am UTC](https://discuss.elastic.co/t/how-to-find-difference-in-ingested-logs-using-kibana-visualization/228944/8 "2020-05-22T04:03:30Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.