# To find all the correlation id where response is not send back to the consumer

**URL:** <https://discuss.elastic.co/t/to-find-all-the-correlation-id-where-response-is-not-send-back-to-the-consumer/251896>\
**Category:** Elasticsearch\
**Created:** [October 13, 2020, 11:46am UTC](https://discuss.elastic.co/t/to-find-all-the-correlation-id-where-response-is-not-send-back-to-the-consumer/251896 "2020-10-13T11:46:43Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![rohitarorait82](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohitarorait82/32/82981_2.png) [@rohitarorait82](https://discuss.elastic.co/u/rohitarorait82)\
**Post date:** [October 13, 2020, 11:46am UTC](https://discuss.elastic.co/t/to-find-all-the-correlation-id-where-response-is-not-send-back-to-the-consumer/251896/1 "2020-10-13T11:46:43Z")

</div>

Hi All,

In our logs; For a transaction , two legs will get generated "Consumer to backend" and "Backend to Consumer" .However in case of failure , only one leg "Consumer to backend" will get generated.

Unique correlation id will get generated for each transaction consisting of these two legs.  
Is there a way to find out all the correlation id for which second leg is not generated.

---

<div class="post-metadata">

**Author:** ![mastersmit](https://avatars.discourse-cdn.com/v4/letter/m/f475e1/32.png) [@mastersmit](https://discuss.elastic.co/u/mastersmit)\
**Post date:** [October 13, 2020, 12:22pm UTC](https://discuss.elastic.co/t/to-find-all-the-correlation-id-where-response-is-not-send-back-to-the-consumer/251896/2 "2020-10-13T12:22:27Z")

</div>

you may use Logstash Aggregation to aggregate by correlationid.

And as many events you receive for that correlationid you may increase the count.

if the count is 2, you could know it was successful...

Just a quick suggestion...

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [October 13, 2020, 1:09pm UTC](https://discuss.elastic.co/t/to-find-all-the-correlation-id-where-response-is-not-send-back-to-the-consumer/251896/3 "2020-10-13T13:09:00Z")

</div>

Sounds like a typical use case for [transforms](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html)

---

<div class="post-metadata">

**Author:** ![rohitarorait82](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohitarorait82/32/82981_2.png) [@rohitarorait82](https://discuss.elastic.co/u/rohitarorait82)\
**Post date:** [October 14, 2020, 9:38am UTC](https://discuss.elastic.co/t/to-find-all-the-correlation-id-where-response-is-not-send-back-to-the-consumer/251896/4 "2020-10-14T09:38:25Z")

</div>

Thanks @Mark_Harwood and @mastersmit : I applied some filters and in the visualization, I can see for few correl\_id I am getting count = 1 and for few I am getting count =2. Is there a way to display only where count = 1.

Something like **min\_doc\_count** , do we have **max\_doc\_count** in v7.7 ELK

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/c/8c94009d8c70f087589664384f04e62f10bf2198.png)

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [October 14, 2020, 10:09am UTC](https://discuss.elastic.co/t/to-find-all-the-correlation-id-where-response-is-not-send-back-to-the-consumer/251896/5 "2020-10-14T10:09:10Z")

</div>

If you use the transforms API to create an entity-centric index with a single doc for correlation ID then you can have properties like `count` and `duration` as physical fields which you can sort and filter on.

---

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [October 14, 2020, 10:12am UTC](https://discuss.elastic.co/t/to-find-all-the-correlation-id-where-response-is-not-send-back-to-the-consumer/251896/6 "2020-10-14T10:12:55Z")

</div>

One quick solution that may help is to visualize using Canevas , that rely a lot on SQL so you can get quickly all id that occure only once

```
POST _sql?format=txt
{
  "query": """
  SELECT clientip, count(*) FROM kibana_sample_data_logs group by clientip having count(*) <= 5 order by 2 desc
  """
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 11, 2020, 10:13am UTC](https://discuss.elastic.co/t/to-find-all-the-correlation-id-where-response-is-not-send-back-to-the-consumer/251896/7 "2020-11-11T10:13:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.