To find all the correlation id where response is not send back to the consumer

Hi All,

In our logs; For a transaction , two legs will get generated "Consumer to backend" and "Backend to Consumer" .However in case of failure , only one leg "Consumer to backend" will get generated.

Unique correlation id will get generated for each transaction consisting of these two legs.
Is there a way to find out all the correlation id for which second leg is not generated.

1 Like

you may use Logstash Aggregation to aggregate by correlationid.

And as many events you receive for that correlationid you may increase the count.

if the count is 2, you could know it was successful...

Just a quick suggestion...

Sounds like a typical use case for transforms

Thanks @Mark_Harwood and @mastersmit : I applied some filters and in the visualization, I can see for few correl_id I am getting count = 1 and for few I am getting count =2. Is there a way to display only where count = 1.

Something like min_doc_count , do we have max_doc_count in v7.7 ELK

If you use the transforms API to create an entity-centric index with a single doc for correlation ID then you can have properties like count and duration as physical fields which you can sort and filter on.

One quick solution that may help is to visualize using Canevas , that rely a lot on SQL so you can get quickly all id that occure only once

POST _sql?format=txt
  "query": """
  SELECT clientip, count(*) FROM kibana_sample_data_logs group by clientip having count(*) <= 5 order by 2 desc

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.