# Correlation in ELK dashboard

**URL:** <https://discuss.elastic.co/t/correlation-in-elk-dashboard/260160>\
**Category:** Kibana\
**Created:** [January 5, 2021, 6:38am UTC](https://discuss.elastic.co/t/correlation-in-elk-dashboard/260160 "2021-01-05T06:38:47Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sivajanani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sivajanani/32/75053_2.png) [@Sivajanani](https://discuss.elastic.co/u/Sivajanani)\
**Post date:** [January 5, 2021, 6:38am UTC](https://discuss.elastic.co/t/correlation-in-elk-dashboard/260160/1 "2021-01-05T06:38:47Z")

</div>

Hello All,  
I have an index with below fields and data set,

```auto
    1.indicator:13.12.14.15
	  action:success
	
	2.indicators:13.12.14.15
	  tags:elk-search
	
	3.indicator:14.4.5.67.84
	  action:success 

```

I have built a dashboard for the above with 4 panels indicator,indicators,action and tags.  
Now if I search for success only 2 panels are filtering data i.e,

```auto
      indicator:13.12.14.15
	  action:success

```

But I wanted data in 4 panels because success is related to 13.12.14.15 and the IP is present in one more meta called indicators which is related to tags.  
To summarize if i filter success in my dashboard I need to get the below correlated data,

```auto
    indicator:13.12.14.15
	action:success
	indicators:13.12.14.15
	tags:elk-search 

```

Is there any ways to achieve this using ELK queries? Kindly suggest your ideas as I am a beginner in writing queries.

---

<div class="post-metadata">

**Author:** ![tsullivan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tsullivan/32/31077_2.png) [@tsullivan](https://discuss.elastic.co/u/tsullivan)\
**Post date:** [January 12, 2021, 10:23pm UTC](https://discuss.elastic.co/t/correlation-in-elk-dashboard/260160/2 "2021-01-12T22:23:04Z")

</div>

Hi Sivajanani, it sounds like the data is fragmented and you need a way to connect it back together so it can be searched effectively. Kibana doesn't have a way to connect the fragments for you in its visualizations.

I think your best bet is to re-index the data using a query that groups the data into single entities. If `indicator` is a numeric type or`keyword` type field, and and it is how you want the data to be connected, you could run this query to put the entities together:

```auto
POST /data-index-*/_search
{
  "size": 5000, // search for 5000 indicators
  "query": {
    "match_all": {}
  },
  "collapse": {
    "field": "indicator", // designate indicator as the field to group other data by
    "inner_hits": {
      "name": "most_recent",
      "size": 100, // find up to 100 other documents with the same indicator
      "sort": [{ "@date": "desc" }] // sort with newest documents first
    }
  },
  "sort": [
    {
      "indicator": {
        "order": "asc"
      }
    }
  ]
}

```

My advice is to create a script that runs that query, and takes the inner\_hit JSON data to create new documents, and save those documents in a new Elasticsearch index.

---

<div class="post-metadata">

**Author:** ![Sivajanani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sivajanani/32/75053_2.png) [@Sivajanani](https://discuss.elastic.co/u/Sivajanani)\
**Post date:** [January 18, 2021, 11:00am UTC](https://discuss.elastic.co/t/correlation-in-elk-dashboard/260160/3 "2021-01-18T11:00:06Z")

</div>

Hello @tsullivan  
Thanks for the solution. But still I will be able to get only the indicators field.  
So I tried another way around using join. But still I am facing issues. my join field is created but am not able to correlate still.  
I think I have missed out some concepts. Please help.

---

<div class="post-metadata">

**Author:** ![tsullivan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tsullivan/32/31077_2.png) [@tsullivan](https://discuss.elastic.co/u/tsullivan)\
**Post date:** [January 21, 2021, 5:37am UTC](https://discuss.elastic.co/t/correlation-in-elk-dashboard/260160/4 "2021-01-21T05:37:21Z")

</div>

> [@Sivajanani](#):
>
> But still I will be able to get only the indicators field

I encourage you to look more into it. With field collapsing, you will get each entire document that correlates to the same indicator value.

See [Expand Collapse Results](https://www.elastic.co/guide/en/elasticsearch/reference/current/collapse-search-results.html#expand-collapse-results)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 18, 2021, 5:37am UTC](https://discuss.elastic.co/t/correlation-in-elk-dashboard/260160/5 "2021-02-18T05:37:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.