# Need to join three indices on some columns to get the data

**URL:** <https://discuss.elastic.co/t/need-to-join-three-indices-on-some-columns-to-get-the-data/152613>\
**Category:** Elasticsearch\
**Created:** [October 16, 2018, 9:18am UTC](https://discuss.elastic.co/t/need-to-join-three-indices-on-some-columns-to-get-the-data/152613 "2018-10-16T09:18:56Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bharath\_Pusuluri](https://avatars.discourse-cdn.com/v4/letter/b/b9bd4f/32.png) [@Bharath\_Pusuluri](https://discuss.elastic.co/u/Bharath_Pusuluri)\
**Post date:** [October 16, 2018, 9:18am UTC](https://discuss.elastic.co/t/need-to-join-three-indices-on-some-columns-to-get-the-data/152613/1 "2018-10-16T09:18:56Z")

</div>

Is joining the indices based on column to fetch the details is possible in Elasticsearch?

If it is static data I would have done programming, my requirement is on data retreving from http requests.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 16, 2018, 9:51am UTC](https://discuss.elastic.co/t/need-to-join-three-indices-on-some-columns-to-get-the-data/152613/2 "2018-10-16T09:51:56Z")

</div>

Elasticsearch doesn't have columns, so not sure what you mean there.

But it cannot join by itself. You would need to collect the data and then join in your code.

---

<div class="post-metadata">

**Author:** ![Bharath\_Pusuluri](https://avatars.discourse-cdn.com/v4/letter/b/b9bd4f/32.png) [@Bharath\_Pusuluri](https://discuss.elastic.co/u/Bharath_Pusuluri)\
**Post date:** [October 17, 2018, 6:31am UTC](https://discuss.elastic.co/t/need-to-join-three-indices-on-some-columns-to-get-the-data/152613/3 "2018-10-17T06:31:20Z")

</div>

Could you please elaborate more. I have three indexes like below:

index1 - \> memoryutilization = 50 & jobid = A1  
index2 -\> jobid = A1 & jobexecutor = B1  
index3 -\> jobexecutor = B1 & username = C1

Now I need to represent a report saying

C1 B1 A1 50

In order to achieve this How I need to write a query or code?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 17, 2018, 6:43am UTC](https://discuss.elastic.co/t/need-to-join-three-indices-on-some-columns-to-get-the-data/152613/4 "2018-10-17T06:43:42Z")

</div>

Then index documents like:

```auto
{
  "jobid": "A1",
  "jobexecutor": "B1",
  "username": "C1",
  "memoryutilization": 50
}

```

---

<div class="post-metadata">

**Author:** ![Bharath\_Pusuluri](https://avatars.discourse-cdn.com/v4/letter/b/b9bd4f/32.png) [@Bharath\_Pusuluri](https://discuss.elastic.co/u/Bharath_Pusuluri)\
**Post date:** [October 17, 2018, 7:16am UTC](https://discuss.elastic.co/t/need-to-join-three-indices-on-some-columns-to-get-the-data/152613/5 "2018-10-17T07:16:37Z")

</div>

The three indices are loading from three different sources. All are loading via logstash by running refreshing every 60 sec.

As per your suggestion I need to create a new document by joining from source as it is. Do you have any sample code for logstash to do this. All source are in JSON format. (Details are from timeline db server)

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 17, 2018, 7:53am UTC](https://discuss.elastic.co/t/need-to-join-three-indices-on-some-columns-to-get-the-data/152613/6 "2018-10-17T07:53:21Z")

</div>

> As per your suggestion I need to create a new document by joining from source as it is.

Yes that's what @warkolm was telling with:

> You would need to collect the data and then join in your code.

> Do you have any sample code for logstash to do this.

No. You can do db lookups when ingesting events. Look at:

- [Jdbc\_static filter plugin | Logstash Reference [8.11] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-filters-jdbc_static.html)
- [Jdbc\_streaming filter plugin | Logstash Reference [8.11] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-filters-jdbc_streaming.html)

---

<div class="post-metadata">

**Author:** ![Bharath\_Pusuluri](https://avatars.discourse-cdn.com/v4/letter/b/b9bd4f/32.png) [@Bharath\_Pusuluri](https://discuss.elastic.co/u/Bharath_Pusuluri)\
**Post date:** [October 17, 2018, 6:59am UTC](https://discuss.elastic.co/t/need-to-join-three-indices-on-some-columns-to-get-the-data/152613/7 "2018-10-17T06:59:38Z")

</div>

The index are coming real time from three different sources. So any idea I can achieve this via logstash? Or some other way?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 18, 2018, 5:55am UTC](https://discuss.elastic.co/t/need-to-join-three-indices-on-some-columns-to-get-the-data/152613/8 "2018-10-18T05:55:59Z")

</div>

No. May be ask in #logstash channel?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 15, 2018, 5:56am UTC](https://discuss.elastic.co/t/need-to-join-three-indices-on-some-columns-to-get-the-data/152613/9 "2018-11-15T05:56:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.