Need to join three indices on some columns to get the data


(Bharath Pusuluri) #1

Is joining the indices based on column to fetch the details is possible in Elasticsearch?

If it is static data I would have done programming, my requirement is on data retreving from http requests.


(Mark Walkom) #2

Elasticsearch doesn't have columns, so not sure what you mean there.

But it cannot join by itself. You would need to collect the data and then join in your code.


(Bharath Pusuluri) #3

Could you please elaborate more. I have three indexes like below:

index1 - > memoryutilization = 50 & jobid = A1
index2 -> jobid = A1 & jobexecutor = B1
index3 -> jobexecutor = B1 & username = C1

Now I need to represent a report saying

C1 B1 A1 50

In order to achieve this How I need to write a query or code?


(David Pilato) #4

Then index documents like:

{
  "jobid": "A1",
  "jobexecutor": "B1",
  "username": "C1",
  "memoryutilization": 50
}

(Bharath Pusuluri) #5

The three indices are loading from three different sources. All are loading via logstash by running refreshing every 60 sec.

As per your suggestion I need to create a new document by joining from source as it is. Do you have any sample code for logstash to do this. All source are in JSON format. (Details are from timeline db server)


(David Pilato) #6

As per your suggestion I need to create a new document by joining from source as it is.

Yes that's what @warkolm was telling with:

You would need to collect the data and then join in your code.

Do you have any sample code for logstash to do this.

No. You can do db lookups when ingesting events. Look at:


(Bharath Pusuluri) #7

The index are coming real time from three different sources. So any idea I can achieve this via logstash? Or some other way?


(David Pilato) #8

No. May be ask in #logstash channel?


(system) #9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.