# Help with doing "Join" **URL:** \ **Category:** Elasticsearch\ **Created:** [September 6, 2017, 6:46am UTC](https://discuss.elastic.co/t/help-with-doing-join/99524 "2017-09-06T06:46:51Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![tomer](https://avatars.discourse-cdn.com/v4/letter/t/41988e/32.png) [@tomer](https://discuss.elastic.co/u/tomer)\ **Post date:** [September 6, 2017, 6:46am UTC](https://discuss.elastic.co/t/help-with-doing-join/99524/1 "2017-09-06T06:46:51Z")
Hi, I saw the [ES explenation](https://www.elastic.co/guide/en/elasticsearch/reference/current/joining-queries.html) to do "JOIN" in ES, but I am confused and I think I have a complicated case, so will be happy for more help. First I will lay out my problem: I have about 5 logs that represent 5 stages in a process (for SQL this is like having 5 tables). In all the logs I have the same UID. In all the logs I have the parameters but the parameters that irrelevant for stage one is or "0" or an empty String. Here is a general picture: Log a: A: 5 , B: 0 ,C: , D:0 , E:0 , UID: xyz Log b: A: 0 , B: 8 ,C: , D:0 , E:0 , UID: xyz Log c: A: 5 , B: 0 ,C: 9 , D:0 , E:0 , UID: xyz Log d: A: 5 , B: 0 ,C: , D:2 , E:0 , UID: xyz Log e: A: 5 , B: 0 ,C: , D:0 , E:4 , UID: xyz I want as a result: A: 5 , B: 8, C 9, D: 2, E: 4, UID:“xyz” ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 6, 2017, 7:06am UTC](https://discuss.elastic.co/t/help-with-doing-join/99524/2 "2017-09-06T07:06:09Z")
It's not clear what you want to do the "join" on, can you clarify that please? ---
**Author:** ![tomer](https://avatars.discourse-cdn.com/v4/letter/t/41988e/32.png) [@tomer](https://discuss.elastic.co/u/tomer)\ **Post date:** [September 6, 2017, 7:24am UTC](https://discuss.elastic.co/t/help-with-doing-join/99524/3 "2017-09-06T07:24:56Z")
Yes I want to join on UID, that means at the end I want: Joined: A: 5 , B: 8, C 9, D: 2, E: 4, UID:"xyz" Thanks on the reply ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 6, 2017, 7:28am UTC](https://discuss.elastic.co/t/help-with-doing-join/99524/4 "2017-09-06T07:28:11Z")
Ok, well the only way to use the joins that link mentions is to restructure your documents so that it's all based around the UID. We refer to that as entity centric indexing. Otherwise you can use something like the aggregation filter in Logstash, to group things before they are sent to Elasticsearch. There's no way to do that sort of join with your data currently looking like that and in Elasticsearch. ---
**Author:** ![tomer](https://avatars.discourse-cdn.com/v4/letter/t/41988e/32.png) [@tomer](https://discuss.elastic.co/u/tomer)\ **Post date:** [September 6, 2017, 7:59am UTC](https://discuss.elastic.co/t/help-with-doing-join/99524/5 "2017-09-06T07:59:08Z")
The problem with the Logstash solution is that it erases all the other logs, this might be ok from my side but might be an issue. does "entity centric indexing" give the same problem? Or alternatively is there a way to keep the 5 logs that are created and that the aggregation will only add an extra log? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 6, 2017, 8:04am UTC](https://discuss.elastic.co/t/help-with-doing-join/99524/6 "2017-09-06T08:04:35Z")
You can use the Split filter in Logstash to have the originals but also an aggregated one. Entity centric indexing will also mean you don't maintain the original logs. if you want both the original and the aggregated one, then you will need two copies of the data in Elasticsearch. ---
**Author:** ![tomer](https://avatars.discourse-cdn.com/v4/letter/t/41988e/32.png) [@tomer](https://discuss.elastic.co/u/tomer)\ **Post date:** [September 6, 2017, 8:12am UTC](https://discuss.elastic.co/t/help-with-doing-join/99524/7 "2017-09-06T08:12:14Z")
Ok, to make sure I understood: So, basiclly my machines will send to LS the 5 logs and in LS I will have a split filter that will duplicate these logs. and on the duplicated part I will use the aggregation? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 6, 2017, 8:12am UTC](https://discuss.elastic.co/t/help-with-doing-join/99524/8 "2017-09-06T08:12:53Z")
Yep! ---
**Author:** ![tomer](https://avatars.discourse-cdn.com/v4/letter/t/41988e/32.png) [@tomer](https://discuss.elastic.co/u/tomer)\ **Post date:** [September 6, 2017, 9:09am UTC](https://discuss.elastic.co/t/help-with-doing-join/99524/9 "2017-09-06T09:09:54Z")
Does this work also with "clone" filter? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 6, 2017, 9:15am UTC](https://discuss.elastic.co/t/help-with-doing-join/99524/10 "2017-09-06T09:15:17Z")
Yep. ---
**Author:** ![tomer](https://avatars.discourse-cdn.com/v4/letter/t/41988e/32.png) [@tomer](https://discuss.elastic.co/u/tomer)\ **Post date:** [September 6, 2017, 12:22pm UTC](https://discuss.elastic.co/t/help-with-doing-join/99524/11 "2017-09-06T12:22:02Z")
Hi @warkolm, I tried to do as you said as following: ``` filter { json{ source => "message" } clone { clones => ["clone"] } if [type] == "clone" { aggregate { task_id => "%{transactionId}" code => "map['eventTimestamp'] = 0; event.set(‘Aggregetion’, true)" push_map_as_event_on_timeout => true timeout_task_id_field => "transactionId" timeout => 100 } } } ``` but from some reason nothing is shown, no duplicated logs and no aggregation ☹ ,maybe you know why? Could this be b/c it does deal with the idea that in log A field "a" is x and in log B field "a" is y? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 12, 2017, 5:47am UTC](https://discuss.elastic.co/t/help-with-doing-join/99524/15 "2017-09-12T05:47:23Z")
Can you update that other thread with the info? You won't get much help on Logstash configs here is all 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 10, 2017, 5:47am UTC](https://discuss.elastic.co/t/help-with-doing-join/99524/16 "2017-10-10T05:47:44Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.