# Merge two log file using Elasticsearch script

**URL:** <https://discuss.elastic.co/t/merge-two-log-file-using-elasticsearch-script/40621>\
**Category:** Elasticsearch\
**Created:** [February 1, 2016, 9:40am UTC](https://discuss.elastic.co/t/merge-two-log-file-using-elasticsearch-script/40621 "2016-02-01T09:40:31Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![binurajps](https://avatars.discourse-cdn.com/v4/letter/b/eb8c5e/32.png) [@binurajps](https://discuss.elastic.co/u/binurajps)\
**Post date:** [February 1, 2016, 9:40am UTC](https://discuss.elastic.co/t/merge-two-log-file-using-elasticsearch-script/40621/1 "2016-02-01T09:40:31Z")

</div>

Hello Team,

Please find below scenarios.

File 1:  
Session\_ID : S001  
From\_City\_Name : TYO  
To\_City\_Name : LAX

File 2:  
Session\_ID : S001  
Product\_Id : P001

I want to merge these two log file based on the session ID. Is that possible?

Script will be : Select From\_City\_Name, To\_City\_Name, Product\_Id where Session\_ID = Session\_ID;

Result raw will be : S001, P001, TYO, LAX

Regards,  
Binuraj

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 1, 2016, 9:41am UTC](https://discuss.elastic.co/t/merge-two-log-file-using-elasticsearch-script/40621/2 "2016-02-01T09:41:41Z")

</div>

This is something you need to do before indexing into elasticsearch so you'd better ask on the logstash forum.

I'm moving your question there.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 1, 2016, 12:29pm UTC](https://discuss.elastic.co/t/merge-two-log-file-using-elasticsearch-script/40621/3 "2016-02-01T12:29:02Z")

</div>

There's nothing in stock Logstash to merge multiple input files in the way you describe. What might work is reading them independently and updating the ES index twice. You'd have to pick a well-defined document id (perhaps the Session\_ID field?). The first time a given session id is seen it'll create the document and the next time it'll update it with the additional fields. However, I don't think Logstash's elasticsearch output does partial document updates so you'd have to use an elasticsearch filter to fetch the missing fields. Yuck. I think this is something you'll want to do outside of Logstash.

---

<div class="post-metadata">

**Author:** ![binurajps](https://avatars.discourse-cdn.com/v4/letter/b/eb8c5e/32.png) [@binurajps](https://discuss.elastic.co/u/binurajps)\
**Post date:** [February 3, 2016, 5:27am UTC](https://discuss.elastic.co/t/merge-two-log-file-using-elasticsearch-script/40621/4 "2016-02-03T05:27:50Z")

</div>

> [@binurajps](#):
>
> Hello Team,
> 
> Please find below scenarios.
> 
> File 1:Session\_ID : S001From\_City\_Name : TYOTo\_City\_Name : LAX
> 
> File 2:Session\_ID : S001Product\_Id : P001
> 
> I want to merge these two log file based on the session ID. Is that possible?
> 
> Script will be : Select From\_City\_Name, To\_City\_Name, Product\_Id where Session\_ID = Session\_ID;
> 
> Result raw will be : S001, P001, TYO, LAX
> 
> Regards,Binuraj

Hello Team,

Please find below scenarios.

File 1:  
Session\_ID : S001  
From\_City\_Name : TYO  
To\_City\_Name : LAX

File 2:  
Session\_ID : S001  
Product\_Id : P001

I want to merge these two log file based on the session ID. Is that possible?

Script will be : Select From\_City\_Name, To\_City\_Name, Product\_Id where Session\_ID = Session\_ID;

Result raw will be : S001, P001, TYO, LAX

Regards,  
Binuraj

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [February 3, 2016, 7:33am UTC](https://discuss.elastic.co/t/merge-two-log-file-using-elasticsearch-script/40621/5 "2016-02-03T07:33:46Z")

</div>

This seems to be exactly the same question that has already been answered. What is it that is not clear?

---

<div class="post-metadata">

**Author:** ![binurajps](https://avatars.discourse-cdn.com/v4/letter/b/eb8c5e/32.png) [@binurajps](https://discuss.elastic.co/u/binurajps)\
**Post date:** [February 4, 2016, 7:04am UTC](https://discuss.elastic.co/t/merge-two-log-file-using-elasticsearch-script/40621/6 "2016-02-04T07:04:33Z")

</div>

I'm checking elasticsearch support this kind of scripting or not?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 4, 2016, 8:11am UTC](https://discuss.elastic.co/t/merge-two-log-file-using-elasticsearch-script/40621/7 "2016-02-04T08:11:02Z")

</div>

> I'm checking elasticsearch support this kind of scripting or not?

And I responded to your question. If some part of the answer is unclear, please ask a specific question about that. Don't post the same question all over again.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:19pm UTC](https://discuss.elastic.co/t/merge-two-log-file-using-elasticsearch-script/40621/8 "2017-07-05T23:19:01Z")

</div>


