# Combine multiple sources **URL:** \ **Category:** Logstash\ **Created:** [August 4, 2015, 12:07pm UTC](https://discuss.elastic.co/t/combine-multiple-sources/26795 "2015-08-04T12:07:29Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Hornov](https://avatars.discourse-cdn.com/v4/letter/h/c67d28/32.png) [@Hornov](https://discuss.elastic.co/u/Hornov)\ **Post date:** [August 4, 2015, 12:07pm UTC](https://discuss.elastic.co/t/combine-multiple-sources/26795/1 "2015-08-04T12:07:29Z")
I've logs from multiple sources for the same user. But I need to report fields of this differents sources. Is it possible to store all this data in the same document ? Thanks ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 4, 2015, 12:17pm UTC](https://discuss.elastic.co/t/combine-multiple-sources/26795/2 "2015-08-04T12:17:43Z")
Please see this topic: > [@Possible to use logstash to merge data from multiple files?](https://discuss.elastic.co/t/possible-to-use-logstash-to-merge-data-from-multiple-files/26653): > > I've got two files, an xml file and a csv file. When I ingest the data from the csv file I want to add\_fields from the xml file. Here's what my xml looks like \ \07.31.2015\ \ \John\ \12345\ \ \ And here is my csv column1,column2,column3 example1,example2,example3 example4,example5,example6 Now I want to pull Date and ID from the xml file and add those fields to each row of the csv when I pull it into elastic search, so… ---
**Author:** ![Hornov](https://avatars.discourse-cdn.com/v4/letter/h/c67d28/32.png) [@Hornov](https://discuss.elastic.co/u/Hornov)\ **Post date:** [August 4, 2015, 2:23pm UTC](https://discuss.elastic.co/t/combine-multiple-sources/26795/3 "2015-08-04T14:23:47Z")
Thanks but I don't understand how to do the merging on the Elasticsearch side. Somebody could help me ? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 4, 2015, 2:40pm UTC](https://discuss.elastic.co/t/combine-multiple-sources/26795/4 "2015-08-04T14:40:38Z")
It'll be easier to help if you can ask a more specific question. What kind of sources do you have? What fields will each source contribute with? ---
**Author:** ![Hornov](https://avatars.discourse-cdn.com/v4/letter/h/c67d28/32.png) [@Hornov](https://discuss.elastic.co/u/Hornov)\ **Post date:** [August 5, 2015, 8:48am UTC](https://discuss.elastic.co/t/combine-multiple-sources/26795/5 "2015-08-05T08:48:27Z")
I've a xml file like this : \ ``` < devices> < device id="413"> < uuid>2ab941ea-fa62-477b-bfc7-e5a101ac2df7 < principal>152100 ... < /device> < /devices> ``` a csv file like this : PRINCIPAL;FUNCTIONAL\_EXPORT\_DATE;LENGTH(FILE\_CONTENT) 152100;01/07/2015;88576 and an other like this : PRINCIPAL;CART\_ID;UPDATE\_DATE;TOTAL\_AMOUNT 152100;10776692;03/08/2015;2 and I would like to join all of this on the field PRINCIPAL ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 5, 2015, 9:43am UTC](https://discuss.elastic.co/t/combine-multiple-sources/26795/6 "2015-08-05T09:43:28Z")
Argh, crap. Sorry. Elasticsearch won't actually merge the documents for you. In this case I don't have a simple solution for you. ---
**Author:** ![Hornov](https://avatars.discourse-cdn.com/v4/letter/h/c67d28/32.png) [@Hornov](https://discuss.elastic.co/u/Hornov)\ **Post date:** [September 3, 2015, 3:42pm UTC](https://discuss.elastic.co/t/combine-multiple-sources/26795/7 "2015-09-03T15:42:56Z")
If someone meets the same difficulty I managed to use in logstash the filter plugin elasticsearch. ---
**Author:** ![sush](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sush/32/6186_2.png) [@sush](https://discuss.elastic.co/u/sush)\ **Post date:** [December 2, 2015, 2:57pm UTC](https://discuss.elastic.co/t/combine-multiple-sources/26795/8 "2015-12-02T14:57:31Z")
Hi Hornov, I need to do the same with two csv files, Can you give me a sample of your logstash config file, as how exactly did you use the elasticsearch plugin. I have been playing around the options in the plugin, couldn't find it Thanks ---
**Author:** ![Hornov](https://avatars.discourse-cdn.com/v4/letter/h/c67d28/32.png) [@Hornov](https://discuss.elastic.co/u/Hornov)\ **Post date:** [December 2, 2015, 3:29pm UTC](https://discuss.elastic.co/t/combine-multiple-sources/26795/9 "2015-12-02T15:29:49Z")
Not being a native English speaker, sorry in advance for English With the elasticsearch plugin I can search the first document, my rule to find the original document is a litle bit more complicated than keysourcea=keysourceb but the idea is : I store my file A then my file B. The config file of the file b is something like : elasticsearch { hosts =\> ["myhost/myindex"] query =\> "myquerrystring AND keysourcea:%{keysourceb}" fields =\> ["MY\_FIELD1","MY\_FIELD1","MY\_FIELD2","MY\_FIELD2",...] sort =\> "TIMESTAMP\_EVT:desc" } but the best way if you can define a key is to put in your output elasticsearch : action =\> "update" doc\_as\_upsert =\> true document\_id =\> "%{yourkey}" The doc\_as\_upsert is only useful if both files can come in the same time. ---
**Author:** ![sush](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sush/32/6186_2.png) [@sush](https://discuss.elastic.co/u/sush)\ **Post date:** [December 4, 2015, 9:35am UTC](https://discuss.elastic.co/t/combine-multiple-sources/26795/10 "2015-12-04T09:35:29Z")
Thank you so much for the reply. But it didn't work for my case, I needed many to many relation to work. Any how your reply did help in clarifying some doubts. Thank again 🙂 ---
**Author:** ![Hornov](https://avatars.discourse-cdn.com/v4/letter/h/c67d28/32.png) [@Hornov](https://discuss.elastic.co/u/Hornov)\ **Post date:** [December 4, 2015, 11:38am UTC](https://discuss.elastic.co/t/combine-multiple-sources/26795/11 "2015-12-04T11:38:28Z")
If you've many to many relations the elasticsearch filter can't help you because, you can only find 1 other event. I think the best way is to make the relation before if you can (repeat the second file (or a part) for every (or some) lines of the first). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:19am UTC](https://discuss.elastic.co/t/combine-multiple-sources/26795/12 "2017-07-06T05:19:58Z")