# Correlation of two log (sources)

**URL:** <https://discuss.elastic.co/t/correlation-of-two-log-sources/41752>\
**Category:** Logstash\
**Created:** [February 15, 2016, 9:20am UTC](https://discuss.elastic.co/t/correlation-of-two-log-sources/41752 "2016-02-15T09:20:15Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![mr\_rob](https://avatars.discourse-cdn.com/v4/letter/m/e36b37/32.png) [@mr\_rob](https://discuss.elastic.co/u/mr_rob)\
**Post date:** [February 15, 2016, 9:20am UTC](https://discuss.elastic.co/t/correlation-of-two-log-sources/41752/1 "2016-02-15T09:20:15Z")

</div>

Hello  
Newbie here, can someone help me out please?

2 log servers producing:  
connection logs (user ip etc) (generated every 10 mins)  
http logs (user actions) (generated on the fly)

the connection logs have username and ip (sent via filebeat to logstash)  
the http logs are missing the username (sent via syslog, then loaded from syslog file to logstash)  
(i cannot change this, i dont think)

how is best to fuse these two? can i add something in the logstash, or is there something I can do to merge the logs in a search in kibana?

thanks

---

<div class="post-metadata">

**Author:** ![Ganesh2303](https://avatars.discourse-cdn.com/v4/letter/g/57b2e6/32.png) [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\
**Post date:** [February 15, 2016, 9:49am UTC](https://discuss.elastic.co/t/correlation-of-two-log-sources/41752/2 "2016-02-15T09:49:44Z")

</div>

Hi mr\_rob,  
can use correlation filter to correlate the field from one index to another index by matching value from both logs.

---

<div class="post-metadata">

**Author:** ![mr\_rob](https://avatars.discourse-cdn.com/v4/letter/m/e36b37/32.png) [@mr\_rob](https://discuss.elastic.co/u/mr_rob)\
**Post date:** [February 15, 2016, 10:05am UTC](https://discuss.elastic.co/t/correlation-of-two-log-sources/41752/3 "2016-02-15T10:05:54Z")

</div>

mant thanks...... could you give a hint to where to look for help on this? (sorry dont want to appear too demanding)😃

---

<div class="post-metadata">

**Author:** ![Ganesh2303](https://avatars.discourse-cdn.com/v4/letter/g/57b2e6/32.png) [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\
**Post date:** [February 15, 2016, 10:09am UTC](https://discuss.elastic.co/t/correlation-of-two-log-sources/41752/4 "2016-02-15T10:09:19Z")

</div>

By using of the this filter you can achieve tat @mr_rob,  
[https://www.elastic.co/guide/en/logstash/current/plugins-filters-elasticsearch.html](https://www.elastic.co/guide/en/logstash/current/plugins-filters-elasticsearch.html)

---

<div class="post-metadata">

**Author:** ![mr\_rob](https://avatars.discourse-cdn.com/v4/letter/m/e36b37/32.png) [@mr\_rob](https://discuss.elastic.co/u/mr_rob)\
**Post date:** [February 15, 2016, 11:00am UTC](https://discuss.elastic.co/t/correlation-of-two-log-sources/41752/5 "2016-02-15T11:00:44Z")

</div>

oooh interesting. I will look, i think i might have an issue with the timing sequence of the logs... they at the moment dont arrive together. I could look into that, its a manual script that is run to produce a list of customer's current IP addresses.

---

<div class="post-metadata">

**Author:** ![mr\_rob](https://avatars.discourse-cdn.com/v4/letter/m/e36b37/32.png) [@mr\_rob](https://discuss.elastic.co/u/mr_rob)\
**Post date:** [March 3, 2016, 1:07pm UTC](https://discuss.elastic.co/t/correlation-of-two-log-sources/41752/6 "2016-03-03T13:07:37Z")

</div>

the elasticsearch plugin, do i put that in another config file? I have one config that inputs and processes logs from the two sources.

i then want to merge these two as above

---

<div class="post-metadata">

**Author:** ![Basem\_Mohammed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/basem_mohammed/32/13234_2.png) [@Basem\_Mohammed](https://discuss.elastic.co/u/Basem_Mohammed)\
**Post date:** [November 16, 2016, 3:38pm UTC](https://discuss.elastic.co/t/correlation-of-two-log-sources/41752/7 "2016-11-16T15:38:34Z")

</div>

Did you get it done ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:30am UTC](https://discuss.elastic.co/t/correlation-of-two-log-sources/41752/8 "2017-07-06T04:30:31Z")

</div>


