# Correlating data from 2 sources

**URL:** <https://discuss.elastic.co/t/correlating-data-from-2-sources/83665>\
**Category:** Kibana\
**Created:** [April 26, 2017, 8:51am UTC](https://discuss.elastic.co/t/correlating-data-from-2-sources/83665 "2017-04-26T08:51:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![10past10](https://avatars.discourse-cdn.com/v4/letter/1/58956e/32.png) [@10past10](https://discuss.elastic.co/u/10past10)\
**Post date:** [April 26, 2017, 8:51am UTC](https://discuss.elastic.co/t/correlating-data-from-2-sources/83665/1 "2017-04-26T08:51:55Z")

</div>

Starting with the following;

- 1 large data-set of network traffic to be investigated (JSON)
- 1 tls-fingerprinting data-set or 1 'bad/blocked' IP data-set or ....

Use case;  
When both the network traffic and fingerprint/ip data-sets are in Elasticsearch, how is it possible to correlate the values from both sources? Like some sort of translation, where the network-data gets matched with a fingerprinting data-set.

I would like to add a row to each event in the large data-set of network traffic, saying like 'Bad IP: Yes/No', so that I can create timelines or other visualizations only on data containing 'bad ip's'. Or matching specific values like a TLS fingerprint, adding an application name row to a network event based on a fingerprinting database.

For example using fingerprints found here: [https://github.com/LeeBrotherston/tls-fingerprinting](https://github.com/LeeBrotherston/tls-fingerprinting)

Does anyone have any ideas about how to accomplish this with the Elastic stack?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 26, 2017, 9:08am UTC](https://discuss.elastic.co/t/correlating-data-from-2-sources/83665/2 "2017-04-26T09:08:51Z")

</div>

That's a join, which ES/KB cannot do for you at this time.

The best option would be to use something like a translate filter in LS during ingestion and then do the lookup there.

---

<div class="post-metadata">

**Author:** ![10past10](https://avatars.discourse-cdn.com/v4/letter/1/58956e/32.png) [@10past10](https://discuss.elastic.co/u/10past10)\
**Post date:** [April 26, 2017, 5:16pm UTC](https://discuss.elastic.co/t/correlating-data-from-2-sources/83665/3 "2017-04-26T17:16:00Z")

</div>

Thanks for your fast reply! I wanted to know for sure that this option isn't in the Elastic-stick, before I start to look at other solutions.

I'm using bro to process the pcap's and create the json files. I think I either need to write some sort of python script or see if I can modify bro to create some extra output I want (something like what's described here: [https://www.securityartwork.es/2017/02/02/tls-client-fingerprinting-with-bro/](https://www.securityartwork.es/2017/02/02/tls-client-fingerprinting-with-bro/)). I think the best option would be to add specific fields to the JSON events created with bro. Hmm interesting 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 24, 2017, 5:19pm UTC](https://discuss.elastic.co/t/correlating-data-from-2-sources/83665/4 "2017-05-24T17:19:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.