Correlation of two log (sources)

Elastic Stack Logstash
1

Hello
Newbie here, can someone help me out please?

2 log servers producing:
connection logs (user ip etc) (generated every 10 mins)
http logs (user actions) (generated on the fly)

the connection logs have username and ip (sent via filebeat to logstash)
the http logs are missing the username (sent via syslog, then loaded from syslog file to logstash)
(i cannot change this, i dont think)

how is best to fuse these two? can i add something in the logstash, or is there something I can do to merge the logs in a search in kibana?

thanks

2

Hi mr_rob,
can use correlation filter to correlate the field from one index to another index by matching value from both logs.

3

mant thanks...... could you give a hint to where to look for help on this? (sorry dont want to appear too demanding):smiley:

4

By using of the this filter you can achieve tat @mr_rob,
https://www.elastic.co/guide/en/logstash/current/plugins-filters-elasticsearch.html

5

oooh interesting. I will look, i think i might have an issue with the timing sequence of the logs... they at the moment dont arrive together. I could look into that, its a manual script that is run to produce a list of customer's current IP addresses.

6

the elasticsearch plugin, do i put that in another config file? I have one config that inputs and processes logs from the two sources.

i then want to merge these two as above

7

Did you get it done ?