Correlate Events using Kibana

[Mysterious_Ze]

Hello,

I have 2 log_type in Kibana : alerts and traffic bearing logs of my Web Application firewall. I would like to know the correct syntax for correlating data from both log_types.

Ex : select alert.name, alert.description, traffic.x-forwarded-for where (traffic.traffic_id = alert.traffic_id)

Thanks,

[LeeDr (Lee Drengenberg)]

Your alerts and traffic logs are separate docs in Elasticsearch and there isn't really a join like in databases. But there are some things you could do.

Are these 2 types in the same index or in 2 separate indices? If they're in 2 separate indices you would probably want to create an index pattern that matches both of those and no others. For example, if one index was logs_alerts-2018.01.26 (daily indices) and the other was logs_traffic-2018.01.26, then in in Kibana > Management > Index Patterns you could create an index pattern like logs-* that would match both.
If they're already in the same indices then you don't have to worry about that ^.

Then in Discover tab, you should be able to put in the query bar something like;
traffic.traffic_id:1234 AND alert.traffic_id:1234

or if your field name is actually traffic_id then you could just do traffic_id:1234

and that should show you the logs of both types that match that id. They're still separate docs.

It would be easy to aggregate the data across doc types if the field name was exactly the same traffic_id. If it's not, and your field names really are traffic.traffic_id and alert.traffic_id then it might make sense to create a scripted field that gets the value from either of those so you have a single field you can aggregate on.

Let me know and I can guide you further.

Lee