# Data mining firewall logs with machine learning

**URL:** <https://discuss.elastic.co/t/data-mining-firewall-logs-with-machine-learning/86751>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-machine-learning\
**Created:** [May 23, 2017, 2:30am UTC](https://discuss.elastic.co/t/data-mining-firewall-logs-with-machine-learning/86751 "2017-05-23T02:30:42Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![mariofcs](https://avatars.discourse-cdn.com/v4/letter/m/838e76/32.png) [@mariofcs](https://discuss.elastic.co/u/mariofcs)\
**Post date:** [May 23, 2017, 2:30am UTC](https://discuss.elastic.co/t/data-mining-firewall-logs-with-machine-learning/86751/1 "2017-05-23T02:30:42Z")

</div>

Can anybody tell if the method described in the following url can be done with Elastic - Machine Learning :

> **[Data mining firewall logs : Principal Component Analysis](http://blog.davidvassallo.me/2015/10/28/data-mining-firewall-logs-principal-component-analysis/)**
>
> In this article we’ll explore how Principal Component Analysis \[PCA\] \[1\] – a popular data reduction technique – can help a busy security or network administrator. Any such adminis…

Regards

Mario

---

<div class="post-metadata">

**Author:** ![stevedodson](https://avatars.discourse-cdn.com/v4/letter/s/ac91a4/32.png) [@stevedodson](https://discuss.elastic.co/u/stevedodson)\
**Post date:** [May 24, 2017, 7:40am UTC](https://discuss.elastic.co/t/data-mining-firewall-logs-with-machine-learning/86751/2 "2017-05-24T07:40:27Z")

</div>

We can perform a similar analysis using the 'population' analysis feature that is built into Elastic ML ([https://www.elastic.co/guide/en/x-pack/current/ml-api-definitions.html#ml-detectorconfig](https://www.elastic.co/guide/en/x-pack/current/ml-api-definitions.html#ml-detectorconfig) - over\_field\_name). Population analysis automatically profiles entities across several dimensions and identifies unusual entities. Internally, this analysis uses techniques similar to PCA and will automatically cluster entities into peer groups.

As Elastic ML is native to the Elastic stack this can be simply run on data in Elasticsearch in real-time, and the methods we use are significantly less prone to relying on data spread, orthogonality and skew than naive PCA.

Given the interest in ML on firewall logs, we'll try to publish some suggested configurations and use cases shortly.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 21, 2017, 7:40am UTC](https://discuss.elastic.co/t/data-mining-firewall-logs-with-machine-learning/86751/3 "2017-06-21T07:40:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.