Machine Learning on Web server logs

I am new to this machine learning features. I am having my web access logs. Could you help me with what kind of anomalies or single metric data or how can I use forecasting mechanism

A simple one would be to look for the rate of requests, for every (split on) HTTP status code

How am I gonna detect the anomalies in such pattern??

Well, that's what the product does for you automatically! Here are sample results:

Can you please help me how would I obtain such results using my logs????I am seriously stucked/stalled on machine learning topic..

Even If could help me with any documentation that would also be very helpful

Docs:

Getting started tutorial: https://www.elastic.co/guide/en/x-pack/current/ml-getting-started.html

API docs: https://www.elastic.co/guide/en/x-pack/current/ml-api-quickref.html

Other “how to” docs: https://www.elastic.co/guide/en/x-pack/current/ml-configuring.html

Function reference: https://www.elastic.co/guide/en/x-pack/current/ml-functions.html

ML Videos:

8 Min Tutorial #1 - How to create a single metric job:

8 Min Tutorial #2 - How to create a multi-metric job:

8 Min Tutorial #3 - Detect outliers in a population:

30 Min Meetup w/ Steve Dodson:

Useful ML Blogs:

Alerting on Machine Learning Jobs in Elasticsearch v5.5 (https://www.elastic.co/blog/alerting-on-machine-learning-jobs-in-elasticsearch-v55)

Machine Learning Anomaly Scoring and Elasticsearch - How it Works (https://www.elastic.co/blog/machine-learning-anomaly-scoring-elasticsearch-how-it-works)

Custom Elasticsearch Aggregations for Machine Learning Jobs (https://www.elastic.co/blog/custom-elasticsearch-aggregations-for-machine-learning-jobs)

Scheduled Events (https://www.elastic.co/blog/scheduled-events-and-the-amorous-anomaly-elasticsearch-machine-learning)

Filtering Input Data to Refine Machine Learning Jobs (https://www.elastic.co/blog/filtering-input-data-to-refine-machine-learning-jobs)

How to Capture Domain Knowledge in Elastic Machine Learning Jobs with Custom Rules (https://www.elastic.co/blog/how-to-capture-domain-knowledge-in-elastic-machine-learning-jobs-with-custom-rules)

Temporal vs. Population Analysis in Elastic Machine Learning

(https://www.elastic.co/blog/temporal-vs-population-analysis-in-elastic-machine-learning)

Changes to Elastic Machine Learning Anomaly Scoring in 6.5 (https://www.elastic.co/blog/changes-to-elastic-machine-learning-anomaly-scoring-in-6-5)

Importing CSV and Log Data into Elasticsearch with File Data Visualizer

(https://www.elastic.co/blog/importing-csv-and-log-data-into-elasticsearch-with-file-data-visualizer)

@richcollier Thanks for these links and they are really helpful. Can you share some links how does elasticsearch machine learning features actually works. I mean what's going on behind the scene how it normalizes or lets say how it creates benchmark and they decide what is anomaly and what kind of warning shall we produce

@richcollier h!!! I didn't wen through all the links I guess the answers are in those links....Will post if I had any other query..

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.