Not able to create a machine learning job

[maarab]

Hi,

I activated a platinum trial license to do a hands-on machine learning. I have an index and have created an index pattern with the same index name without specifying a time field and no wild card. While creating the job I am able to select the index but it gives the following messages : "Index pattern X is not time based. Anomaly detection can only be run over indices which are time based." I don't see why a time based field is required as the anomalies can be detected while documents are getting indexed. Am I missing something here with respect to understanding? Please excuse my ignorance. I just want the job for e.g. to raise anomaly if it sees for a third value while all the documents would ideally be having 2 values.

[richcollier]

You are indeed missing something! What you're describing is exactly a time-based situation - documents getting indexed over time. So, how would one know if what you see right now is "expected" or "unexpected" if you haven't paid attention to the behavior over time in the past?

But, keep in mind that ML doesn't solve every problem. If you have a simple use case (as in "this document should never have a 3rd value" - as you've described) - then you don't need ML. You can accomplish this with a simple rule/threshold in Alerting/Watcher.

[maarab]

Thank you for your reply. I understand that the behaviour is over time, but is there really a time field required as a field in the index? Can that time not be the time when documents are added to the index i.e. the created date? For e.g. if I have an index with fields (id, transNo, transType, isActive, countryId), do I need to have a createdDate field in the index as well?
Also, thank you for the tip.

[richcollier]

You only need a timestamp field in your index if you plan on using ML. And yes, it can be the date in which the document was indexed - but I assume that somehow relates to when the transaction was created in the first place?

[maarab]

That about sums it up. Thanks.