ML job for calendar date other than @timestamp

Elastic Stack Elasticsearch
1

Hello Machine Learning team,

I went to all kinds of hoops :curly_loop: to finally have x-pack and ML on the cloud (elastic cloud does not offer ML feature yet :disappointed_relieved:) on a 3 node EC2 cluster.

I have a simple dataset where @timestamp (used simply to reference dataset ingestion time) is the default time field I picked in the index pattern. However I have another date field (Claim Date in the screenshot) that I would like to use as my time axis for forecasting several metrics I have.

Data%20Types
Data Types.jpg1276×594 49.8 KB

Can someone please tell me that this is possible and how to do it?
Does ML module only work with the @timestamp field :frowning: ?
What if my dataset has several datetime fields I want to create a ML job over and use as the x/time-axis?

Thank you in advance,
George Zoto

2

Hi George,

It's not that ML requires the date field to be @timestamp - rather, most of the config UIs to create the jobs will use whatever field is designated to be the time the events occurred, denoted by the clock icon:

image

If you want to analyze data using a different field than this field, then you have two options.

  1. Use the Advanced Job wizard, and make sure you select the Time-field name in the Datafeed tab:

image
image.png1394×1584 148 KB

  1. Use the API to configure the job, making sure you set the time_field to the name of the field you'd like ML to use as the x/time-axis

See: https://www.elastic.co/guide/en/elasticsearch/reference/6.2/ml-job-resource.html#ml-datadescription