Kibana ML

machine-learning

(Prem) #1

i am using kibana 6.5, in machine learning i have uploaded a time series data, one column is date and other is currency value, while indexing i have selected the date column as time filed, but showing error as index pattern * is not time based, may i know how to set it? Thanks


(Mark Walkom) #2

https://www.elastic.co/guide/en/kibana/current/tutorial-define-index.html should help you out.


(Prem) #3

Thanks for that, Date field in the image set as string, i can't edit through the link, may i know how to set that,Thanks


(Mark Walkom) #4

You need to use a mapping in Elasticsearch to set it before the data is passed in, you can't change it once it is there.

How are you sending data to Elasticsearch?


(Prem) #5

Hi have uploaded via ML, from there created the inedx name and imported


(Prem) #6

After loading data, i select Time Field in override setting the Date column as in the image


(Mark Walkom) #7

You will need to delete the index that already exists and reprocess the file,


(Prem) #8

Actually, when you load a fresh file, this is the page we are getting, may i know which index you were mentioning please


(Bryan Stuhlsatz) #9

If your "Date" field is still a string, you have not setup a template or a mapping for your index. You can look at: https://www.elastic.co/guide/en/elasticsearch/reference/6.5/indices-templates.html.

You need to have a mapping that says that the Date field is of type date. Below is an example. Dates come in many formats and you need to match what is being ingested.

PUT _template/bryan
{
"index_patterns": ["bry*"],
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0
},
"mappings": {
"doc": {
"_source": {
"enabled": true
},
"properties": {
"time": {"type": "date", "format": "MM-dd-yyyy HH:mm" },
"temperature": {"type": "integer"},
"humidity": {"type": "integer"}
}
}
}
}


(rich collier) #10

The "File Visualizer" is an experimental feature. You can see from the docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/ml-find-file-structure.html that it will properly guess the following date formats:

  • dd/MMM/YYYY:HH:mm:ss Z
  • EEE MMM dd HH:mm zzz YYYY
  • EEE MMM dd HH:mm:ss YYYY
  • EEE MMM dd HH:mm:ss zzz YYYY
  • EEE MMM dd YYYY HH:mm zzz
  • EEE MMM dd YYYY HH:mm:ss zzz
  • EEE, dd MMM YYYY HH:mm Z
  • EEE, dd MMM YYYY HH:mm ZZ
  • EEE, dd MMM YYYY HH:mm:ss Z
  • EEE, dd MMM YYYY HH:mm:ss ZZ
  • ISO8601
  • MMM d HH:mm:ss
  • MMM d HH:mm:ss,SSS
  • MMM d YYYY HH:mm:ss
  • MMM dd HH:mm:ss
  • MMM dd HH:mm:ss,SSS
  • MMM dd YYYY HH:mm:ss
  • MMM dd, YYYY h:mm:ss a
  • TAI64N
  • UNIX
  • UNIX_MS
  • YYYY-MM-dd HH:mm:ss
  • YYYY-MM-dd HH:mm:ss,SSS
  • YYYY-MM-dd HH:mm:ss,SSS Z
  • YYYY-MM-dd HH:mm:ss,SSSZ
  • YYYY-MM-dd HH:mm:ss,SSSZZ
  • YYYY-MM-dd HH:mm:ssZ
  • YYYY-MM-dd HH:mm:ssZZ
  • YYYYMMddHHmmss

Notice that your timestamp format does not fall into one of those.

2/01/2014 could be 1st February or 2nd January, depending on the user's locale.