Need help with dates using the Data Visualizer

[Oscar73 (Óscar Gómez Fernández)]

Hi, I'm trying to import a CSV file into my Kibana (6.7.2) using the Data Visualizer tool inside the Machine Learning tab, under a index pattern already created.

But when it comes to see its content, I have two problems:

1. I don't see anything in Discover
   

   

2. I get an error in all my visualizations (by the date format)

This is my date mapping when importing the CSV in the Data Visualizer

{
      "Fecha de notificación": {
        "type": "date",
		"format": "dd/MM/YYYY HH:mm:ss",
		"ignore_malformed": true
      },
      "Fina Afectación": {
        "type": "date",
		"format": "dd/MM/YYYY HH:mm:ss",
		"ignore_malformed": true
      },
      "Finalización prevista": {
        "type": "date",
		"format": "dd/MM/YYYY HH:mm:ss",
		"ignore_malformed": true
      },
      "Finalización real": {
        "type": "date",
		"format": "dd/MM/YYYY HH:mm:ss",
		"ignore_malformed": true
      },
}

Can somebody help me, please?

[richcollier]

What does your CSV look like? It would seem that the answer is right in the error message:

failed to parse date field [2018-04-01] with format [dd/MM/YYYY HH:mm:ss]

that date format doesn't look like the date field value.

You can see from the docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/ml-find-file-structure.html that it will automatically properly guess the following date formats:

• dd/MMM/YYYY:HH:mm:ss Z
• EEE MMM dd HH:mm zzz YYYY
• EEE MMM dd HH:mm:ss YYYY
• EEE MMM dd HH:mm:ss zzz YYYY
• EEE MMM dd YYYY HH:mm zzz
• EEE MMM dd YYYY HH:mm:ss zzz
• EEE, dd MMM YYYY HH:mm Z
• EEE, dd MMM YYYY HH:mm ZZ
• EEE, dd MMM YYYY HH:mm:ss Z
• EEE, dd MMM YYYY HH:mm:ss ZZ
• ISO8601
• MMM d HH:mm:ss
• MMM d HH:mm:ss,SSS
• MMM d YYYY HH:mm:ss
• MMM dd HH:mm:ss
• MMM dd HH:mm:ss,SSS
• MMM dd YYYY HH:mm:ss
• MMM dd, YYYY h:mm:ss a
• TAI64N
• UNIX
• UNIX_MS
• YYYY-MM-dd HH:mm:ss
• YYYY-MM-dd HH:mm:ss,SSS
• YYYY-MM-dd HH:mm:ss,SSS Z
• YYYY-MM-dd HH:mm:ss,SSSZ
• YYYY-MM-dd HH:mm:ss,SSSZZ
• YYYY-MM-dd HH:mm:ssZ
• YYYY-MM-dd HH:mm:ssZZ
• YYYYMMddHHmmss

If your CSV used one of those formats, you wouldn't have worry about anything - the tool would properly recognize the date pattern in your CSV and everything would work

[Oscar73 (Óscar Gómez Fernández)]

Hi, thanks for the answer, dates on my CSV looks like this:

[Josh1 (Josh)]

The dates typically need to look something like this:

2019-06-05T03:22:05
Year-month-dayT time

[richcollier]

Again:

@richcollier

You can see from the docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/ml-find-file-structure.html that it will automatically properly guess the following date formats:

• dd/MMM/YYYY:HH:mm:ss Z
• EEE MMM dd HH:mm zzz YYYY
• EEE MMM dd HH:mm:ss YYYY
• EEE MMM dd HH:mm:ss zzz YYYY
• EEE MMM dd YYYY HH:mm zzz
• EEE MMM dd YYYY HH:mm:ss zzz
• EEE, dd MMM YYYY HH:mm Z
• EEE, dd MMM YYYY HH:mm ZZ
• EEE, dd MMM YYYY HH:mm:ss Z
• EEE, dd MMM YYYY HH:mm:ss ZZ
• ISO8601
• MMM d HH:mm:ss
• MMM d HH:mm:ss,SSS
• MMM d YYYY HH:mm:ss
• MMM dd HH:mm:ss
• MMM dd HH:mm:ss,SSS
• MMM dd YYYY HH:mm:ss
• MMM dd, YYYY h:mm:ss a
• TAI64N
• UNIX
• UNIX_MS
• YYYY-MM-dd HH:mm:ss
• YYYY-MM-dd HH:mm:ss,SSS
• YYYY-MM-dd HH:mm:ss,SSS Z
• YYYY-MM-dd HH:mm:ss,SSSZ
• YYYY-MM-dd HH:mm:ss,SSSZZ
• YYYY-MM-dd HH:mm:ssZ
• YYYY-MM-dd HH:mm:ssZZ
• YYYYMMddHHmmss

You can modify your CSV to conform to one of the above formats and it will be imported without issue.

[Oscar73 (Óscar Gómez Fernández)]

The solution was deleting the @timestamp field in the "Mappings" of the "Data Visualizer" when you import a CSV.

[richcollier]

Ok, but now you've essentially made your data have no time element. Is that what you really wanted?