Need help with dates using the Data Visualizer

Elastic Stack Kibana
1

Hi, I'm trying to import a CSV file into my Kibana (6.7.2) using the Data Visualizer tool inside the Machine Learning tab, under a index pattern already created.

image

image

image
image.png1432×268 23.9 KB

image
image.png1120×755 51.1 KB

But when it comes to see its content, I have two problems:

  1. I don't see anything in Discover

    image
    image.png1818×474 59.6 KB

  2. I get an error in all my visualizations (by the date format)

image
image.png467×628 27.1 KB

This is my date mapping when importing the CSV in the Data Visualizer

{
      "Fecha de notificación": {
        "type": "date",
		"format": "dd/MM/YYYY HH:mm:ss",
		"ignore_malformed": true
      },
      "Fina Afectación": {
        "type": "date",
		"format": "dd/MM/YYYY HH:mm:ss",
		"ignore_malformed": true
      },
      "Finalización prevista": {
        "type": "date",
		"format": "dd/MM/YYYY HH:mm:ss",
		"ignore_malformed": true
      },
      "Finalización real": {
        "type": "date",
		"format": "dd/MM/YYYY HH:mm:ss",
		"ignore_malformed": true
      },
}

Can somebody help me, please?

2

What does your CSV look like? It would seem that the answer is right in the error message:

failed to parse date field [2018-04-01] with format [dd/MM/YYYY HH:mm:ss]

that date format doesn't look like the date field value.

You can see from the docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/ml-find-file-structure.html that it will automatically properly guess the following date formats:

  • dd/MMM/YYYY:HH:mm:ss Z
  • EEE MMM dd HH:mm zzz YYYY
  • EEE MMM dd HH:mm:ss YYYY
  • EEE MMM dd HH:mm:ss zzz YYYY
  • EEE MMM dd YYYY HH:mm zzz
  • EEE MMM dd YYYY HH:mm:ss zzz
  • EEE, dd MMM YYYY HH:mm Z
  • EEE, dd MMM YYYY HH:mm ZZ
  • EEE, dd MMM YYYY HH:mm:ss Z
  • EEE, dd MMM YYYY HH:mm:ss ZZ
  • ISO8601
  • MMM d HH:mm:ss
  • MMM d HH:mm:ss,SSS
  • MMM d YYYY HH:mm:ss
  • MMM dd HH:mm:ss
  • MMM dd HH:mm:ss,SSS
  • MMM dd YYYY HH:mm:ss
  • MMM dd, YYYY h:mm:ss a
  • TAI64N
  • UNIX
  • UNIX_MS
  • YYYY-MM-dd HH:mm:ss
  • YYYY-MM-dd HH:mm:ss,SSS
  • YYYY-MM-dd HH:mm:ss,SSS Z
  • YYYY-MM-dd HH:mm:ss,SSSZ
  • YYYY-MM-dd HH:mm:ss,SSSZZ
  • YYYY-MM-dd HH:mm:ssZ
  • YYYY-MM-dd HH:mm:ssZZ
  • YYYYMMddHHmmss

If your CSV used one of those formats, you wouldn't have worry about anything - the tool would properly recognize the date pattern in your CSV and everything would work

3

Hi, thanks for the answer, dates on my CSV looks like this:

image
image.png795×580 29 KB

4

The dates typically need to look something like this:

2019-06-05T03:22:05
Year-month-dayT time

5

Again:

You can modify your CSV to conform to one of the above formats and it will be imported without issue.

6

The solution was deleting the @timestamp field in the "Mappings" of the "Data Visualizer" when you import a CSV.

image

7

Ok, but now you've essentially made your data have no time element. Is that what you really wanted?

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.