Can't use specific date field as primary time field

[Travis]

Hello !

I have the field "Generated Time" in my message. I want to use it as primary date field in Kibana.

To do so, I create my index pattern and choose "Generated Time" as primary date field. But once in Discover, nothing is displayed. Like if Kibana don't understand this field.

GET myindex/_search :

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "palo",
        "_type" : "_doc",
        "_id" : "lZdviXkBTGy11OBhDMD8",
        "_score" : 1.0,
        "_source" : {
          "@timestamp" : "2021-05-20T11:00:30.917Z",
          "Receive Time" : "2021/05/20 14:06:46",
          "Packets Received" : "1",
          "Session ID" : "1299262",
          "Destination Port" : "53",
          "Destination VM UUID" : null,
          "Generated Time" : "2021/05/20 14:06:46",
          "NAT Destination IP" : "0.0.0.0",
          "Monitor Tag/IMEI" : null,
          "SCTP Chunks Sent" : "0",
          "syslog_timestamp" : "May 20 14:06:47",
          "Device Group Hierarchy Level 1" : "1829",
          "NAT Destination Port" : "0"
        }
      }
    ]
  }
}

mapping :

"Generated Time" : {
  "type" : "date",
  "format" : "yyyy/MM/dd HH:mm:ss||yyyy/MM/dd||epoch_millis"
},

If I choose @timestamp as primary date field, it works. I can see the log.

Thanks for your help !

[flash1293 (Joe Reuter)]

In Discover there's an "Inspect" button in the top right - could you copy/paste the request Discover is doing here? I suspect the space in the field name to be problematic.

[Travis]

Strange...

I changed

"Generated Time" : {
  "type" : "date",
  "format" : "yyyy/MM/dd HH:mm:ss||yyyy/MM/dd||epoch_millis"
},

with

"Generated Time" : {
  "type" : "date",
  "format" : "yyyy/MM/dd HH:mm:ss"
},

and now it's working.

Two pipe in date format = "or" right ?

[Travis]

Hmm...it looks like it was not representativ as it doesn't working anymore (I juste tested with one log yesterday)

Here is the request Discover @flash1293

{
  "version": true,
  "size": 500,
  "sort": [
    {
      "Generated Time": {
        "order": "desc",
        "unmapped_type": "boolean"
      }
    }
  ],
  "aggs": {
    "2": {
      "date_histogram": {
        "field": "Generated Time",
        "calendar_interval": "1d",
        "time_zone": "UTC",
        "min_doc_count": 1
      }
    }
  },
  "stored_fields": [
    "*"
  ],
  "script_fields": {},
  "docvalue_fields": [
    {
      "field": "@timestamp",
      "format": "date_time"
    },
    {
      "field": "Generated Time",
      "format": "date_time"
    },
    {
      "field": "Receive Time",
      "format": "date_time"
    },
    {
      "field": "Start Time",
      "format": "date_time"
    }
  ],
  "_source": {
    "excludes": []
  },
  "query": {
    "bool": {
      "must": [],
      "filter": [
        {
          "match_all": {}
        },
        {
          "range": {
            "Generated Time": {
              "gte": "2021-02-20T09:20:05.739Z",
              "lte": "2021-05-21T09:20:05.739Z",
              "format": "strict_date_optional_time"
            }
          }
        }
      ],
      "should": [],
      "must_not": []
    }
  },
  "highlight": {
    "pre_tags": [
      "@kibana-highlighted-field@"
    ],
    "post_tags": [
      "@/kibana-highlighted-field@"
    ],
    "fields": {
      "*": {}
    },
    "fragment_size": 2147483647
  }
}