Unable to create a transform that uses "min"

Elastic Stack Kibana
1

I'm trying to create a transform that aggregates on min of a date field. This field is mapped in the index like so:

"event_time": {
          "type": "date",
          "format": "yyyy-MM-dd HH:mm:ss.nnnnnn"
        },

Its value is not 8601 compliant. When I try and use this in a transform everything looks ok and I can see a preview of my data. But when I run it it fails with

Failed to index documents into destination index due to permanent error: [BulkIndexingException[Bulk index experienced [500] failures and at least 1 irrecoverable [TransformException[Destination index mappings are incompatible with the transform configuration.]; nested: MapperParsingException[failed to parse field [event_time.min] of type [date] in document with id 'MNmjKZLf5Er5fYejFHLyAkoAAAAAAAAA'. Preview of field's value: '2020-04-01 16:01:53.000000']; nested: IllegalArgumentException[failed to parse date field [2020-04-01 16:01:53.000000] with format [strict_date_optional_time||epoch_millis]]; nested: NotSerializableExceptionWrapper[date_time_parse_exception: Failed to parse with all enclosed parsers];; MapperParsingException[failed to parse field [event_time.min] of type [date] in document with id 'MNmjKZLf5Er5fYejFHLyAkoAAAAAAAAA'. Preview of field's value: '2020-04-01 16:01:53.000000']; nested: IllegalArgumentException[failed to parse date field [2020-04-01 16:01:53.000000] with format [strict_date_optional_time||epoch_millis]]; nested: NotSerializableExceptionWrapper[date_time_parse_exception: Failed to parse with all enclosed parsers];; java.lang.IllegalArgumentException: failed to parse date field [2020-04-01 16:01:53.000000] with format [strict_date_optional_time||epoch_millis]]. Other failures: ]; nested: TransformException[Destination index mappings are incompatible with the transform

...and on it goes.

The formatting hint doesn't seem to carry over from the source column.

2

The fix appears to be to specify a mapping in the index for the aggregate. So no event_time but event_time.min.

"properties": {
            "event_time.min": {
                "type": "date",
                "format": "yyyy-MM-dd HH:mm:ss.nnnnnn"
            }
        }
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.