Index Date values in multiple formats

Elastic Stack Elasticsearch
1

I wish to index a date value into several fields each in different format.

for example, index "1/1/2011 10:11:12" into two fields:

  • time: "HH:mm:ss"
  • date: "dd/MM/yyyy HH:mm:ss"

I tried using both multi fields and copy_to features using the bellow schema but with no luck.
I get "Invalid format: "1/1/2011 10:11:12" is malformed at "/1/2011 10:11:12""

copy_to
{
"mappings": {
"myIndex": {
"properties": {
"time": {
"type": "date",
"format": "HH:mm:ss"
},
"date": {
"type": "date",
"format": "dd/MM/yyyy HH:mm:ss",
"copy_to": "time"
}
}
}
}
}

multi field

{
"mappings": {
"myIndex": {
"properties": {
"date": {
"type": "date",
"format": "dd/MM/yyyy HH:mm:ss",
"fields": {
"time": {
"type": "date",
"format": "HH:mm:ss"
}
}
}
}
}
}
}

Is there a way to do it - without using scripts - because i cannot afford having an impact on performance?

2

Hi,

it looks like you essentially want to split the input field on whitespace first, then parse each part with a different format. This is usually done e.g. in Logstash before indexing the documents, but I think starting with version 5 of Elasticsearchm this can now also be achieved by using an Ingest Node and a Grok Processor. Other than that I don't see how you can apply formats that only partially match the input field.

3

Thank you for the quick response.

I am interacting with Elasticsearch using the Java SDK, not logstash.
I can do it in code, but hoped Elasticsearch will do it for me.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.