Hi
Is it possible to ignore data not passing the mapping, like date type for @timestamp. It messes up my index pattern where 99% is all fine.
Most of my index mappings looks like these, which are fine:
{
"mapping": {
"unitylog": {
"properties": {
"@timestamp": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss.SSS"
},
"Component": {
"type": "text"
},
"Details": {
"type": "text"
},
"Level": {
"type": "integer"
},
"OperationId": {
"type": "text"
},
"SystemLogId": {
"type": "integer"
},
"TenantId": {
"type": "text"
},
"host": {
"type": "text"
},
"message": {
"type": "text"
}
}
}
}
}
But a few indexes looks like these, I suspect it being some illegal data:
{
"mapping": {
"unitylog": {
"properties": {
"@timestamp": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Component": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Details": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Level": {
"type": "long"
},
"OperationId": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"SystemLogId": {
"type": "long"
},
"TenantId": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"message": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}