Need help with filebeat and Kibana index pattern creation

I was out on vacation last week and while I was out there was an issue with one of our kibana index patterns getting corrupted (for lack of a better term).

We have a 'responseTime' stamp in our incoming payload that we set for our primary time field and up until May 14th it was working fine (see below)

Then at midnight on May 15th something happened that caused this pattern to get corrupted which I still haven't figured out but my main priority is getting this pattern working again. Below is what is showing when you try to create a new index pattern

I recreated the index template in the Index Management area and it looks fine and when I do a search from within the Dev area the results are as I would expect which is showing customer payload information but trying to create a new index pattern isn't working.

Here is a copy of my mappings for the index (you can see the responseTime field about 1/2 way down --

{
"_doc": {
"_meta": {
"beat": "filebeat",
"version": "7.1.1"
},
"dynamic_templates": ,
"date_detection": false,
"properties": {
"trustedId": {
"ignore_above": 1024,
"type": "keyword"
},
"@timestamp": {
"type": "date"
},
"payload": {
"norms": false,
"type": "text"
},
"responseTime": {
"type": "date"
},
"amazonTraceId": {
"norms": false,
"type": "text"
},
"uuid": {
"ignore_above": 1024,
"type": "keyword"
},
"podId": {
"ignore_above": 1024,
"type": "keyword"
}
}
}
}

I've tried everything I can think of short of blowing everything away and starting over from scratch and am at a loss as to how or where to proceed so any and all help is appreciated.

TIA,
Bill Youngman

It looks like maybe a Filebeat module is loading template information into Elasticsearch. Or something else is writing data into that index and creating the extra fields.

Are there other instances of Filebeat running anywhere?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.