Elapsed Plugin 'expired' events create new index with incorrect field mapping

[stevedearl (Steve Earl)]

Hi All,

I had an issue a few months ago where a float field in my documents began to be displayed as a numeric (no decimal values). The field was 'elapsedtime' which is measured seconds/millis.

I raised this here in an earlier question ( Kibana suddenly rounding number field to single digit) and thought I'd managed to resolve it with the help of people on here.

However, today I've got the same issue and I'm now pretty sure there's a second problem.

I'm using the Elapsed plugin to calculate the time difference between an 'END' event and its' corresponding 'START' event. This works fine most of the time but in some cases there may be no 'END' event - for example when there was an error in processing so the 'END' event wasn't generated.

In these cases, the Elapsed plugin goes on to create an 'Expired Event' document, and this gets inserted into Elasticsearch with a timestamp equal to the current date/time that logstash is running.

What I've noticed today is that even though I have a composable index template which specifies all new indexes for my log data should define the 'elapsedtime' field mapping as 'float', when an index is created for the purposes of holding an Elapsed 'Expired Event', the 'elapsedtime' field mapping within that index is set to 'long'.

The result is that all the subsequent documents added to that index - including my log events - store the 'elapsedtime' as a 'long' value and I lose my precision on the data.

Apologies if the above is a bit confusing - it's not the easiest thing to try to explain.

Below is my composable index template - this should be applied to all indexes with the "logstash-clef-" index pattern:

{
  "template": {
    "settings": {
      "index": {
        "number_of_shards": "1",
        "number_of_replicas": "0",
        "refresh_interval": "5s"
      }
    },
    "mappings": {
      "properties": {
        "elapsedtime": {
          "type": "float"
        }
      }
    },
    "aliases": {}
  }
}

and below is the index mapping for an index which was generated with the initial document being one of my 'real' log events (I've shortened this as there are lots of fields that are not relevant). You can see in this that 'elapsedtime' is a 'float' type:

{
  "logstash-clef-prd-2021.36" : {
    "mappings" : {
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        },
        "@version" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "activityname" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "elapsedtime" : {
          "type" : "float"
        }
      }
    }
  }
}

and below is the mapping of an index which is generated on the creation of one of the Elapsed plugin 'Event Expired' events. Here the index has been created with the 'elapsedtime' field but it has been set to type 'long':

{
  "logstash-clef-prd-2021.37" : {
    "mappings" : {
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        },
        "@version" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "elapsedtime" : {
          "type" : "long"
        },
        "host" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "tags" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    }
  }

If anyone can help with why this is happening and how to resolve it that would be great.

Ideally I'd like to be able to configure the Elapsed plugin to not generate 'Event Expired ' events at all so these indexes weren't created by them. Alternatively, I'd also like to understand why the index is created with an incorrect field mapping for 'elapsedtime' and whether there's any automated process I can run (in Logstash or elsewhere) to delete the Event Expired documents automatically.

Thanks in advance,
Steve