Type conflict

[shwesinhan (ssh)]

Hi there
I got type conflict in kibana when new month started.

here is my scenario.
I use elapsed filter to get elapsed_time in logstash.
But later I added one grok pattern like %{DATA:elapsed_time} using same field name and I forgot the type aware. End of the month, I got type conflict in kibana.
One thing I am wondering I use same structure demo/prod but it is ok in my demo env only happen in prod env.

Please kindly give suggestion how to solve this problem?

[Nathan_Reese]

Float fields have doc_values and will work with aggregations. Text fields do not have doc_values on by default and will not work with aggregations. You will get shard failures for aggregation requests that use elapsed_time field and span the logstash-2019.05 index. These shard failures will not harm your cluster, but they will prevent data from logstash-2019.05 from being displayed in Kibana so you will have missing data in any visualization that uses the elapsed_time field and spans the logstash-2019.05 index.

If you never use elapsed_time in any aggregations then there will never be a problem.

If you use elapsed_time in aggregations then you should re-index your logstash-2019.05 so that elapsed_time is mapped as a float.

[shwesinhan (ssh)]

Thank you @Nathan_Reese
Yes I use elapsed_time in several plots. I agree with you about missing data.
Because when I check the plots, I faced data are gone but I saw them on discover.
Oh something is wrong!! Then I found that elapsed_time type got conflict since end of Apr.

By re-index logstash-2019.05, elapsed_time is mapped automatically as a float in new index? do i need to map manually?

If you use elapsed_time in aggregations then you should re-index your logstash-2019.05 so that elapsed_time is mapped as a float.

Thank you.

[Nathan_Reese]

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html