Hi,
I have a 'duration' field being indexed in my ES as of May 5th 2016. Logstash automatically created it as type String. A few days and a few records later I realised this problem and configured Logstash to set this field's type to Float via the mutate filter. I see in the logstash logs that the field is now being parsed as a Float.
In Kibana on the settings page for my logstash-* index, on clicking the 'refresh field list' link, i see the expected warning:
Mapping conflict! A field is defined as several types (string, integer, etc) across the indices that match this pattern. You may still be able to use these conflict fields in parts of Kibana, but they will be unavailable for functions that require Kibana to know their type. Correcting this issue will require reindexing your data.
The field list contains the 'duration' field twice with the following types: a) string b) conflict
After reading this article https://www.elastic.co/guide/en/elasticsearch/guide/current/reindex.html I ran the following in the Sense plugin for each daily index in May where this 'duration' field is indexed, eg for 2016.05.18 :
GET /logstash-2016.05.18/_search?scroll=1m
{
"sort": ["_doc"],
"size": 1000
}
Here's an example result JSON:
{
"_index": "logstash-2016.05.18",
"_type": "web",
"_id": "facd0827b3cd3462c4088d8e1fd12b9554daca35",
"_score": null,
"_source": {
"@version": "1",
"@timestamp": "2016-05-18T05:00:05.000Z",
"path": "/var/log/nginx/web_pixels.log",
"host": "nginx-1.c.test-paper-doll.internal",
"type": "web",
"request": "GET /tr/web?event=Page%20Unloaded&duration=50.32600000000001&publisherDomain=bunte.de HTTP/1.1",
"clientip": "192.114.23.209",
"status": "200",
"req_time": "0.000",
"remoteUser": "-",
"logdate": "18/May/2016:05:00:05 +0000",
"referer": "https://localhost:4443/app/desktop.html",
"agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36",
"event": "Page%20Unloaded",
"duration": 50.32600000000001,
"publisherDomain": "bunte.de",
"ua_name": "Chrome",
"ua_os": "Windows 10",
"ua_os_name": "Windows 10",
"ua_device": "Other",
"ua_major": "50",
"ua_minor": "0",
"ua_patch": "2661"
},
"sort": [
147
]
}
I can see the 'duration' field is now a float.
On returning to the Settings tab in Kibana and clicking the 'refresh field list' link again, i still see the duration field twice: a) String b) conflict.
What have I done wrong?
Many thanks