# Type of field changes - Mapping conflict

**URL:** <https://discuss.elastic.co/t/type-of-field-changes-mapping-conflict/79545>\
**Category:** Elasticsearch\
**Created:** [March 22, 2017, 7:42am UTC](https://discuss.elastic.co/t/type-of-field-changes-mapping-conflict/79545 "2017-03-22T07:42:02Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![andre22](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@andre22](https://discuss.elastic.co/u/andre22)\
**Post date:** [March 22, 2017, 7:42am UTC](https://discuss.elastic.co/t/type-of-field-changes-mapping-conflict/79545/1 "2017-03-22T07:42:02Z")

</div>

Hi,

I am feeding log data through Logstash through Elasticsearch. Without making any change to the data itself or Logstash Config, I am getting a mapping conflict on a new index I have created yesterday (because of the same problem).

 ![](https://us1.discourse-cdn.com/elastic/original/3X/f/2/f2a7cf20b65ab8437344a5cc642fb8e340dfb387.png)

Logstash Filter (the mapping change happens on the field "KNX-Wert")

```
filter {
  if "knxmonitor" in [tags] {
    csv {
      columns => ["eventtime", "ms", "Typ", "PA", "GA_KO", "KO-ID", "KNX-Name", "KNX-Wert"]
      convert => { "KNX-Wert" => "float" }
    }
    date {
      locale => "en"
      match => ['eventtime' , 'yyyy-MM-dd HH:mm:ss']
    }
    mutate {
     remove_field => ["message", "offset", "eventtime", "ms"]
    }
  }
}

```

Not all messages contain valid float data in KNX-Message, but I am not interested in these messages and wouldn't mind them being discarded at worst. Anyway, they are giving me errors in the elasticsearch.log:

```
[2017-03-22T00:00:04,329][DEBUG][o.e.a.b.TransportShardBulkAction] [LAw6q-9] [knx2-2017.03.21][1] failed to execute bulk item (index) index {[knx2-2017.03.21][log][AVrzGUZoG7YEgFcI0c6J], source[{"KNX-Wert":"05:45:22","KNX-Name...
rg.elasticsearch.index.mapper.MapperParsingException: failed to parse [KNX-Wert]
	at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:298) ~[elasticsearch-5.2.2.jar:5.2.2]
...
Caused by: java.lang.NumberFormatException: For input string: "05:45:22"
	at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043) ~[?:?]
	at sun.misc.FloatingDecimal.parseFloat(FloatingDecimal.java:122) ~[?:?]
	at java.lang.Float.parseFloat(Float.java:451) ~[?:1.8.0_121]
	at org.elasticsearch.common.xcontent.support.AbstractXContentParser.floatValue(AbstractXContentParser.java:174) ~[elasticsearch-5.2.2.jar:5.2.2]
	at org.elasticsearch.index.mapper.NumberFieldMapper$NumberType$2.parse(NumberFieldMapper.java:278) ~[elasticsearch-5.2.2.jar:5.2.2]
	at org.elasticsearch.index.mapper.NumberFieldMapper$NumberType$2.parse(NumberFieldMapper.java:264) ~[elasticsearch-5.2.2.jar:5.2.2]
	at org.elasticsearch.index.mapper.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:1018) ~[elasticsearch-5.2.2.jar:5.2.2]
	at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:287) ~[elasticsearch-5.2.2.jar:5.2.2]
...

```

Is this the reason? If so, how can I tell elasticsearch to drop these messages instead of forcing them into the fields?

Or, how can I avoid field mappings being changed automatically like this?

thank you in advance

---

<div class="post-metadata">

**Author:** ![wenpos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wenpos/32/15326_2.png) [@wenpos](https://discuss.elastic.co/u/wenpos)\
**Post date:** [March 23, 2017, 1:05am UTC](https://discuss.elastic.co/t/type-of-field-changes-mapping-conflict/79545/2 "2017-03-23T01:05:12Z")

</div>

It looks like date time string mismatching. In the output process, you cloud try to filter the "tags" with "\_dateparsefailure". Hope this help.

---

<div class="post-metadata">

**Author:** ![anhlqn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anhlqn/32/5454_2.png) [@anhlqn](https://discuss.elastic.co/u/anhlqn)\
**Post date:** [March 23, 2017, 3:19am UTC](https://discuss.elastic.co/t/type-of-field-changes-mapping-conflict/79545/3 "2017-03-23T03:19:24Z")

</div>

One way is to keep the field `KNX-Wert` as is and use grok to parse only number into a different field. It should look as below

```auto
filter {
  if "knxmonitor" in [tags] {
    csv {
      columns => ["eventtime", "ms", "Typ", "PA", "GA_KO", "KO-ID", "KNX-Name", "KNX-Wert"]      
    }
    
    # Retrieve numbers from KNX-Wert
    grok {
        match => {
            "KNX-Wert" => [
                "%{NUMBER:KNX-Wert-float}",
                "%{GREEDYDATA}"                
            ]
        }
    }
    date {
      locale => "en"
      match => ['eventtime' , 'yyyy-MM-dd HH:mm:ss']
    }
    mutate {
     remove_field => ["message", "offset", "eventtime", "ms"]
    }
  }
}

```

In Elasticsearch, map `KNX-Wert` as string type if you want to keep this field, or use mutate filter in Logstash to remove it completely. Map `KNX-Wert-float` as float or double as you wish.

Perhaps you can also use ruby to check if the field contains a valid float and act on that.

---

<div class="post-metadata">

**Author:** ![andre22](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@andre22](https://discuss.elastic.co/u/andre22)\
**Post date:** [March 23, 2017, 8:02am UTC](https://discuss.elastic.co/t/type-of-field-changes-mapping-conflict/79545/4 "2017-03-23T08:02:26Z")

</div>

@wenpos and @anhlqn - thank you both. I will give it a try as soon as I can put my hands on it.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 20, 2017, 8:02am UTC](https://discuss.elastic.co/t/type-of-field-changes-mapping-conflict/79545/5 "2017-04-20T08:02:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.