Can I add dynamic index mapping from Logstash

[ankitdevnalkar]

I am creating index dynamically(e.g wallet-tx-customer-%{+YYYY.MM}) from Logstash. For some field I am getting an error mapper [usd] cannot be changed from type [long] to [float] and that event doesn't get indexed.

I want to do index mapping dynamically from Logstash. Is that possible?

[Badger]

@ankitdevnalkar

I want to do index mapping dynamically from Logstash. Is that possible?

Yes, it is, and most likely that is exactly what is happening. elasticsearch sets the type of the field based on the first event that contains that field. Once it is set it cannot change. The rules for dynamic mapping say that it the JSON received from logstash contains an integer, then the field type is set to long, and if the JSON contains a double, then the field type is set to float.

So if the first event contains "usd": 5 then the field type will be set to long. If a subsequent event arrives that contains "usd": 4.99 then it cannot be indexed.

You could fix this with an index template to force the mapping to be float, or you could just use

mutate { convert => { "usd" => "float" } }

so that first event would contain "usd": 5.0.

Lastly, if the 4.99 arrives first, elasticsearch will have no problem convert 5 to 5.0 itself.

[ankitdevnalkar]

Thanks @Badger it really helped.