ElasticSearch Index: Type defaults to long instead of float

Elastic Stack Kibana
1

Hi,
I am absolutely new to ELK and don't get my problem solved by Google... I already figured out where the problem is. But some basic knowledge is missing to solve it...

My problem: I can not filter for a float-Number (0 <= confidence < 0.5). Seems like it is handeled as int.

The Reason: It is mapped as long instead of float. I can see it in the mapping-section of the Elasticsearch Index.

I tried:

PUT logstash-2020.03.02/_mapping/doc
{
"properties": {
"data": {
"properties": {
"nlu": {
"properties": {
"intent": {
"properties": {
"confidence": {
"type": "float"
}
}
}
}
}
}
}
}
}

This gives me:
mapper [data.nlu.intent.confidence] cannot be changed from type [long] to [float]

Okay, so how do I solve this? And how to make this persistent?
Currently only one application sends all its data to logstash which forwards it all. Why is there a new index every day? Where does it come from?

Sorry for the newbie problem. I am not sure where to start...

2

As you already said, the field is already mapped to a long in the existing index. This is likely due to having sent a first document where the automatic mapping rules detected the field value as being numeric, but not floating point. In this case the field is mapped as a long.

Mappings for existing fields cannot be changed for an index. I'd recommend creating a complete mapping and storing it in an index template before sending the first data document to an index.

3

Thanks.
Yes, often this value is just 1 - this must have been the case for the first document.

I now created a new Index with correct mapping. Can I move existing documents to this index (Its not that crucial, just a question)?

Just to be sure: So I have to change the config of logstash to use my own index for the future? There is no way to configure this in Kibana?

4

Yes, have a look at the Reindex API.

While you can configure Logstash to format numbers in either integer or floating point format, I'd rather make sure that the correct mapping is part of an index template. Then, regardless of the source setting the correct datatype will be applied for a given field.

5

Thanks, this worked great. I created a template, reindexed everything and now I can filter for float values.

Thanks, have a nice day :slight_smile:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.