Template index with dynamic mapping to float

Elastic Stack Elasticsearch
1

Hi all, :slight_smile:
how I can manage and deploy a dynamic mapping to elasticsearch (which i will merge into index template later on) that will take a specific type and all his fields will be converted to float type.

The point is to not mutate the fields in logstash for the specific type.

my not working example (via sense):
type-a fields will be converted into floats and type-b will be converted to string not_analyzed

PUT /_template/my_type
{
"template": "test*",
"settings": {
"number_of_replicas": "1",
"number_of_shards": "5",
"refresh_interval": "5s"
},
"mappings": {
"type-a" : {
"dynamic_templates": [
{
"float_fields": {
"mapping": { "type": "float", "null_value": "NULL", "omit_norms": true},
"match": "",
"match_mapping_type": "float"
}
}
]
},
"type-b" : {
"dynamic_templates": [
{
"string_fields": {
"mapping": { "index": "not_analyzed", "type": "string", "omit_norms": true},
"match" : "",
"match_mapping_type": "string"
}
}
]
}
}
}

thanks! :smile:

2

bump :smiley:

3

Can you give example of the data you intend to index and how you expect it to be mapped?

4

Edit:
Its generally kind of dynamic mapping for float/double, if you have any example that I can use it will be much appreciate. Fields mapping into float instead of string with dynamic mapping so all new fields as well will be floats

I have csv with only float/double values inside it, I don't want for each field to write mutate on logstash conf and to manage the conf it self (if csv columns changes), therefore I created a template index with in it I added dynamic mapping for float (tried it with double as well) but eventually the mapping remain string

for example:
file with the columns col_a, col_b, col_c
1.3,1.4,1.5
1.6,1.7,1.8

col_a, col_b, col_c need to be float/double without me saying mutate and any other field that will be added to this type (this file) will be mapped as float/double before inserting to elasticsearch.
Any ideas?

my template index with dynamic mapping:
PUT /_template/test
{
"template": "test*",
"mappings": {
"testme" : {
"dynamic_templates": [
{
"float_fields": {
"match": "*"
"mapping": { "type": "float" },
"match_mapping_type": "float"
}
}
]
}
}
}

5

As the CSV filter can not convert strings to other types, you will need to use the mutate filter. You can not send a string containing a float to Elasticsearch and get it converted to a float through mappings. If the value is a string in the document it will be mapped as a string in Elasticsearch.

1 Like
6

how i can do dynamic mapping to float?
logstash getting a field and without mutate to set it as float in the elasticsearch, csv is just the plugin that i used, i can get the field via grok or something else...

how is it working that for string i can say not_analyzed, it looks for me kind the same
take this field and treat him like float/double :smile:

appreciate your efforts!