Dealing with unknown field types


#1

What is the best way to define mappings / settings where there are about two dozen known fixed fields with a set type but also a large set of unknown fields and types. I have tried disable auto mapping but that does not seem to work. It is still automatically defining the mapping and then of course having indexing errors as a result after only a few documents.


(Mark Walkom) #2

Do you want those other fields?


#3

Definitely. I am just having a hard time trying to come up with a one size fits all solution that tells ES to just take any key that does not have a defined mapping and make it a string.


#4

+1, I'm trying to solve this as well. Reading the elasticsearch docs I don't see an easy way to do this, and it looks like the logstash mutate convert function only has the ability to match known fields. I'm trying to implement a solution where known fields are mapped to a predefined type, and unknown fields are forced to string. I have multiple groups sharing ES clusters and it's fairly easy for them to step on each other with differing datatypes.


(system) #5