Does every day start the same?
Do you find yourself re-using the same index settings and mappings frequently? Do those mappings have lots of fields with repetitive naming conventions?
If so, fear not! You aren't alone, and there's relief at hand. Let's talk about Dynamic Templates and Index Templates.
Dynamic Templates
Dynamic templates are useful for indexes with large and varying numbers of fields, such as gathering sensor data, as well as for when automatic type detection needs consistent tweaking.
With dynamic templates, we assign types to new fields based on name, path, and detected type, in any combination. This simple example maps all fields beginning with f_ to the float type and all fields beginning with l_ to the long type, except for fields ending with _text - perhaps we have a data shipper that incorrectly encodes numeric types so that they are detected as text, and we'd also like to annotate each field with a description:
PUT my_index
{
"mappings": {
"_doc": {
"dynamic_templates": [
{
"break_texts": {
"match": "*_text",
"mapping": {
"type": "text"
}
}
},
{
"map_floats": {
"match": "f_*",
"mapping": {
"type": "float"
}
}
},
{
"map_longs": {
"match": "l_*",
"mapping": {
"type": "long"
}
}
}
]
}
}
}
PUT my_index/_doc/1
{
"l_sensor_42": "32769",
"l_sensor_42_text": "The world is full of magic things, patiently waiting for our senses to grow sharper",
"f_nom_num": "3.14159"
}
Dynamic field creation can be selectively allowed and disallowed with the dynamic setting.
Index Templates
When creating indexes, you may often apply common settings and template configurations. For example, until Elasticsearch 7.0, each new index defaults to 5 primary shards. However, since that may be overkill with indexes less than 20-30GB in size, there's a simple way to avoid having to specify number_of_shards: 1 each time an index is created:
PUT _template/default_settings
{
"index_patterns": ["*"],
"order": 0,
"settings": {
"number_of_shards": 1
}
}
The order parameter specifies the order in which templates are applied, low to high. In this way, a second template could be specified for your large customers indexes, which need more primary shards to hold all your customer data:
PUT _template/customers_shards
{
"index_patterns": ["customers-*"],
"order": 10,
"settings": {
"number_of_shards": 6
}
}
Should you wish to change these values later, any settings explicitly specified at index creation time will override template-derived values. Also, it is recommended to leave some spacing in between order values, so that you may insert a new template at a later time.
Similarly, mapping configurations can also be templated, and here we provide the dynamic templates we created above for use by any index:
PUT _template/default_dynamic_templates
{
"index_patterns": ["*"],
"order": 5,
"mappings": {
"_doc": {
"dynamic_templates": [
{ ... }
]
}
}
}
This brief overview of dynamic templates in action should get you started on a streamlined day-to-day operational flow. Please do read through the documentation, as there's much more cool functionality in each of these concepts!