Hi everyone,
I’m trying to apply my first ElasticSearch Index Template, and am having some issues. I think I’m both a little confused about the process of wedding a Template to an Index, and about the exact syntax of the template itself.
The mission: My ElasticSearch receives network data from a Logstash server. Within the raw data are IP addresses, which ES currently interprets to be text strings. I need an Index Template to transform those strings into proper “ip” data structures.
My plan was this:
- Reconfigure Logstash to send data into a new index called “MyIndex.” But do not start Logstash yet.
- Create a new Index Template called “my_template” in ES. Make sure template “my_template” specifies that it is to be used with “MyIndex” (See below)
- Start Logstash
I’m not clear how ES understands which templates get applied to what Indices, but I think that’s done with a line in the template itself.
Speaking of, here’s the Index Template I cobbled together:
curl -X PUT "localhost:9200/_template/my_template" -H 'Content-Type: application/json' -d'
{
"template": "MyIndex",
"order": 1,
"settings": {
"index": {
"refresh_interval": "5s"
}
},
"mappings": {
"default": {
"_all": {
"norms": false,
"enabled": true
},
"properties": {
"Packet.L3.Src": { "type": "ip"},
"Packet.L3.Dst": { "type": "ip"}
}
}
}
}'
As you can see, all I really want it to do is interpret the data fields “Packet.L3.Src” and “Packet.L3.Dst” as IP addresses. Also, note that first line within the outer brackets: "template": "MyIndex". I assume this is how ES understands that this template is to be applied to Index “MyIndex.”
As a safety check, I ran the above template through a JSON validator. (here) Everything looked good on that front.
So I should be ready to rock-n-roll now. However, when I cut-n-paste the template into ES, I get the following error (I’ve included newlines for better readability):
{"error":
{"root_cause":
[{"type":"mapper_parsing_exception",
"reason":"Root mapping definition has unsupported parameters:
[default : {_all={norms=false, enabled=true},
properties={Packet.L3.Src={type=ip}, Packet.L3.Dst={type=ip}}}]"}],
"type":"mapper_parsing_exception",
"reason":"Failed to parse mapping [_doc]:
Root mapping definition has unsupported parameters:
[default :
{_all={norms=false, enabled=true},
properties={Packet.L3.Src={type=ip}, Packet.L3.Dst={type=ip}}}]",
"caused_by":
{"type":"mapper_parsing_exception",
"reason":"Root mapping definition has unsupported parameters:
[default : {_all={norms=false, enabled=true},
properties={Packet.L3.Src={type=ip}, Packet.L3.Dst={type=ip}}}]"
}
}
,"status":400 }
I’m not sure what this means, but when I see things like “parsing exception,” I assume I have a syntax error that is throwing off the parsing of my template. Like, maybe I have a comma at the end of a line where I shouldn’t… or vice versa…? I don’t know.
FULL DISCLOSURE: I’ve been working on this issue in the ES Forum, and posted an earlier part of my issues was posted here.
Anyway, I’m hoping someone can point out the error of my ways. Any advice is appreciated!