Template Problems

Elastic Stack Elasticsearch
1

Hello there, we are playing arround with elasticsearch and logstash, and i want to change the index template for having more indexed at once:

basically we need that in the template:
curl -XPUT 'localhost:9200/OURINDEX/_settings' -d '
{
"index.mapping.total_fields.limit": 3000
}'

i failed so far, and wonder what i did wrong, i have seen
https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html
and for example this:
https://github.com/Icinga/icingabeat/blob/master/vendor/github.com/elastic/beats/filebeat/filebeat.template.json

this is our actual template:
/etc/logstash# curl -XGET 'bacchus.sit.fraunhofer.de:9200/_template/logstash'

{"logstash":{"order":0,"template":"logstash-","settings":{"index":{"refresh_interval":"5s"}},"mappings":{"default":{"dynamic_templates":[{"message_field":{"mapping":{"fielddata":{"format":"disabled"},"index":"analyzed","omit_norms":true,"type":"string"},"match_mapping_type":"string","match":"message"}},{"string_fields":{"mapping":{"fielddata":{"format":"disabled"},"index":"analyzed","omit_norms":true,"type":"string","fields":{"raw":{"ignore_above":256,"index":"not_analyzed","type":"string"}}},"match_mapping_type":"string","match":""}}],"_all":{"omit_norms":true,"enabled":true},"properties":{"@timestamp":{"type":"date"},"geoip":{"dynamic":true,"properties":{"ip":{"type":"ip"},"latitude":{"type":"float"},"location":{"type":"geo_point"},"longitude":{"type":"float"}}},"@version":{"index":"not_analyzed","type":"string"}}}},"aliases":{}}}

i tried putting the updated with different syntax etc but it would not let me

for example this one :
{"template":"logstash-","settings":{"index":{"mapping":{"total_fields":{"limit":"3000"}},"refresh_interval":"5s"}},"mappings":{"default":{"dynamic_templates":[{"message_field":{"mapping":{"fielddata":{"format":"disabled"},"index":"analyzed","omit_norms":true,"type":"string"},"match_mapping_type":"string","match":"message"}},{"string_fields":{"mapping":{"fielddata":{"format":"disabled"},"index":"analyzed","omit_norms":true,"type":"string","fields":{"raw":{"ignore_above":256,"index":"not_analyzed","type":"string"}}},"match_mapping_type":"string","match":""}}],"_all":{"omit_norms":true,"enabled":true},"properties":{"@timestamp":{"type":"date"},"geoip":{"dynamic":true,"properties":{"ip":{"type":"ip"},"latitude":{"type":"float"},"location":{"type":"geo_point"},"longitude":{"type":"float"}}},"@version":{"index":"not_analyzed","type":"string"}}}},"aliases":{}}}

what would be the correct syntax ? i didnt find examples ? can you name some tools which mape it more human friendly like the .json examples i found maybe ?

thanks for input

2

Why not? Was there an error, what was the error?

3

Hello,

yes in this case here i put the "upload" into a file and got:
curl -XPUT 'http://localhost:9200/_template/logstash' -d@/home/sitadm/indices_mapping.sh
{"error":{"root_cause":[{"type":"not_x_content_exception","reason":"Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}],"type":"not_x_content_exception","reason":"Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"},"status":500}

as i was putting it in a line i got:

{"error":{"root_cause":[{"type":"action_request_validation_exception","reason":"Validation Failed: 1: template is missing;"}],"type":"action_request_validation_exception","reason":"Validation Failed: 1: template is missing;"},"status":400}

for reference, here is the content of the last try:

curl -XPUT 'http://localhost:9200/_template/logstash' -d@/home/sitadm/logstash.json
{"error":{"root_cause":[{"type":"not_x_content_exception","reason":"Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}],"type":"not_x_content_exception","reason":"Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"},"status":500}

content: (i added PUT _template/logstash copared to another try, i moved "template":"logstash-*"} to the bottom )

PUT _template/logstash
{
"settings":{
"index":{
"mapping":{
"total_fields":{
"limit":"3000"
}
},
"refresh_interval":"5s"
}
},
"mappings":{
"default":{
"dynamic_templates":[
{
"message_field":{
"mapping":{
"fielddata":{
"format":"disabled"
},
"index":"analyzed","omit_norms":true,"type":"string"
},
"match_mapping_type":"string","match":"message"
}
},
{
"string_fields":{
"mapping":{
"fielddata":{
"format":"disabled"
},
"index":"analyzed","omit_norms":true,"type":"string","fields":{
"raw":{
"ignore_above":256,"index":"not_analyzed","type":"string"
}
}
},
"match_mapping_type":"string","match":""
}
}
],
"_all":{
"omit_norms":true,"enabled":true
},
"properties":{
"@timestamp":{
"type":"date"
},
"geoip":{
"dynamic":true,"properties":{
"ip":{
"type":"ip"
},
"latitude":{
"type":"float"
},
"location":{
"type":"geo_point"
},
"longitude":{
"type":"float"
}
}
},
"@version":{
"index":"not_analyzed","type":"string"
}
}
}
},
"aliases":{
},
"template":"logstash-"}

4

please, that must be something obvious or easy i am missing ?!

5

I just tested that on 5.4.0 using dev tools and it worked fine, so your syntax seems to be good :slight_smile:

6

thanks , i googled a bit about

and just removed the first Line:

PUT _template/logstash

i guess i could have changed the curl command instead, but it seems to have eaten it !

Have a nice day and thanks alot again

7

Oh, I didn't realise that first line was in the file.
You need that for Console, but not curl :slight_smile:

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.