Defined Template from logstash not being used by elastic search for mapping


(Chris) #1

Hi all,

I have the following logstash output config to go into elasticsearch from a postgres database

https://pastebin.com/BFCH3tuZ

I have defined the location and my template as the following:

https://pastebin.com/mK5qshKM

When I run logstash I see the output as follows:

[2017-05-24T20:54:10,828][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2017-05-24T20:54:10,982][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>#<URI::HTTP:0xff97ab URL:http://localhost:9200/>}
[2017-05-24T20:54:10,985][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>"/etc/logstash/universe_template.json"}
[2017-05-24T20:54:11,045][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"universe_elastic", "settings"=>{"analysis"=>{"filter"=>{"gr$
[2017-05-24T20:54:11,052][INFO ][logstash.outputs.elasticsearch] Installing elasticsearch template to _template/universe_elastic
[2017-05-24T20:54:11,145][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>[#<URI::Generic:0xe60519 URL://localhost:9200$
[2017-05-24T20:54:11,154][INFO ][logstash.pipeline        ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inf$
[2017-05-24T20:54:11,988][INFO ][logstash.pipeline        ] Pipeline main started
[2017-05-24T20:54:12,079][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2017-05-24T20:54:12,108][INFO ][logstash.inputs.jdbc     ] (0.101000s) select planet.id, planet.x || ':' || planet.y || ':' || planet.z coords, planet.x, planet.y, planet.z ,planetname,ru$
[2017-05-24T20:54:15,006][WARN ][logstash.agent           ] stopping pipeline {:id=>"main"}

When I query elasticseach templates I can see my template listed at:

http://xxxx:9200/_template/
{
"universe_elastic": {
"order": 0,
"template": "universe_elastic",
"settings": {
"index": {
"analysis": {
"filter": {
"gramFilter": {
"token_chars": [
"letter",
"digit",
"punctuation",
"symbol"
], ETC ETC ETC......

However when I run a check on my "universe" index the mapping haven't come through:

https://pastebin.com/hw9hYfLn

I would expect to see the _all field and the include in all references set to true/false. But nothing.. Also the queries do not then use the analyzers I have specified.

Any ideas what might be going wrong here? I have deleted out all the other possible templates created as well as re-created indexes etc.


(Christian Dahlqvist) #2

The template will apply to indices named universe_elastic not just universe. Change this to universe or universe* in order to make it apply.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.