Adding compound word token filter to a template results in “Failed to install template - response code 500 contacting Elasticsearch”

Elastic Stack Logstash
1

After adding hyphenation decompounder to my custom template, logstash fails to install it: "Got response code '500' contacting Elasticsearch at URL...".

I have already had different token filters like stopwords and stemmers and the template used to work just fine. Also dictionary decompounder with a word list as an array works. Problems start when I supply a path to a word list and for hyphenation decompounder when I supply paths for both word list and hyphenation patterns. I have tried both, absolute and relative paths. As I work with kubernetes, I have made sure that the files exist on the pod with configmap. The elasticsearch version is 7.1.1. and logstash version 6.2.3. My assumption is that either the paths can't be resolved or the files are not compatible with my elasticsearch or logstash version. However, I have implemented the decompounder according to the manual from the elasticsearch documentation: Compound Word Token Filters | Elasticsearch Guide [7.1] | Elastic

Here are the settings for my index:

"index.refresh_interval":"5s",
"analysis":{
"filter":{
"english_stop":{
"type":"stop",
"stopwords":"english"
},
"light_english_stemmer":{
"type":"stemmer",
"language":"light_english"
},
"english_possessive_stemmer":{
"type":"stemmer",
"language":"possessive_english"
},
"german_stop":{
"type":"stop",
"stopwords":"german"
},
"german_stemmer":{
"type":"stemmer",
"language":"light_german"
},
"german_hyphenation_decompounder": {
"type" : "hyphenation_decompounder",
"word_list_path": "/user/share/logstash/config/german_dictionary.txt",
"hyphenation_patterns_path": "/user/share/logstash/config/hyphentation_grammar_ngerman.xml",
"max_subword_size": 22
}
},
"analyzer":{
"custom_english":{
"tokenizer":"standard",
"filter":[
"lowercase",
"english_stop",
"english_possessive_stemmer",
"light_english_stemmer"
]
},
"custom_german":{
"type": "custom",
"tokenizer":"standard",
"filter":[
"lowercase",
"german_stop",
"german_normalization",
"german_stemmer",
"german_hyphenation_decompounder"
]
}
}
}
},

And this is how I install my custom template:

template => "/user/share/logstash/config/elasticsearch-template.custom.json"
template_overwrite => true
template_name => "mytemplate"

I would like to know what the error response '500' means and why I receive this.

2

Is there an error message logged in the elasticsearch logs associated with this 500 error?

3

Yes, this is the whole error message:

[ERROR] 2019-07-17 16:00:39.612 [Ruby-0-Thread-5: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/common.rb:42] elasticsearch - Failed to install template. {:message=>"Got response code '500' contacting Elasticsearch at URL 'http://elasticsearch-master:9200/_template/mytemplate'", :class=>"LogStash::Outputs::Elasticsearch::HttpClient::Pool::BadResponseCodeError", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb:80:in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:291:in perform_request_to_url'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:278:in block in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:373:in with_connection'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:277:in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:285:in block in put'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:352:in template_put'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:86:in template_install'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:28:in install'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:16:in install_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/common.rb:130:in install_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/common.rb:51:in block in setup_after_successful_connection'"]}

4

I asked about an error in the elasticsearch logs, not in the logstash logs.

5

Then no, I can't see any other errors.

6

The word_list_path strings refer to files on the elasticsearch server, right?

7

As I use kubernetes, I have pasted the files into the same folder as the template file on the logstash pod. Logstash configuration files are in the same folder too.

8

I cannot test it to be sure, but I think the word list files should be on the elasticsearch server, not the logstash server.

9

But then the error should appear on the elasticsearch side and not in logstash right? Do you know the difference between the error 400 and 500, because when I change the paths a bit I receive 400.

10

If elasticsearch is returning an error I would always expect it to log something on its side. The status code does not contain enough information to diagnose the problem. You need the error message to do that.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.