Failed to install template logstash error


#1

I am trying to send data from MySQL to Elasticsearch using Logstash. However, after it starts Logstash fails with the following error:

Failed to install template. {:message=>"undefined method update' for nil:NilClass", :class=>"NoMethodError"om,mon.rb:50:inblock in setup_after_successful_connection'
[/details]

Here the full stacktrace:

Stacktrace

Failed to install template. {:message=>"undefined method update' for nil:NilClass", :class=>"NoMethodError", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.0.1-java/lib/logstash/outputs/elasticsearch/template_manager.rb:39:inadd_ilm_settings_to_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.0.1-java/lib/logstash/outputs/elasticsearch/template_manager.rb:14:in install_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.0.1-java/lib/logstash/outputs/elasticsearch/common.rb:129:ininstall_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.0.1-java/lib/logstash/outputs/elasticsearch/common.rb:50:in `block in setup_after_successful_connection'

Here is my Logstash config

input {
    jdbc {
        jdbc_connection_string => "jdbc:mysql://exampleserver:3306/resource_map_development"
        jdbc_driver_library => "/usr/share/logstash/lib/mysql-connector.jar"
        jdbc_driver_class => "com.mysql.cj.jdbc.Driver"
        jdbc_user => "***"
        jdbc_password => "**"
        statement => "select * from site
        where st.id > :sql_last_value
        order by site_id;"
        
        schedule => "*/10 * * * *"
        use_column_value => true
        tracking_column => site_id
    }
}

filter {
    json {
        source => "property"
    }

    date {
        match => ["createdAt", "YYYY-MM-dd HH:mm:ss"]
        target => "createdAt"
     }

    if ![grade] {
        mutate {
            add_field => {"grade" => -1}
        }
    }
    if [latitude] != 0 and [longitude] != 0 {
        mutate { convert => {"latitude" => "float"} }
        mutate { convert => {"longitude" => "float"} }
        mutate { rename => {"latitude" => "[location][lat]"} }
        mutate { rename => {"longitude" => "[location][lon]"} }

    }
    else {
        mutate {
            remove_field => ["latitude", "longitude"]
        }
    }

output {
    stdout {codec => json_lines}
    elasticsearch {
        template => "/usr/share/template/elastic-template.json"
        hosts => "elasticsearch:9200"
        index => "facility"
        template_overwrite => true
        manage_template => true
        document_id => "%{site_id}"
    }
}

I am using a docker setup to run Logstash and Elasticsearch using this repo as a template

How can I solve this issue?


(David Kemp) #2

I am getting a similar error after upgrading to Elasticsearch 7.0.0 (was working in 6.3).

I am getting

[2019-04-14T09:15:36,933][ERROR][logstash.outputs.elasticsearch] Failed to install template. {:message=>"undefined method `update' for nil:NilClass", :class=>"NoMethodError", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.0.1-java/lib/logstash/outputs/elasticsearch/template_manager.rb:39:in `add_ilm_settings_to_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.0.1-java/lib/logstash/outputs/elasticsearch/template_manager.rb:14:in `install_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.0.1-java/lib/logstash/outputs/elasticsearch/common.rb:129:in `install_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.0.1-java/lib/logstash/outputs/elasticsearch/common.rb:50:in `block in setup_after_successful_connection'"]}

#3

@davidbkemp You are right. The problem occurs on v7.x. I downgraded Logstash to v6.7 and it is working.