I am trying to load data from sql server to Elasticsearch using logstash. For index creation i am using index template that is specified in the config file of logstash.
Output field of logstash.config :
output{
stdout { codec => json_lines }
elasticsearch {
hosts => ["localhost:9200"]
document_id => "%{[id]}"
manage_template => true
template => "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/logstash_test.json"
template_name => "logstash_test"
template_overwrite => true
}
}
Error while running logstash :
[2021-01-25T19:48:28,232][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2021-01-25T19:48:28,298][INFO ][logstash.outputs.elasticsearch][main] Using mapping template from {:path=>"C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/logstash_test.json"}
[2021-01-25T19:48:28,370][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, "pipeline.sources"=>["C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/sampleconf.conf"], :thread=>"#<Thread:0x797ce1a8 run>"}
[2021-01-25T19:48:28,509][ERROR][logstash.outputs.elasticsearch][main] Failed to install template. {:message=>"undefined method `update' for nil:NilClass", :class=>"NoMethodError", :backtrace=>["C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:42:in `add_ilm_settings_to_template'", "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:15:in `install_template'", "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/common.rb:218:in `install_template'", "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/common.rb:49:in `block in setup_after_successful_connection'"]}
[2021-01-25T19:48:28,546][INFO ][logstash.outputs.elasticsearch][main] Creating rollover alias <logstash-{now/d}-000001>
[2021-01-25T19:48:29,463][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>1.08}
[2021-01-25T19:48:29,802][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
[2021-01-25T19:48:29,888][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
Index template configuration:
{
"index_patterns":"*-logstash_test-*",
"priority":1,
"template":{
"settings":{
"analysis":{
"analyzer":{
"test_autocomplete":{
"tokenizer":"test_tokenizer"
}
},
"tokenizer":{
"test_tokenizer":{
"custom_token_chars":"'-",
"max_gram":"3",
"min_gram":"3",
"token_chars":[
],
"type":"ngram"
}
}
},
"mappings":{
"properties":{
"test_id":{
"type":"text",
"analyzer":"test_autocomplete"
}
}
}
}
}
}
Although it fails the index gets created with the data, but the analyzers specified in the settings{..}
does not reflect. I am new to ELK stack and not sure on why this is happening.
Elasticsearch Version : 7.10.1
Logstash Version : 7.10.1