%{YEAR}-%{MONTHNUM}-%{MONTHDAY})"
]
}
mutate {
rename => {
"app_log_time" => "[@metadata][app_log_time]"
}
}
}
output {
if [env] == "prod" or [env] == "common" {
elasticsearch_java {
#For daily index creation used the time notation, Remove if not required.
index => "jabong-%{env}-%{app}-%{iver}-%{[@metadata][app_log_time]}"
cluster => "elasticsearch"
network_host => "172.16.84.230"
hosts => ["172.16.84.230:9300"]
protocol => "transport"
}
file {
path => "/var/log/shop/%{env}/%{app}/%{app}_%{host}_%{[@metadata][app_log_time]}.log"
}
stdout { codec => rubydebug }
}
}
```
In elasticsearch I am using custom template for indices `jabong-*`
I am using `Kafka 0.8` and `logstash-input-kafka plugin` is at `2.0.8` and Elasticsearch version `1.7.1`
If I start the logstash in a verbose mode, logstash seems to be stuck with the below message:-
```
Using mapping template from {:path=>nil, :level=>:info, :file=>"logstash/outputs/elasticsearch/template_manager.rb", :line=>"6", :method=>"install_template"}
Attempting to install template {:manage_template=>{"template"=>"logstash-*", "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "omit_norms"=>true}, "dynamic_templates"=>[{"message_field"=>{"match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"string", "index"=>"analyzed", "omit_norms"=>true, "fielddata"=>{"format"=>"disabled"}}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"string", "index"=>"analyzed", "omit_norms"=>true, "fielddata"=>{"format"=>"disabled"}, "fields"=>{"raw"=>{"type"=>"string", "index"=>"not_analyzed", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"string", "index"=>"not_analyzed"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"float"}, "longitude"=>{"type"=>"float"}}}}}}}, :level=>:info, :file=>"logstash/outputs/elasticsearch/template_manager.rb", :line=>"8", :method=>"install_template"}
```
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 27, 2016, 8:33pm UTC](https://discuss.elastic.co/t/logstash-2-3-4-getting-stuck-while-attempting-to-install-template-in-elasticsearch/59098/2 "2016-08-27T20:33:50Z")
How do you know the input is working okay, i.e. that Logstash actually has data to process? What happens if you disable the elasticsearch\_java output? Any particular reason to use elasticsearch\_java over the standard elasticsearch output?
---
### Author:  [@Deb](https://discuss.elastic.co/u/Deb)
#### Post date: [August 27, 2016, 8:54pm UTC](https://discuss.elastic.co/t/logstash-2-3-4-getting-stuck-while-attempting-to-install-template-in-elasticsearch/59098/3 "2016-08-27T20:54:05Z")
I modified my output plugin as follows:-
```
output {
if [env] == "prod" or [env] == "common" {
stdout { codec => rubydebug }
}
}
```
Logstash log said:-
`Pipeline main started {:file=>"logstash/agent.rb",:line=>"473", :method=>"start_pipeline"}`
Also I started getting data on the stdout.
We are using `transport` protocol so we are using `elasticsearch_java` output.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 28, 2016, 1:15pm UTC](https://discuss.elastic.co/t/logstash-2-3-4-getting-stuck-while-attempting-to-install-template-in-elasticsearch/59098/4 "2016-08-28T13:15:11Z")
> Also I started getting data on the stdout.
Have you tried increasing the log level by starting Logstash with `--verbose` or even `--debug`? That might give extra clues.
> We are using transport protocol so we are using elasticsearch\_java output.
Yes, but why are you insisting on using the transport protocol?
---
### Author:  [@Deb](https://discuss.elastic.co/u/Deb)
#### Post date: [August 28, 2016, 2:35pm UTC](https://discuss.elastic.co/t/logstash-2-3-4-getting-stuck-while-attempting-to-install-template-in-elasticsearch/59098/5 "2016-08-28T14:35:04Z")
Only after starting the logstash with `--debug` I got the template message that I mentioned in my first post. Because of space issue here I could not post the entire log here. But I have placed full debug log [here](https://gist.github.com/debraj-manna/b3db0ac2f6f60027e280a9ebd40cf615).
It was legacy when the cluster was step-up couple of years back that time we observed some performance issues while using http protocol over the light transport protocol. This is working fine with logstash-1.5.3.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 28, 2016, 2:40pm UTC](https://discuss.elastic.co/t/logstash-2-3-4-getting-stuck-while-attempting-to-install-template-in-elasticsearch/59098/6 "2016-08-28T14:40:47Z")
If it works without the elasticsearch\_java output I'd assume that's where the problem lies. If you sniff the traffic, are any connections being made to the cluster? What if you replace it with a (HTTP based) elasticsearch output?
---
### Author:  [@Deb](https://discuss.elastic.co/u/Deb)
#### Post date: [August 28, 2016, 6:28pm UTC](https://discuss.elastic.co/t/logstash-2-3-4-getting-stuck-while-attempting-to-install-template-in-elasticsearch/59098/7 "2016-08-28T18:28:52Z")
If I switch to elasticsearch output then it is working fine:-
```
output {
if [env] == "prod" or [env] == "common" {
elasticsearch {
index => "jabong-%{env}-%{app}-%{iver}-%{[@metadata][app_log_time]}"
hosts => ["es-master1","es-master2","es-master3"]
}
file {
path => "/var/log/shop/%{env}/%{app}/%{app}_%{host}_%{[@metadata][app_log_time]}.log"
}
stdout { codec => rubydebug }
}
}
```
The issue seems to be only with `elasticsearch_java` output plugin. Is that an issue on `elasticsearch_java`? Should I file an issue?
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 29, 2016, 5:26am UTC](https://discuss.elastic.co/t/logstash-2-3-4-getting-stuck-while-attempting-to-install-template-in-elasticsearch/59098/8 "2016-08-29T05:26:19Z")
It certainly sounds like a bug in the elasticsearch\_java plugin, if nothing else that it doesn't seem to log much to help out in situations like this. I wouldn't wait on this to be fixed though as the plugin isn't particularly actively maintained.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:41am UTC](https://discuss.elastic.co/t/logstash-2-3-4-getting-stuck-while-attempting-to-install-template-in-elasticsearch/59098/9 "2017-07-06T04:41:15Z")