Hi Team,
My Company is now working towards the microservices in azure platform and i have been assigned to work on Azure Eventhub integration to my on prem ELK.
I have completed required conf changes and updated the logstash conf but unable to execute the plugin.
Error Details:
[2020-03-26T10:06:17,527][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x2f1d98c6 run>"}
[2020-03-26T10:06:17,653][INFO ][logstash.agent ] Pipelines running {:count=>1, :pipelines=>["main"]}
[2020-03-26T10:06:17,658][INFO ][logstash.inputs.azureeventhubs] Event Hub eventhubentityname is initializing...
[2020-03-26T10:06:17,689][INFO ][com.microsoft.azure.eventprocessorhost.EventProcessorHost] host logstash-28c1a14c-334a-4c4a-8064-0bb9bc5ff6bb: New EventProc
essorHost created.
[2020-03-26T10:06:17,740][ERROR][com.microsoft.azure.eventprocessorhost.EventProcessorHost] host logstash-28c1a14c-334a-4c4a-8064-0bb9bc5ff6bb: Failure initi
alizing default lease and checkpoint manager.
[2020-03-26T10:06:17,747][ERROR][logstash.inputs.azureeventhubs] Event Hub failed during initialization. {:event_hub_name=>"eventhubentityname", :exception=>
java.lang.RuntimeException: Failure initializing Storage lease manager, :backtrace=>["com.microsoft.azure.eventprocessorhost.EventProcessorHost.registerEvent
ProcessorFactory(com/microsoft/azure/eventprocessorhost/EventProcessorHost.java:480)", "java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)",
"org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:423)", "org.jruby.javasupport.JavaMethod.invokeDire
ct(org/jruby/javasupport/JavaMethod.java:290)", "opt.bea.ELKSTACK.logstash_minus_6_dot_2_dot_4.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_m
inus_azure_event_hubs_minus_1_dot_2_dot_2.lib.logstash.inputs.azure_event_hubs.block in run(/opt/bea/ELKSTACK/logstash-6.2.4/vendor/bundle/jruby/2.3.0/gems/l
ogstash-input-azure_event_hubs-1.2.2/lib/logstash/inputs/azure_event_hubs.rb:430)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:289)", "org.jruby.RubyPr
oc.call(org/jruby/RubyProc.java:246)", "java.lang.Thread.run(java/lang/Thread.java:748)"]}
[2020-03-26T10:06:19,422][INFO ][logstash.pipeline ] Pipeline has terminated {:pipeline_id=>"main", :thread=>"#<Thread:0x2f1d98c6 run>"}
[2020-03-26T10:12:12,252][WARN ][logstash.runner ] SIGINT received. Shutting down.
Logstash Conf file.
input {
azure_event_hubs {
config_mode => "advanced"
event_hubs => [
{ "eventhubentityname" => {
event_hub_connection => "Endpoint=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
}}
]
storage_connection => "DefaultEndpointsProtocol=yyyyyyyyyyyyyyyyyyyyyy"
type => "azure_event_hub"
}
}
filter {
if [type] == "azure_event_hub" {
json {
source => "message"
}
split {
field => ["records"]
}
}
}
output {
if [type] == "azure_event_hub" {
elasticsearch {
action => "index"
hosts => ["http://localhost:9200" ]
index => "azure_event_hub"
}
}
}