Logstash MetricNotFound

Please have a look at the error in the logstash-plain.log file:

[2021-09-28T15:51:47,918][DEBUG][logstash.agent           ] 2021-09-28 15:51:47 - LogStash::Instrument::MetricStore::MetricNotFound - For path: events. Map keys: [:reloads]:
	/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:241:in `block in get_recursively'
	org/jruby/RubyArray.java:1820:in `each'
	/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:240:in `get_recursively'
	/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:251:in `block in get_recursively'
	org/jruby/RubyArray.java:1820:in `each'
	/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:240:in `get_recursively'
	/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:111:in `block in get'
	org/jruby/ext/thread/Mutex.java:164:in `synchronize'
	/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:110:in `get'
	/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:124:in `get_shallow'
	/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:173:in `block in extract_metrics'
	org/jruby/RubyArray.java:1820:in `each'
	org/jruby/RubyEnumerable.java:1126:in `inject'
	/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:149:in `extract_metrics'
	/usr/share/logstash/logstash-core/lib/logstash/api/service.rb:45:in `extract_metrics'
	/usr/share/logstash/logstash-core/lib/logstash/api/commands/base.rb:37:in `extract_metrics'
	/usr/share/logstash/logstash-core/lib/logstash/api/commands/stats.rb:73:in `events'
	/usr/share/logstash/logstash-core/lib/logstash/api/modules/node_stats.rb:61:in `events_payload'
	/usr/share/logstash/logstash-core/lib/logstash/api/modules/node_stats.rb:37:in `block in NodeStats'
	org/jruby/RubyMethod.java:115:in `call'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1675:in `block in compile!'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1013:in `block in route!'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1032:in `route_eval'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1013:in `block in route!'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1061:in `block in process_route'
	org/jruby/RubyKernel.java:1189:in `catch'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1059:in `process_route'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1011:in `block in route!'
	org/jruby/RubyArray.java:1820:in `each'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1008:in `route!'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1129:in `block in dispatch!'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `block in invoke'
	org/jruby/RubyKernel.java:1189:in `catch'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `invoke'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1124:in `dispatch!'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:939:in `block in call!'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `block in invoke'
	org/jruby/RubyKernel.java:1189:in `catch'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `invoke'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:939:in `call!'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:929:in `call'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-protection-2.1.0/lib/rack/protection/xss_header.rb:18:in `call'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-protection-2.1.0/lib/rack/protection/path_traversal.rb:16:in `call'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-protection-2.1.0/lib/rack/protection/json_csrf.rb:26:in `call'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-protection-2.1.0/lib/rack/protection/base.rb:50:in `call'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-protection-2.1.0/lib/rack/protection/base.rb:50:in `call'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-protection-2.1.0/lib/rack/protection/frame_options.rb:31:in `call'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-2.2.3/lib/rack/null_logger.rb:11:in `call'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:216:in `call'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1991:in `call'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-2.2.3/lib/rack/urlmap.rb:74:in `block in call'
	org/jruby/RubyArray.java:1820:in `each'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `call'
	/usr/share/logstash/logstash-core/lib/logstash/api/rack_app.rb:74:in `call'
	/usr/share/logstash/logstash-core/lib/logstash/api/rack_app.rb:48:in `call'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-2.2.3/lib/rack/builder.rb:244:in `call'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/puma-4.3.8-java/lib/puma/server.rb:718:in `handle_request'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/puma-4.3.8-java/lib/puma/server.rb:472:in `process_client'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/puma-4.3.8-java/lib/puma/server.rb:328:in `block in run'
	/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/puma-4.3.8-java/lib/puma/thread_pool.rb:134:in `block in spawn_thread'
[2021-09-28T15:51:47,928][ERROR][logstash.agent           ] Internal API server error {:status=>500, :request_method=>"GET", :path_info=>"/_node/stats", :query_string=>"vertices=true", :http_version=>"HTTP/1.1", :http_accept=>nil, :error=>"Unexpected Internal Error", :class=>"LogStash::Instrument::MetricStore::MetricNotFound", :message=>"For path: events. Map keys: [:reloads]", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:241:in `block in get_recursively'", "org/jruby/RubyArray.java:1820:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:240:in `get_recursively'", "/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:251:in `block in get_recursively'", "org/jruby/RubyArray.java:1820:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:240:in `get_recursively'", "/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:111:in `block in get'", "org/jruby/ext/thread/Mutex.java:164:in `synchronize'", "/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:110:in `get'", "/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:124:in `get_shallow'", "/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:173:in `block in extract_metrics'", "org/jruby/RubyArray.java:1820:in `each'", "org/jruby/RubyEnumerable.java:1126:in `inject'", "/usr/share/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:149:in `extract_metrics'", "/usr/share/logstash/logstash-core/lib/logstash/api/service.rb:45:in `extract_metrics'", "/usr/share/logstash/logstash-core/lib/logstash/api/commands/base.rb:37:in `extract_metrics'", "/usr/share/logstash/logstash-core/lib/logstash/api/commands/stats.rb:73:in `events'", "/usr/share/logstash/logstash-core/lib/logstash/api/modules/node_stats.rb:61:in `events_payload'", "/usr/share/logstash/logstash-core/lib/logstash/api/modules/node_stats.rb:37:in `block in NodeStats'", "org/jruby/RubyMethod.java:115:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1675:in `block in compile!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1013:in `block in route!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1032:in `route_eval'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1013:in `block in route!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1061:in `block in process_route'", "org/jruby/RubyKernel.java:1189:in `catch'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1059:in `process_route'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1011:in `block in route!'", "org/jruby/RubyArray.java:1820:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1008:in `route!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1129:in `block in dispatch!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `block in invoke'", "org/jruby/RubyKernel.java:1189:in `catch'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `invoke'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1124:in `dispatch!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:939:in `block in call!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `block in invoke'", "org/jruby/RubyKernel.java:1189:in `catch'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `invoke'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:939:in `call!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:929:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-protection-2.1.0/lib/rack/protection/xss_header.rb:18:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-protection-2.1.0/lib/rack/protection/path_traversal.rb:16:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-protection-2.1.0/lib/rack/protection/json_csrf.rb:26:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-protection-2.1.0/lib/rack/protection/base.rb:50:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-protection-2.1.0/lib/rack/protection/base.rb:50:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-protection-2.1.0/lib/rack/protection/frame_options.rb:31:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-2.2.3/lib/rack/null_logger.rb:11:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:216:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1991:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-2.2.3/lib/rack/urlmap.rb:74:in `block in call'", "org/jruby/RubyArray.java:1820:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `call'", "/usr/share/logstash/logstash-core/lib/logstash/api/rack_app.rb:74:in `call'", "/usr/share/logstash/logstash-core/lib/logstash/api/rack_app.rb:48:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rack-2.2.3/lib/rack/builder.rb:244:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/puma-4.3.8-java/lib/puma/server.rb:718:in `handle_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/puma-4.3.8-java/lib/puma/server.rb:472:in `process_client'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/puma-4.3.8-java/lib/puma/server.rb:328:in `block in run'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/puma-4.3.8-java/lib/puma/thread_pool.rb:134:in `block in spawn_thread'"]}
[2021-09-28T15:51:47,929][ERROR][logstash.agent           ] API HTTP Request {:status=>500, :request_method=>"GET", :path_info=>"/_node/stats", :query_string=>"vertices=true", :http_version=>"HTTP/1.1", :http_accept=>nil}


If you need any other information, please let me know.

[2021-09-28T15:51:50,888][ERROR][logstash.javapipeline    ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: Using api_key authentication requires SSL/TLS secured communication using the `ssl => true` option>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.3-java/lib/logstash/plugin_mixins/elasticsearch/common.rb:48:in `validate_authentication'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.3-java/lib/logstash/plugin_mixins/elasticsearch/common.rb:26:in `build_client'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.3-java/lib/logstash/outputs/elasticsearch.rb:275:in `register'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:131:in `register'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:68:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:228:in `block in register_plugins'", "org/jruby/RubyArray.java:1820:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:227:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:585:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:240:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:185:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:137:in `block in start'"], "pipeline.sources"=>["/etc/logstash/conf.d/haproxy-parse.conf", "/etc/logstash/conf.d/pexip-admin.conf", "/etc/logstash/conf.d/pexip-web.conf", "/etc/logstash/conf.d/wowza-access-stats-error.conf"], :thread=>"#<Thread:0x140a61b3 run>"}
[2021-09-28T15:51:50,890][INFO ][logstash.javapipeline    ][main] Pipeline terminated {"pipeline.id"=>"main"}
[2021-09-28T15:51:50,906][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[2021-09-28T15:51:50,960][DEBUG][logstash.instrument.periodicpoller.os] Stopping
[2021-09-28T15:51:50,966][DEBUG][logstash.instrument.periodicpoller.jvm] Stopping
[2021-09-28T15:51:50,967][DEBUG][logstash.instrument.periodicpoller.persistentqueue] Stopping
[2021-09-28T15:51:50,967][DEBUG][logstash.instrument.periodicpoller.deadletterqueue] Stopping
[2021-09-28T15:51:50,970][DEBUG][logstash.agent           ] Shutting down all pipelines {:pipelines_count=>1}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.