Logstash pipelines not visible in stack monitoring

[SamehSaeed]

I'm unable to monitor pipelines through kibana
==>

So i tried to pick elasticsearch, then ingest pipelines and this error popped up upon trying to install elasticsearch integration
==>

How can i monitor pipelines? Is it mandatory to install metric beat?

Using windows, ELK stack running as a service (through NSSM)

logstash.yml :

 pipeline.workers: 1
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.url: "http://localhost:9200"
(the rest of the file is commented)

[Rios]

If you using LS 7.x you could use collectors internal metrics which deprecated. Starting with 8.x, you should use metricbeat and LS module, which is explained here. Also is possible to use APIs for monitoring in raw JSON format.

[SamehSaeed]

Thanks Rios, I'll look into it.
Regarding monitoring APIs, why does it return all pipelines even if some of them were terminated and not working due to some error? (found through logstash-plain log)

Command used through power shell :
Invoke-RestMethod -Method GET -Uri 'http://localhost:9600/_node/pipelines?pretty'

[Rios]

From the documentation:

You can see the stats for a specific pipeline by including the pipeline ID. In the following example, the ID of the pipeline is test:

curl -XGET 'localhost:9600/_node/stats/pipelines/test?pretty'

[SamehSaeed]

Well I have 72 pipelines, sometimes when running logstash all 72 pipelines would start, while at other times some of them gets terminated (still trying to figure out why)

[2023-12-25T09:41:45,779][ERROR][logstash.javapipeline    ][bkgetivrnumbersoffers] Pipeline error {:pipeline_id=>"bkgetivrnumbersoffers", :exception=>#<NameError: uninitialized constant Sequel::JDBC::Oracle::DatabaseMethods
Did you mean?  Sequel::JDBC::Database>, :backtrace=>["org/jruby/RubyModule.java:4309:in `const_missing'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc/oracle.rb:11:in `block in JDBC'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:379:in `adapter_initialize'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/database/misc.rb:169:in `initialize'", "org/jruby/RubyClass.java:904:in `new'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/database/connecting.rb:57:in `connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/core.rb:124:in `connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:123:in `block in jdbc_connect'", "org/jruby/RubyKernel.java:1586:in `loop'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:120:in `jdbc_connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:163:in `open_jdbc_connection'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/inputs/jdbc.rb:309:in `register'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-mixin-ecs_compatibility_support-1.3.0-java/lib/logstash/plugin_mixins/ecs_compatibility_support/target_check.rb:48:in `register'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:237:in `block in register_plugins'", "org/jruby/RubyArray.java:1987:in `each'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:236:in `register_plugins'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:395:in `start_inputs'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:320:in `start_workers'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:194:in `run'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:146:in `block in start'"], "pipeline.sources"=>["D:/ELK 8.10.2/logstash-8.10.2/config/conf.d/bk_services/bkgetivrnumbersoffers_config.conf"], :thread=>"#<Thread:0x1917a21a D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[2023-12-25T09:41:48,671][INFO ][logstash.javapipeline    ][bkgetivrnumbersoffers] Pipeline terminated {"pipeline.id"=>"bkgetivrnumbersoffers"}

The problem is, when i try to check status of a terminated pipeline, status is green

PS C:\Users\ssabdelkader> Invoke-RestMethod -Method GET -Uri 'http://localhost:9600/_node/pipelines/bkgetivrnumbersoffers?pretty'


host         : SCPrWUWS05
version      : 8.10.2
http_address : 127.0.0.1:9600
id           : 1f4078e7-f92c-45a6-9366-956fe385d154
name         : SCPrWUWS05
ephemeral_id : 24367b98-ee2d-4df5-a747-58df975fc306
status       : green
snapshot     : False
pipeline     : @{workers=1; batch_size=125; batch_delay=50}
pipelines    : @{bkgetivrnumbersoffers=}

yet the query was not executed with the other running pipelines.