Hi,
I setup logstash 7.11.1 pipeline on windows 10 but it is showing No configuration found in the configured sources
I am running logstash by logstash --debug command with administrator in cmd.
Below is my complete log details:
"Using bundled JDK: ""
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Sending Logstash logs to C:/Users/myuser/Desktop/ELKSetup/logstash/logs which is now configured via log4j2.properties
[2021-03-03T08:26:17,224][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.11.1", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc OpenJDK 64-Bit Server VM 11.0.8+10 on 11.0.8+10 +indy +jit [mswin32-x86_64]"}
[2021-03-03T08:26:17,380][DEBUG][logstash.modules.scaffold] Found module {:module_name=>"fb_apache", :directory=>"C:/Users/myuser/Desktop/ELKSetup/logstash/modules/fb_apache/configuration"}
[2021-03-03T08:26:17,380][DEBUG][logstash.plugins.registry] Adding plugin to the registry {:name=>"fb_apache", :type=>:modules, :class=>#<LogStash::Modules::Scaffold:0x3b82fb7 @directory="C:/Users/myuser/Desktop/ELKSetup/logstash/modules/fb_apache/configuration", @module_name="fb_apache", @kibana_version_parts=["6", "0", "0"]>}
[2021-03-03T08:26:17,396][DEBUG][logstash.modules.scaffold] Found module {:module_name=>"netflow", :directory=>"C:/Users/myuser/Desktop/ELKSetup/logstash/modules/netflow/configuration"}
[2021-03-03T08:26:17,396][DEBUG][logstash.plugins.registry] Adding plugin to the registry {:name=>"netflow", :type=>:modules, :class=>#<LogStash::Modules::Scaffold:0x67cb11de @directory="C:/Users/myuser/Desktop/ELKSetup/logstash/modules/netflow/configuration", @module_name="netflow", @kibana_version_parts=["6", "0", "0"]>}
[2021-03-03T08:26:17,458][DEBUG][logstash.runner ] -------- Logstash Settings (* means modified) ---------
[2021-03-03T08:26:17,462][DEBUG][logstash.runner ] node.name: "elasticnode1"
[2021-03-03T08:26:17,463][DEBUG][logstash.runner ] path.data: "C:/Users/myuser/Desktop/ELKSetup/logstash/data"
[2021-03-03T08:26:17,465][DEBUG][logstash.runner ] modules.cli: <Java::OrgLogstashUtil::ModulesSettingArray:1 []>
[2021-03-03T08:26:17,466][DEBUG][logstash.runner ] modules: []
[2021-03-03T08:26:17,468][DEBUG][logstash.runner ] modules_list: []
[2021-03-03T08:26:17,469][DEBUG][logstash.runner ] modules_variable_list: []
[2021-03-03T08:26:17,474][DEBUG][logstash.runner ] modules_setup: false
[2021-03-03T08:26:17,477][DEBUG][logstash.runner ] config.test_and_exit: false
[2021-03-03T08:26:17,478][DEBUG][logstash.runner ] config.reload.automatic: false
[2021-03-03T08:26:17,480][DEBUG][logstash.runner ] config.reload.interval: #<Java::OrgLogstashUtil::TimeValue:0x82d6e07>
[2021-03-03T08:26:17,482][DEBUG][logstash.runner ] config.support_escapes: false
[2021-03-03T08:26:17,483][DEBUG][logstash.runner ] config.field_reference.parser: "STRICT"
[2021-03-03T08:26:17,485][DEBUG][logstash.runner ] metric.collect: true
[2021-03-03T08:26:17,486][DEBUG][logstash.runner ] pipeline.id: "main"
[2021-03-03T08:26:17,487][DEBUG][logstash.runner ] pipeline.system: false
[2021-03-03T08:26:17,489][DEBUG][logstash.runner ] pipeline.workers: 8
[2021-03-03T08:26:17,492][DEBUG][logstash.runner ] pipeline.batch.size: 125
[2021-03-03T08:26:17,493][DEBUG][logstash.runner ] pipeline.batch.delay: 50
[2021-03-03T08:26:17,494][DEBUG][logstash.runner ] pipeline.unsafe_shutdown: false
[2021-03-03T08:26:17,496][DEBUG][logstash.runner ] pipeline.java_execution: true
[2021-03-03T08:26:17,498][DEBUG][logstash.runner ] pipeline.reloadable: true
[2021-03-03T08:26:17,499][DEBUG][logstash.runner ] pipeline.plugin_classloaders: false
[2021-03-03T08:26:17,501][DEBUG][logstash.runner ] pipeline.separate_logs: false
[2021-03-03T08:26:17,503][DEBUG][logstash.runner ] pipeline.ordered: "auto"
[2021-03-03T08:26:17,505][DEBUG][logstash.runner ] pipeline.ecs_compatibility: "disabled"
[2021-03-03T08:26:17,508][DEBUG][logstash.runner ] path.plugins: []
[2021-03-03T08:26:17,509][DEBUG][logstash.runner ] config.debug: false
[2021-03-03T08:26:17,511][DEBUG][logstash.runner ] *log.level: "debug" (default: "info")
[2021-03-03T08:26:17,513][DEBUG][logstash.runner ] version: false
[2021-03-03T08:26:17,515][DEBUG][logstash.runner ] help: false
[2021-03-03T08:26:17,517][DEBUG][logstash.runner ] log.format: "plain"
[2021-03-03T08:26:17,519][DEBUG][logstash.runner ] http.enabled: true
[2021-03-03T08:26:17,520][DEBUG][logstash.runner ] http.host: "127.0.0.1"
[2021-03-03T08:26:17,523][DEBUG][logstash.runner ] http.port: 9600..9700
[2021-03-03T08:26:17,525][DEBUG][logstash.runner ] http.environment: "production"
[2021-03-03T08:26:17,526][DEBUG][logstash.runner ] queue.type: "memory"
[2021-03-03T08:26:17,528][DEBUG][logstash.runner ] queue.drain: false
[2021-03-03T08:26:17,529][DEBUG][logstash.runner ] queue.page_capacity: 67108864
[2021-03-03T08:26:17,531][DEBUG][logstash.runner ] queue.max_bytes: 1073741824
[2021-03-03T08:26:17,533][DEBUG][logstash.runner ] queue.max_events: 0
[2021-03-03T08:26:17,535][DEBUG][logstash.runner ] queue.checkpoint.acks: 1024
[2021-03-03T08:26:17,537][DEBUG][logstash.runner ] queue.checkpoint.writes: 1024
[2021-03-03T08:26:17,539][DEBUG][logstash.runner ] queue.checkpoint.interval: 1000
[2021-03-03T08:26:17,541][DEBUG][logstash.runner ] queue.checkpoint.retry: false
[2021-03-03T08:26:17,543][DEBUG][logstash.runner ] dead_letter_queue.enable: false
[2021-03-03T08:26:17,544][DEBUG][logstash.runner ] dead_letter_queue.max_bytes: 1073741824
[2021-03-03T08:26:17,546][DEBUG][logstash.runner ] dead_letter_queue.flush_interval: 5000
[2021-03-03T08:26:17,548][DEBUG][logstash.runner ] slowlog.threshold.warn: #<Java::OrgLogstashUtil::TimeValue:0x5fbda121>
[2021-03-03T08:26:17,550][DEBUG][logstash.runner ] slowlog.threshold.info: #<Java::OrgLogstashUtil::TimeValue:0x4d3d33b6>
[2021-03-03T08:26:17,551][DEBUG][logstash.runner ] slowlog.threshold.debug: #<Java::OrgLogstashUtil::TimeValue:0x5e76eb5a>
[2021-03-03T08:26:17,553][DEBUG][logstash.runner ] slowlog.threshold.trace: #<Java::OrgLogstashUtil::TimeValue:0x53781df3>
[2021-03-03T08:26:17,555][DEBUG][logstash.runner ] keystore.classname: "org.logstash.secret.store.backend.JavaKeyStore"
[2021-03-03T08:26:17,557][DEBUG][logstash.runner ] keystore.file: "C:/Users/myuser/Desktop/ELKSetup/logstash/config/logstash.keystore"
[2021-03-03T08:26:17,558][DEBUG][logstash.runner ] path.queue: "C:/Users/myuser/Desktop/ELKSetup/logstash/data/queue"
[2021-03-03T08:26:17,561][DEBUG][logstash.runner ] path.dead_letter_queue: "C:/Users/myuser/Desktop/ELKSetup/logstash/data/dead_letter_queue"
[2021-03-03T08:26:17,563][DEBUG][logstash.runner ] path.settings: "C:/Users/myuser/Desktop/ELKSetup/logstash/config"
[2021-03-03T08:26:17,564][DEBUG][logstash.runner ] path.logs: "C:/Users/myuser/Desktop/ELKSetup/logstash/logs"
[2021-03-03T08:26:17,569][DEBUG][logstash.runner ] xpack.management.enabled: false
[2021-03-03T08:26:17,589][DEBUG][logstash.runner ] *xpack.monitoring.elasticsearch.username: "elastic" (default: "logstash_system")
[2021-03-03T08:26:17,591][DEBUG][logstash.runner ] *xpack.monitoring.elasticsearch.password: "mypassword"
[2021-03-03T08:26:17,612][DEBUG][logstash.runner ] --------------- Logstash Settings -------------------
[2021-03-03T08:26:17,652][DEBUG][logstash.config.source.multilocal] Reading pipeline configurations from YAML {:location=>"C:/Users/myuser/Desktop/ELKSetup/logstash/config/pipelines.yml"}
[2021-03-03T08:26:17,738][DEBUG][logstash.agent ] Setting up metric collection
[2021-03-03T08:26:17,790][DEBUG][logstash.instrument.periodicpoller.os] Starting {:polling_interval=>5, :polling_timeout=>120}
[2021-03-03T08:26:17,800][DEBUG][logstash.instrument.periodicpoller.cgroup] One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[2021-03-03T08:26:18,030][DEBUG][logstash.instrument.periodicpoller.jvm] Starting {:polling_interval=>5, :polling_timeout=>120}
[2021-03-03T08:26:18,141][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
[2021-03-03T08:26:18,147][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
[2021-03-03T08:26:18,161][DEBUG][logstash.instrument.periodicpoller.persistentqueue] Starting {:polling_interval=>5, :polling_timeout=>120}
[2021-03-03T08:26:18,175][DEBUG][logstash.instrument.periodicpoller.deadletterqueue] Starting {:polling_interval=>5, :polling_timeout=>120}
[2021-03-03T08:26:18,232][DEBUG][logstash.agent ] Starting agent
[2021-03-03T08:26:18,279][DEBUG][logstash.config.source.multilocal] Reading pipeline configurations from YAML {:location=>"C:/Users/myuser/Desktop/ELKSetup/logstash/config/pipelines.yml"}
[2021-03-03T08:26:18,337][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2021-03-03T08:26:18,362][DEBUG][logstash.agent ] Converging pipelines state {:actions_count=>0}
[2021-03-03T08:26:18,388][DEBUG][logstash.agent ] Starting puma
[2021-03-03T08:26:18,402][DEBUG][logstash.instrument.periodicpoller.os] Stopping
[2021-03-03T08:26:18,411][DEBUG][logstash.agent ] Trying to start WebServer {:port=>9600}
[2021-03-03T08:26:18,413][DEBUG][logstash.instrument.periodicpoller.jvm] Stopping
[2021-03-03T08:26:18,415][DEBUG][logstash.instrument.periodicpoller.persistentqueue] Stopping
[2021-03-03T08:26:18,417][DEBUG][logstash.instrument.periodicpoller.deadletterqueue] Stopping
[2021-03-03T08:26:18,425][DEBUG][logstash.agent ] Shutting down all pipelines {:pipelines_count=>0}
[2021-03-03T08:26:18,431][DEBUG][logstash.agent ] Converging pipelines state {:actions_count=>0}
[2021-03-03T08:26:18,462][DEBUG][logstash.api.service ] [api-service] start
[2021-03-03T08:26:18,693][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2021-03-03T08:26:23,617][INFO ][logstash.runner ] Logstash shut down.
[2021-03-03T08:26:23,634][DEBUG][logstash.agent ] Error in reactor loop escaped: closed stream (IOError)
[2021-03-03T08:26:23,636][DEBUG][logstash.agent ] 2021-03-03 08:26:23 UTC: Listen loop error: #<IOError: closed stream>
org/jruby/RubyIO.java:3552:in `select'
C:/Users/myuser/Desktop/ELKSetup/logstash/vendor/bundle/jruby/2.5.0/gems/puma-4.3.7-java/lib/puma/server.rb:383:in `handle_servers'
C:/Users/myuser/Desktop/ELKSetup/logstash/vendor/bundle/jruby/2.5.0/gems/puma-4.3.7-java/lib/puma/server.rb:356:in `block in run'
[2021-03-03T08:26:23,629][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.13.0.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.13.0.jar:?]
at C_3a_.Users.myuser.Desktop.ELKSetup.logstash.lib.bootstrap.environment.<main>(C:\Users\myuser\Desktop\ELKSetup\logstash\lib\bootstrap\environment.rb:89) ~[?:?]
C:\Users\myuser\Desktop\ELKSetup\logstash\bin>
Note: I removed some content above due to post character limit.
And pipelines.yml is:
- pipeline.id: ap
path.config: "C:/Users/myuser/Desktop/ELKSetup/logstash/config/pipeline/ap2.conf"
pipeline.workers: 1
queue.type: memory
my pipelines.yml file present in config directory and ap2.conf is in config/pipeline directory.
It is working if I try to run directly conf file with below command:
logstash -f C:\Users\myuser\Desktop\ELKSetup\logstash\config\pipeline\ap2.conf
But I want to setup pipeline for multiple conf file.
Thanks in advance