Please add log.level: debug to your logstash.yml, run Logstash, and share the log
@strawgate Thanks. Please find below as requested.
# Default is path.data/dead_letter_queue
#
# path.dead_letter_queue:
#
# ------------ Debugging Settings --------------
#
# Options for log.level:
# * fatal
# * error
# * warn
# * info (default)
# * debug
# * trace
#
log.level: debug
path.logs: /var/log/logstash
#
# ------------ Other Settings --------------
#
# Where to find custom plugins
# path.plugins: []
The program still got stuck as below:
sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/sentinel.conf
Using bundled JDK: /usr/share/logstash/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2025-03-02 21:39:10.006 [main] runner - Starting Logstash {"logstash.version"=>"7.17.13", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.20+8 on 11.0.20+8 +indy +jit [linux-x86_64]"}
[INFO ] 2025-03-02 21:39:10.015 [main] runner - JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djdk.io.File.enableADS=true, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true]
[WARN ] 2025-03-02 21:39:10.349 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2025-03-02 21:39:12.377 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[INFO ] 2025-03-02 21:39:13.793 [Converge PipelineAction::Create<main>] Reflections - Reflections took 96 ms to scan 1 urls, producing 119 keys and 419 values
[WARN ] 2025-03-02 21:39:14.762 [Converge PipelineAction::Create<main>] json - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[WARN ] 2025-03-02 21:39:14.885 [Converge PipelineAction::Create<main>] file - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[WARN ] 2025-03-02 21:39:14.995 [Converge PipelineAction::Create<main>] plain - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[INFO ] 2025-03-02 21:39:15.414 [[main]-pipeline-manager] microsoftsentineloutput - Azure Loganalytics configuration was found valid.
[INFO ] 2025-03-02 21:39:15.491 [[main]-pipeline-manager] javapipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/etc/logstash/conf.d/sentinel.conf"], :thread=>"#<Thread:0x15dba87e run>"}
[INFO ] 2025-03-02 21:39:16.626 [[main]-pipeline-manager] javapipeline - Pipeline Java execution initialization time {"seconds"=>1.13}
[INFO ] 2025-03-02 21:39:16.805 [[main]-pipeline-manager] javapipeline - Pipeline started {"pipeline.id"=>"main"}
[INFO ] 2025-03-02 21:39:16.845 [[main]<file] observingtail - START, creating Discoverer, Watch with file and sincedb collections
[INFO ] 2025-03-02 21:39:16.921 [Agent thread] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
^C[WARN ] 2025-03-02 21:42:30.506 [SIGINT handler] runner - SIGINT received. Shutting down.
[INFO ] 2025-03-02 21:42:30.554 [Converge PipelineAction::StopAndDelete<main>] observingtail - QUIT - closing all files and shutting down.
[INFO ] 2025-03-02 21:42:30.745 [[main]-pipeline-manager] javapipeline - Pipeline terminated {"pipeline.id"=>"main"}
[INFO ] 2025-03-02 21:42:31.589 [Converge PipelineAction::StopAndDelete<main>] pipelinesregistry - Removed pipeline from registry successfully {:pipeline_id=>:main}
[INFO ] 2025-03-02 21:42:31.651 [LogStash::Runner] runner - Logstash shut down.
root@cgysenlfp02:~# sudo systemctl restart logstash
root@cgysenlfp02:~# sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/sentinel.conf
Using bundled JDK: /usr/share/logstash/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2025-03-02 22:29:25.844 [main] runner - Starting Logstash {"logstash.version"=>"7.17.13", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.20+8 on 11.0.20+8 +indy +jit [linux-x86_64]"}
[INFO ] 2025-03-02 22:29:25.850 [main] runner - JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djdk.io.File.enableADS=true, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true]
[WARN ] 2025-03-02 22:29:26.306 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2025-03-02 22:29:28.563 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[INFO ] 2025-03-02 22:29:29.962 [Converge PipelineAction::Create<main>] Reflections - Reflections took 89 ms to scan 1 urls, producing 119 keys and 419 values
[WARN ] 2025-03-02 22:29:31.019 [Converge PipelineAction::Create<main>] json - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[WARN ] 2025-03-02 22:29:31.081 [Converge PipelineAction::Create<main>] file - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[WARN ] 2025-03-02 22:29:31.202 [Converge PipelineAction::Create<main>] plain - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[INFO ] 2025-03-02 22:29:31.464 [[main]-pipeline-manager] microsoftsentineloutput - Azure Loganalytics configuration was found valid.
[INFO ] 2025-03-02 22:29:31.621 [[main]-pipeline-manager] javapipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/etc/logstash/conf.d/sentinel.conf"], :thread=>"#<Thread:0x358e5652 run>"}
[INFO ] 2025-03-02 22:29:32.825 [[main]-pipeline-manager] javapipeline - Pipeline Java execution initialization time {"seconds"=>1.2}
[INFO ] 2025-03-02 22:29:32.987 [[main]-pipeline-manager] javapipeline - Pipeline started {"pipeline.id"=>"main"}
[INFO ] 2025-03-02 22:29:33.031 [Agent thread] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[INFO ] 2025-03-02 22:29:33.041 [[main]<file] observingtail - START, creating Discoverer, Watch with file and sincedb collections
Thanks for the support.
You have not enabled debug logging as you have modified a logstash.yml file that is not used by the logstash instance you're invoking
This does not look good to me.
Either start the service manually, or use systemctl, doing both looks like it might be trouble.
sudo systemctl status logstash
will tell you if another Logstash instance is running.
1 Like
I gave up after trying all sorts. Looks like the new version of logstash plugin v1.4 is not compatible with existing versions of logstash.
Did you mean 1.1.4 ?
I am a bit confused over the versions of microsoft-sentinel-log-analytics-logstash-output-plugin
rubygems has a 1.1.4 at:
but the GitHub page still references 1.1.3
I don't find any release notes for 1.1.4. 1.1.3 should be compatible with the logstash version you noted above, namely "logstash.version"=>"7.17.13".
Yes, the plugin install the new version 1.1.4 by default.