Hello,
logstash 8.2.2Preformatted text for windows failed to run, i have the following output.
Could someone please help
Regards
C:\WINDOWS\system32>logstash -f logstash.conf
"Using bundled JDK: D:\Technical-Docs\Audits\Audit-Security\Tools-2022\Elastic\logstash-8.2.2\jdk\bin\java.exe"
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Sending Logstash logs to D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logs which is now configured via log4j2.properties
[2022-06-14T09:23:15,219][INFO ][logstash.runner ] Log4j configuration path used is: D:\Technical-Docs\Audits\Audit-Security\Tools-2022\Elastic\logstash-8.2.2\config\log4j2.properties
[2022-06-14T09:23:15,227][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"8.2.2", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.14.1+1 on 11.0.14.1+1 +indy +jit [mswin32-x86_64]"}
[2022-06-14T09:23:15,228][INFO ][logstash.runner ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
[2022-06-14T09:23:15,308][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2022-06-14T09:23:16,967][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2022-06-14T09:23:17,233][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [A-Za-z0-9_-], [ \\t\\r\\n], \"#\", \"{\", [A-Za-z0-9_], \",\", \"]\" at line 7, column 23 (byte 76) after output {\r\n elasticsearch {\r\n hosts => [localhost", :backtrace=>["D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:189:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:72:in `initialize'", "D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logstash-core/lib/logstash/java_pipeline.rb:48:in `initialize'", "D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logstash-core/lib/logstash/pipeline_action/create.rb:50:in `execute'", "D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logstash-core/lib/logstash/agent.rb:381:in `block in converge_state'"]}
[2022-06-14T09:23:17,305][INFO ][logstash.runner ] Logstash shut down.
[2022-06-14T09:23:17,314][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby.jar:?]
at D_3a_.Technical_minus_Docs.Audits.Audit_minus_Security.Tools_minus_2022.Elastic.logstash_minus_8_dot_2_dot_2.lib.bootstrap.environment.<main>(D:\Technical-Docs\Audits\Audit-Security\Tools-2022\Elastic\logstash-8.2.2\lib\bootstrap\environment.rb:91) ~[?:?]
C:\WINDOWS\system32>logstash -f logstash.conf
"Using bundled JDK: D:\Technical-Docs\Audits\Audit-Security\Tools-2022\Elastic\logstash-8.2.2\jdk\bin\java.exe"
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Sending Logstash logs to D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logs which is now configured via log4j2.properties
[2022-06-14T09:23:15,219][INFO ][logstash.runner ] Log4j configuration path used is: D:\Technical-Docs\Audits\Audit-Security\Tools-2022\Elastic\logstash-8.2.2\config\log4j2.properties
[2022-06-14T09:23:15,227][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"8.2.2", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.14.1+1 on 11.0.14.1+1 +indy +jit [mswin32-x86_64]"}
[2022-06-14T09:23:15,228][INFO ][logstash.runner ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
[2022-06-14T09:23:15,308][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2022-06-14T09:23:16,967][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2022-06-14T09:23:17,233][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [A-Za-z0-9_-], [ \\t\\r\\n], \"#\", \"{\", [A-Za-z0-9_], \",\", \"]\" at line 7, column 23 (byte 76) after output {\r\n elasticsearch {\r\n hosts => [localhost", :backtrace=>["D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:189:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:72:in `initialize'", "D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logstash-core/lib/logstash/java_pipeline.rb:48:in `initialize'", "D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logstash-core/lib/logstash/pipeline_action/create.rb:50:in `execute'", "D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logstash-core/lib/logstash/agent.rb:381:in `block in converge_state'"]}
[2022-06-14T09:23:17,305][INFO ][logstash.runner ] Logstash shut down.
[2022-06-14T09:23:17,314][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby.jar:?]
at D_3a_.Technical_minus_Docs.Audits.Audit_minus_Security.Tools_minus_2022.Elastic.logstash_minus_8_dot_2_dot_2.lib.bootstrap.environment.<main>(D:\Technical-Docs\Audits\Audit-Security\Tools-2022\Elastic\logstash-8.2.2\lib\bootstrap\environment.rb:91) ~[?:?]
Hello @Airborn
Welcome to elastic community 
!!!
The below error would exactly brief us out that there is issue in your configuration.
[2022-06-14T09:23:17,233][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [A-Za-z0-9_-], [ \\t\\r\\n], \"#\", \"{\", [A-Za-z0-9_], \",\", \"]\" at line 7, column 23 (byte 76) after output {\r\n elasticsearch {\r\n hosts => [localhost", :backtrace=>["D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logstash-core/lib/logstash/compiler.rb:32:in compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:189:in initialize'", "org/logstash/execution/JavaBasePipelineExt.java:72:in initialize'", "D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logstash-core/lib/logstash/java_pipeline.rb:48:in initialize'", "D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logstash-core/lib/logstash/pipeline_action/create.rb:50:in execute'", "D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logstash-core/lib/logstash/agent.rb:381:in block in converge_state'"]}
Hence, Please check out the below line in the config code
hosts => [localhost",
Example code snippet:
output
{
elasticsearch {
hosts => ["http://localhost:9200"]
index => "<Your-index-name>"
user => "<your-es-username>" #if its enabld with authentication
password => "<your-es-pwd>" #if its enabld with authentication
}
}
Keep posted with updates!!! Thanks !!!
Hi
Thanks for replying
bellow is my logstash.conf file, i don't use authentication. I'm using all downloaded files without any change and i'm using Windows 10 version 21H2
I don't have any problems with Elasticsearch and Kibana
Regards
output
{
Elasticsearch {
hosts => ["http://localhost:9200"]
index => "indexforlogstash"
}
}
Hello @Airborn
I believe the "pipeline.id: main" code block in pipelines.yml should be commented out as its not been used.
Could you check the pipeline id called "main" in pipelines.yml
bellow is the content of pipelines.yml
As i'm new, the file is the default one
Regards
List of pipelines to be loaded by Logstash
This document must be a list of dictionaries/hashes, where the keys/values are pipeline settings.
Default values for omitted settings are read from the logstash.yml file.
When declaring multiple pipelines, each MUST have its own pipeline.id.
Example of two pipelines:
- pipeline.id: test
pipeline.workers: 1
pipeline.batch.size: 1
config.string: "input { generator {} } filter { sleep { time => 1 } } output { stdout { codec => dots } }"
- pipeline.id: another_test
queue.type: persisted
path.config: "/tmp/logstash/*.config"
Available options:
# name of the pipeline
pipeline.id: mylogs
# The configuration string to be used by this pipeline
config.string: "input { generator {} } filter { sleep { time => 1 } } output { stdout { codec => dots } }"
# The path from where to read the configuration text
path.config: "/etc/conf.d/logstash/myconfig.cfg"
# How many worker threads execute the Filters+Outputs stage of the pipeline
pipeline.workers: 1 (actually defaults to number of CPUs)
# How many events to retrieve from inputs before sending to filters+workers
pipeline.batch.size: 125
# How long to wait in milliseconds while polling for the next event
# before dispatching an undersized batch to filters+outputs
pipeline.batch.delay: 50
Set the pipeline event ordering. Options are "auto" (the default), "true" # # or "false".
"auto" automatically enables ordering if the 'pipeline.workers' setting
is also set to '1', and disables otherwise.
"true" enforces ordering on a pipeline and prevents logstash from starting
a pipeline with multiple workers allocated.
"false" disable any extra processing necessary for preserving ordering.
pipeline.ordered: auto
# Internal queuing model, "memory" for legacy in-memory based queuing and
# "persisted" for disk-based acked queueing. Defaults is memory
queue.type: memory
# If using queue.type: persisted, the page data files size. The queue data consists of
# append-only data files separated into pages. Default is 64mb
queue.page_capacity: 64mb
# If using queue.type: persisted, the maximum number of unread events in the queue.
# Default is 0 (unlimited)
queue.max_events: 0
# If using queue.type: persisted, the total capacity of the queue in number of bytes.
# Default is 1024mb or 1gb
queue.max_bytes: 1024mb
# If using queue.type: persisted, the maximum number of acked events before forcing a checkpoint
# Default is 1024, 0 for unlimited
queue.checkpoint.acks: 1024
# If using queue.type: persisted, the maximum number of written events before forcing a checkpoint
# Default is 1024, 0 for unlimited
queue.checkpoint.writes: 1024
# If using queue.type: persisted, the interval in milliseconds when a checkpoint is forced on the head page
# Default is 1000, 0 for no periodic checkpoint.
queue.checkpoint.interval: 1000
# Enable Dead Letter Queueing for this pipeline.
dead_letter_queue.enable: false
If using dead_letter_queue.enable: true, the maximum size of dead letter queue for this pipeline. Entries
will be dropped if they would increase the size of the dead letter queue beyond this setting.
Default is 1024mb
dead_letter_queue.max_bytes: 1024mb
If using dead_letter_queue.enable: true, the interval in milliseconds where if no further events eligible for the DLQ
have been created, a dead letter queue file will be written. A low value here will mean that more, smaller, queue files
may be written, while a larger value will introduce more latency between items being "written" to the dead letter queue, and
being available to be read by the dead_letter_queue input when items are are written infrequently.
Default is 5000.
dead_letter_queue.flush_interval: 5000
If using dead_letter_queue.enable: true, the directory path where the data files will be stored.
Default is path.data/dead_letter_queue
path.dead_letter_queue:
Hello @Airborn
As you are not using any pipelines , i would request you to comment all the lines in "pipelines.yml" save it. Then try executing the command.
Keep Posted!!! Thanks !!!
ps: Also please change the tags on this question
just to add that all the lines in pipelines.yml file i'm using are commented (prefixed by Preformatted text # )
As i see,it seems that Preformatted text# has disappeared when i pasted in the post
Bellow is the logstash.conf file
output
{
Elasticsearch {
hosts => ["http://localhost:9200"]
index => "indexforlogstash"
}
}
Hello @Airborn ,
There is a typo in your logstash.conf file
Error: "E" in Elasticsearch
Corrected code syntax:
output
{
elasticsearch {
hosts => ["http://localhost:9200"]
index => "indexforlogstash"
}
}
if you would like to parse your input to index, you can try the below
input {
stdin {
}
}
output
{
elasticsearch {
hosts => ["http://localhost:9200"]
index => "indexforlogstash"
}
}
Keep Posted with updates !!! Thanks and Happy Stashing !!!
https://www.elastic.co/guide/en/logstash/current/configuration.html
I would also request you to mention the input plugin as it is mandatory like the output plugin.
filter plugin is optional.
I did the change and i have the same problem.
for info, i got the .conf file content from Configuring Logstash | Logstash Reference [8.2] | Elastic
what does the following error message means
[2022-06-14T12:53:07,766][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [A-Za-z0-9_-], [ \t\r\n], "#", "{", [A-Za-z0-9_], ",", "]" at line 7, column 23 (byte 76) after output {\r\n Elasticsearch {\r\n hosts => [localhost", :backtrace=>["D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logstash-core/lib/logstash/compiler.rb:28:in compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:189:in initialize'", "org/logstash/execution/JavaBasePipelineExt.java:72:in initialize'", "D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logstash-core/lib/logstash/java_pipeline.rb:48:in initialize'", "D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logstash-core/lib/logstash/pipeline_action/create.rb:50:in execute'", "D:/Technical-Docs/Audits/Audit-Security/Tools-2022/Elastic/logstash-8.2.2/logstash-core/lib/logstash/agent.rb:381:in block in converge_state'"]}
i could still see that typo error in this error message "E" in "Elasticsearch"
Please save the changes and try executing once.
You are right. now it is working
thank you so much for you valuable help and support
1 Like
Thanks a lot for your feedback.
Happy Stashing!!!
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.