Logstash.runner shuts down automatically after the creation of API endpoint

Please look into the output:

bash-4.2$ /usr/share/logstash/bin/logstash -f /usr/share/logstash/test.conf --path.data /usr/share/logstash/df_dev
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.2.9.0.jar) to method sun.nio.ch.NativeThread.signal(long)
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
[2020-06-01T10:33:51,958][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-06-01T10:33:51,975][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.6.2"}
[2020-06-01T10:33:53,577][INFO ][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=>{:removed=>, :added=>[http://elastic:xxxxxx@elasticsearch:9200/]}}
[2020-06-01T10:33:53,921][WARN ][logstash.licensechecker.licensereader] Restored connection to ES instance {:url=>"http://elastic:xxxxxx@elasticsearch:9200/"}
[2020-06-01T10:33:53,988][INFO ][logstash.licensechecker.licensereader] ES Output version determined {:es_version=>7}
[2020-06-01T10:33:53,992][WARN ][logstash.licensechecker.licensereader] Detected a 6.x and above cluster: the type event field won't be used to determine the document _type {:es_version=>7}
[2020-06-01T10:33:54,926][INFO ][logstash.monitoring.internalpipelinesource] Monitoring License OK
[2020-06-01T10:33:54,929][INFO ][logstash.monitoring.internalpipelinesource] Validated license for monitoring. Enabling monitoring pipeline.
[2020-06-01T10:33:57,163][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \t\r\n], "#", "=>" at line 29, column 17 (byte 784) after filter {\n if [path] =~ "access" {\n mutate { replace => { "type" => "apache_access" } }\n grok {\n match => { "message" => "%{COMBINEDAPACHELOG}" }\n }\n }\n mutate {\n\t\tadd_field => { "server_name" => "dbtpa05p.ch3.dev.i.com" }\n\t}\n\tdate {\n\t\tmatch => [ "log_timestamp" , "yyyy-MM-dd HH:mm:ss,SSS" ]\n\t\ttarget => "@timestamp"\n\t}\n\noutput {\n elasticsearch ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:41:in compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:49:in compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in block in compile_sources'", "org/jruby/RubyArray.java:2580:in map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:10:in compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:161:in initialize'", "org/logstash/execution/JavaBasePipelineExt.java:47:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:27:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:36:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:326:in block in converge_state'"]}
[2020-06-01T10:33:57,822][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \t\r\n], "#", "=>" at line 29, column 17 (byte 784) after filter {\n if [path] =~ "access" {\n mutate { replace => { "type" => "apache_access" } }\n grok {\n match => { "message" => "%{COMBINEDAPACHELOG}" }\n }\n }\n mutate {\n\t\tadd_field => { "server_name" => "dbtpa05p.ch3.dev.i.com" }\n\t}\n\tdate {\n\t\tmatch => [ "log_timestamp" , "yyyy-MM-dd HH:mm:ss,SSS" ]\n\t\ttarget => "@timestamp"\n\t}\n\noutput {\n elasticsearch ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:41:in compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:49:in compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in block in compile_sources'", "org/jruby/RubyArray.java:2580:in map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:10:in compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:161:in initialize'", "org/logstash/execution/JavaBasePipelineExt.java:47:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:27:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:36:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:326:in block in converge_state'"]}
[2020-06-01T10:33:58,986][INFO ][org.reflections.Reflections] Reflections took 60 ms to scan 1 urls, producing 20 keys and 40 values
[2020-06-01T10:33:59,167][WARN ][logstash.outputs.elasticsearch] You are using a deprecated config setting "document_type" set in elasticsearch. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Document types are being deprecated in Elasticsearch 6.0, and removed entirely in 7.0. You should avoid this feature If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"document_type", :plugin=><LogStash::Outputs::ElasticSearch bulk_path=>"/_monitoring/bulk?system_id=logstash&system_api_version=7&interval=1s", password=>, hosts=>[http://elasticsearch:9200], sniffing=>false, manage_template=>false, id=>"d4cbb174a55a39717fcf13b5d81caa4604344cceb4833625c192ed4585aa486c", user=>"elastic", document_type=>"%{[@metadata][document_type]}", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_58eb4b1d-2a1e-49b2-a64e-0fcd4d1b06d4", enable_metric=>true, charset=>"UTF-8">, workers=>1, template_name=>"logstash", template_overwrite=>false, doc_as_upsert=>false, script_type=>"inline", script_lang=>"painless", script_var_name=>"event", scripted_upsert=>false, retry_initial_interval=>2, retry_max_interval=>64, retry_on_conflict=>1, ilm_enabled=>"auto", ilm_rollover_alias=>"logstash", ilm_pattern=>"{now/d}-000001", ilm_policy=>"logstash-policy", action=>"index", ssl_certificate_verification=>true, sniffing_delay=>5, timeout=>60, pool_max=>1000, pool_max_per_route=>100, resurrect_delay=>5, validate_after_inactivity=>10000, http_compression=>false>}
[2020-06-01T10:33:59,331][INFO ][logstash.outputs.elasticsearch][.monitoring-logstash] Elasticsearch pool URLs updated {:changes=>{:removed=>, :added=>[http://elastic:xxxxxx@elasticsearch:9200/]}}
[2020-06-01T10:33:59,379][WARN ][logstash.outputs.elasticsearch][.monitoring-logstash] Restored connection to ES instance {:url=>"http://elastic:xxxxxx@elasticsearch:9200/"}
[2020-06-01T10:33:59,392][INFO ][logstash.outputs.elasticsearch][.monitoring-logstash] ES Output version determined {:es_version=>7}
[2020-06-01T10:33:59,392][WARN ][logstash.outputs.elasticsearch][.monitoring-logstash] Detected a 6.x and above cluster: the type event field won't be used to determine the document _type {:es_version=>7}
[2020-06-01T10:33:59,542][INFO ][logstash.outputs.elasticsearch][.monitoring-logstash] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["http://elasticsearch:9200"]}
[2020-06-01T10:33:59,684][INFO ][logstash.javapipeline ][.monitoring-logstash] Starting pipeline {:pipeline_id=>".monitoring-logstash", "pipeline.workers"=>1, "pipeline.batch.size"=>2, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>2, "pipeline.sources"=>["monitoring pipeline"], :thread=>"#<Thread:0x27756e4d run>"}
[2020-06-01T10:34:02,389][INFO ][logstash.javapipeline ][.monitoring-logstash] Pipeline started {"pipeline.id"=>".monitoring-logstash"}
[2020-06-01T10:34:02,905][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9601}
[2020-06-01T10:34:03,701][INFO ][logstash.javapipeline ] Pipeline terminated {"pipeline.id"=>".monitoring-logstash"}
[2020-06-01T10:34:04,637][INFO ][logstash.runner ] Logstash shut down.


and my logstash.yml config is


Default Logstash configuration from Logstash base image.

https://github.com/elastic/logstash/blob/master/docker/data/logstash/config/logstash-full.yml

http.host: "0.0.0.0"
xpack.monitoring.elasticsearch.hosts: [ "http://elasticsearch:9200" ]

X-Pack security credentials

xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: elastic
xpack.monitoring.elasticsearch.password: changeme

Any comments will be more helpful...

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.