My Logstash Docker container cannot start normally, please help me analyze it

I deployed ELK in the Docker container (version 8.2.3), but the Docker container wouldn't start when logStash was configured. The log error is as follows:

{"log":"[2022-07-04T03:29:27,007][INFO ][logstash.javapipeline    ][main] Pipeline terminated {\"pipeline.id\"=\u003e\"main\"}\n","stream":"stdout","time":"2022-07-04T03:29:27.007429671Z"}
{"log":"[2022-07-04T03:29:27,029][ERROR][logstash.agent           ] Failed to execute action {:id=\u003e:main, :action_type=\u003eLogStash::ConvergeResult::FailedAction, :message=\u003e\"Could not execute action: PipelineAction::Create\u003cmain\u003e, action_result: false\", :backtrace=\u003enil}\n","stream":"stdout","time":"2022-07-04T03:29:27.030389881Z"}
{"log":"[2022-07-04T03:29:27,124][INFO ][logstash.runner          ] Logstash shut down.\n","stream":"stdout","time":"2022-07-04T03:29:27.124547303Z"}
{"log":"[2022-07-04T03:29:27,134][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit\n","stream":"stdout","time":"2022-07-04T03:29:27.138519297Z"}
{"log":"org.jruby.exceptions.SystemExit: (SystemExit) exit\n","stream":"stdout","time":"2022-07-04T03:29:27.138539127Z"}
{"log":"\u0009at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby.jar:?]\n","stream":"stdout","time":"2022-07-04T03:29:27.138544184Z"}
{"log":"\u0009at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby.jar:?]\n","stream":"stdout","time":"2022-07-04T03:29:27.138548486Z"}
{"log":"\u0009at usr.share.logstash.lib.bootstrap.environment.\u003cmain\u003e(/usr/share/logstash/lib/bootstrap/environment.rb:91) ~[?:?]\n","stream":"stdout","time":"2022-07-04T03:29:27.138552629Z"}
{"log":"Using bundled JDK: /usr/share/logstash/jdk\n","stream":"stdout","time":"2022-07-04T03:35:31.426388951Z"}
{"log":"OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.\n","stream":"stderr","time":"2022-07-04T03:35:31.657103557Z"}
{"log":"Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties\n","stream":"stdout","time":"2022-07-04T03:35:47.895522636Z"}
{"log":"[2022-07-04T03:35:47,943][INFO ][logstash.runner          ] Log4j configuration path used is: /usr/share/logstash/config/log4j2.properties\n","stream":"stdout","time":"2022-07-04T03:35:47.946292892Z"}
{"log":"[2022-07-04T03:35:47,953][INFO ][logstash.runner          ] Starting Logstash {\"logstash.version\"=\u003e\"8.2.3\", \"jruby.version\"=\u003e\"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.15+10 on 11.0.15+10 +indy +jit [linux-x86_64]\"}\n","stream":"stdout","time":"2022-07-04T03:35:47.954056736Z"}
{"log":"[2022-07-04T03:35:47,955][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Dls.cgroup.cpuacct.path.override=/, -Dls.cgroup.cpu.path.override=/, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]\n","stream":"stdout","time":"2022-07-04T03:35:47.955814187Z"}
{"log":"[2022-07-04T03:35:49,227][WARN ][deprecation.logstash.monitoringextension.pipelineregisterhook] Internal collectors option for Logstash monitoring is deprecated and targeted for removal in the next major version.\n","stream":"stdout","time":"2022-07-04T03:35:49.228059884Z"}
{"log":"Please configure Metricbeat to monitor Logstash. Documentation can be found at: \n","stream":"stdout","time":"2022-07-04T03:35:49.228108756Z"}
{"log":"https://www.elastic.co/guide/en/logstash/current/monitoring-with-metricbeat.html\n","stream":"stdout","time":"2022-07-04T03:35:49.228113669Z"}
{"log":"[2022-07-04T03:35:49,899][INFO ][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=\u003e{:removed=\u003e[], :added=\u003e[http://logstash_internal:xxxxxx@es01:9200/]}}\n","stream":"stdout","time":"2022-07-04T03:35:49.905232271Z"}
{"log":"[2022-07-04T03:35:50,080][INFO ][logstash.licensechecker.licensereader] Failed to perform request {:message=\u003e\"es01:9200 failed to respond\", :exception=\u003eManticore::ClientProtocolException, :cause=\u003eorg.apache.http.NoHttpResponseException: es01:9200 failed to respond}\n","stream":"stdout","time":"2022-07-04T03:35:50.081428267Z"}
{"log":"[2022-07-04T03:35:50,086][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error {:url=\u003e\"http://logstash_internal:xxxxxx@es01:9200/\", :exception=\u003eLogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :message=\u003e\"Elasticsearch Unreachable: [http://es01:9200/][Manticore::ClientProtocolException] es01:9200 failed to respond\"}\n","stream":"stdout","time":"2022-07-04T03:35:50.087194568Z"}
{"log":"[2022-07-04T03:35:50,117][INFO ][logstash.licensechecker.licensereader] Failed to perform request {:message=\u003e\"es01:9200 failed to respond\", :exception=\u003eManticore::ClientProtocolException, :cause=\u003eorg.apache.http.NoHttpResponseException: es01:9200 failed to respond}\n","stream":"stdout","time":"2022-07-04T03:35:50.1180135Z"}
{"log":"[2022-07-04T03:35:50,123][WARN ][logstash.licensechecker.licensereader] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [http://es01:9200/_xpack][Manticore::ClientProtocolException] es01:9200 failed to respond {:url=\u003ehttp://logstash_internal:xxxxxx@es01:9200/, :error_message=\u003e\"Elasticsearch Unreachable: [http://es01:9200/_xpack][Manticore::ClientProtocolException] es01:9200 failed to respond\", :error_class=\u003e\"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError\"}\n","stream":"stdout","time":"2022-07-04T03:35:50.123692086Z"}
{"log":"[2022-07-04T03:35:50,131][WARN ][logstash.licensechecker.licensereader] Attempt to validate Elasticsearch license failed. Sleeping for 0.02 {:fail_count=\u003e1, :exception=\u003e\"Elasticsearch Unreachable: [http://es01:9200/_xpack][Manticore::ClientProtocolException] es01:9200 failed to respond\"}\n","stream":"stdout","time":"2022-07-04T03:35:50.13224101Z"}
{"log":"[2022-07-04T03:35:50,156][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=\u003e\"No Available connections\"}\n","stream":"stdout","time":"2022-07-04T03:35:50.156689578Z"}
{"log":"[2022-07-04T03:35:50,216][ERROR][logstash.monitoring.internalpipelinesource] Failed to fetch X-Pack information from Elasticsearch. This is likely due to failure to reach a live Elasticsearch cluster.\n","stream":"stdout","time":"2022-07-04T03:35:50.216553739Z"}
{"log":"[2022-07-04T03:35:50,357][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=\u003e9600, :ssl_enabled=\u003efalse}\n","stream":"stdout","time":"2022-07-04T03:35:50.358481305Z"}
{"log":"[2022-07-04T03:35:51,406][INFO ][org.reflections.Reflections] Reflections took 89 ms to scan 1 urls, producing 120 keys and 395 values \n","stream":"stdout","time":"2022-07-04T03:35:51.406929913Z"}
{"log":"[2022-07-04T03:35:51,629][INFO ][logstash.javapipeline    ] Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility =\u003e v8` unless explicitly configured otherwise.\n","stream":"stdout","time":"2022-07-04T03:35:51.629685214Z"}
{"log":"[2022-07-04T03:35:51,674][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=\u003e\"LogStash::Outputs::ElasticSearch\", :hosts=\u003e[\"//es01:9200\"]}\n","stream":"stdout","time":"2022-07-04T03:35:51.675209405Z"}
{"log":"[2022-07-04T03:35:51,720][ERROR][logstash.javapipeline    ][main] Pipeline error {:pipeline_id=\u003e\"main\", :exception=\u003e#\u003cErrno::EACCES: Permission denied - ./config/certs/http_ca.crt\u003e, :backtrace=\u003e[\"org/jruby/RubyIO.java:1237:in `sysopen'\", \"org/jruby/RubyFile.java:365:in `initialize'\", \"org/jruby/RubyIO.java:1156:in `open'\", \"org/jruby/RubyKernel.java:317:in `open'\", \"/usr/share/logstash/vendor/jruby/lib/ruby/stdlib/open-uri.rb:37:in `open'\", \"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.8.0-java/lib/manticore/client.rb:645:in `setup_trust_store'\", \"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.8.0-java/lib/manticore/client.rb:633:in `ssl_socket_factory_from_options'\", \"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.8.0-java/lib/manticore/client.rb:397:in `pool_builder'\", \"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.8.0-java/lib/manticore/client.rb:405:in `pool'\", \"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.8.0-java/lib/manticore/client.rb:208:in `initialize'\", \"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb:26:in `initialize'\", \"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:325:in `build_adapter'\", \"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:341:in `build_pool'\", \"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:63:in `initialize'\", \"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:106:in `create_http_client'\", \"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:102:in `build'\", \"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/plugin_mixins/elasticsearch/common.rb:34:in `build_client'\", \"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch.rb:279:in `register'\", \"org/logstash/config/ir/compiler/OutputStrategyExt.java:131:in `register'\", \"org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:68:in `register'\", \"/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:233:in `block in register_plugins'\", \"org/jruby/RubyArray.java:1821:in `each'\", \"/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:232:in `register_plugins'\", \"/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:598:in `maybe_setup_out_plugins'\", \"/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:245:in `start_workers'\", \"/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:190:in `run'\", \"/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:142:in `block in start'\"], \"pipeline.sources\"=\u003e[\"/usr/share/logstash/pipeline/logstash.conf\"], :thread=\u003e\"#\u003cThread:0x54640975 run\u003e\"}\n","stream":"stdout","time":"2022-07-04T03:35:51.721893372Z"}
{"log":"[2022-07-04T03:35:51,723][INFO ][logstash.javapipeline    ][main] Pipeline terminated {\"pipeline.id\"=\u003e\"main\"}\n","stream":"stdout","time":"2022-07-04T03:35:51.723863057Z"}
{"log":"[2022-07-04T03:35:51,745][ERROR][logstash.agent           ] Failed to execute action {:id=\u003e:main, :action_type=\u003eLogStash::ConvergeResult::FailedAction, :message=\u003e\"Could not execute action: PipelineAction::Create\u003cmain\u003e, action_result: false\", :backtrace=\u003enil}\n","stream":"stdout","time":"2022-07-04T03:35:51.746298212Z"}
{"log":"[2022-07-04T03:35:51,840][INFO ][logstash.runner          ] Logstash shut down.\n","stream":"stdout","time":"2022-07-04T03:35:51.841037752Z"}
{"log":"[2022-07-04T03:35:51,853][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit\n","stream":"stdout","time":"2022-07-04T03:35:51.857054538Z"}
{"log":"org.jruby.exceptions.SystemExit: (SystemExit) exit\n","stream":"stdout","time":"2022-07-04T03:35:51.857072751Z"}
{"log":"\u0009at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby.jar:?]\n","stream":"stdout","time":"2022-07-04T03:35:51.857077012Z"}
{"log":"\u0009at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby.jar:?]\n","stream":"stdout","time":"2022-07-04T03:35:51.857080741Z"}
{"log":"\u0009at usr.share.logstash.lib.bootstrap.environment.\u003cmain\u003e(/usr/share/logstash/lib/bootstrap/environment.rb:91) ~[?:?]\n","stream":"stdout","time":"2022-07-04T03:35:51.857084054Z"

My logstash. Yml configuration is as follows:

http.host: "0.0.0.0"

#Elasticsearch address
xpack.monitoring.elasticsearch.hosts: [ "http://es01:9200" ]
xpack.monitoring.enabled: true

My logstash. conf configuration is as follows:

input {
        tcp {
                port => "520"
                codec => plain {
                        charset => "GB2312"
                }
                add_field => {
                        "device_name" => "WINDOWS-QIAOLI"
                }
        }
}

output {
        if [device_name] == "WINDOWS-QIAOLI" {
                elasticsearch {
                        hosts => ["es01:9200"]
                        index => "logstash_syslog-%{+YYYY.MM.dd}"
                        cacert => "./config/certs/http_ca.crt"
                        user => "logstash_internal"
                        password => "xxxxx"
                        ssl => true
                        codec => plain {
                                format => "%{message}"
                                charset => "UTF-8"
                        }
                }
        }
}

My Docker network configuration

[
    {
        "Name": "elastic",
        "Id": "580aa597c94f4197ee3fab04f48fcc5355c937ab59cb79dc45491dc002d81d9d",
        "Created": "2022-06-16T13:31:01.250976085+08:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.19.0.0/16",
                    "Gateway": "172.19.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "e352a327cb1a8d6fccb3af2dd5e9df1209b89b122b0bb6043992b3d100810827": {
                "Name": "kib-01",
                "EndpointID": "9754a033b4fc39d3e8be424695b61a6655958b384bdc66102481e889af0acb27",
                "MacAddress": "02:42:ac:13:00:03",
                "IPv4Address": "172.19.0.3/16",
                "IPv6Address": ""
            },
            "f8e3747258fa3fdb2895876251ea46db5f56048853aeced877fe0e71e07888d9": {
                "Name": "es01",
                "EndpointID": "31225f0bd73c1b33002e0e0c5a4075c51d1eed0378d53677f24c0e2dab4a86fc",
                "MacAddress": "02:42:ac:13:00:02",
                "IPv4Address": "172.19.0.2/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {}
    }
]

[quote][ERROR][logstash.javapipeline ][main] Pipeline error {
:pipeline_id=>"main", :exception=>#<Errno::EACCES: Permission denied - ./config/certs/http_ca.crt>,
:backtrace=>["org/jruby/RubyIO.java:1237:in `sysopen'", "org/jruby/RubyFile.java:365:in `initialize'",
"org/jruby/RubyIO.java:1156:in `open'"[/quote]

The output is unable to open the cert.

This error disappeared after I added readable permissions to the certificate

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.