Completed installation of the ELK stack version 8.4 - single node installation
Used a test pipeline to confirm comms between Elasticsearch and Logstash on local machine on the commandline interface.
So, I tried implementing the ArcSight module by both using the Logstash yml file and through the command-line using both sets of config:
Command-line
./logstash --modules arcsight --setup -M "arcsight.var.input.smartconnector.port=5000" -M "arcsight.var.elasticsearch.hosts=https://localhost:9200" -M "arcsight.var.elasticsearch.username=elastic" -M "arcsight.var.elasticsearch.password=PASSWORD" -M "arcsight.var.kibana.hosts=localhost:5601" -M "arcsight.var.kibana.username=elastic" -M "arcsight.var.kibana.password=PASSWORD" -M "arcsight.var.kibana.ssl.enabled=false" -M "arcsight.var.elasticsearch.ssl.certificate=/etc/logstash/certs/http_ca.crt"
Logstash.yml
- name: arcsight arcsight.var.input.smartconnector.port: 5000 arcsight.var.elasticsearch.hosts: ["https://localhost:9200"] arcsight.var.elasticsearch.username: "elastic" arcsight.var.elasticsearch.password: "PASSWORD" arcsight.var.kibana.host: "localhost:5601" arcsight.var.kibana.username: "elastic" arcsight.var.kibana.password: "PASSWORD" arcsight.var.elasticsearch.ssl.certificate: "/etc/logstash/certs/http_ca.crt"
Error Messages after running both command-line and logastash.yml configurations
`
Using bundled JDK: /usr/share/logstash/jdk
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2022-11-02 10:24:29.443 [main] runner - NOTICE: Running Logstash as superuser is not recommended and won't be allowed in the future. Set 'allow_superuser' to 'false' to avoid startup errors in future releases.
[INFO ] 2022-11-02 10:24:29.453 [main] runner - Starting Logstash {"logstash.version"=>"8.4.3", "jruby.version"=>"jruby 9.3.8.0 (2.6.8) 2022-09-13 98d69c9461 OpenJDK 64-Bit Server VM 17.0.4+8 on 17.0.4+8 +indy +jit [x86_64-linux]"}
[INFO ] 2022-11-02 10:24:29.455 [main] runner - JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
[WARN ] 2022-11-02 10:24:29.643 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2022-11-02 10:24:30.355 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[INFO ] 2022-11-02 10:24:30.660 [Agent thread] licensereader - Elasticsearch pool URLs updated {:changes=>{:removed=>, :added=>[https://elastic:xxxxxx@localhost:9200/]}}
[INFO ] 2022-11-02 10:24:30.832 [Agent thread] licensereader - Failed to perform request {:message=>"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target", :exception=>Manticore::ClientProtocolException, :cause=>#<Java::JavaxNetSsl::SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target>}
[WARN ] 2022-11-02 10:24:30.835 [Agent thread] licensereader - Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://elastic:xxxxxx@localhost:9200/", :exception=>LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [[https://localhost:9200/]Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"}
[INFO ] 2022-11-02 10:24:30.874 [Agent thread] licensereader - Failed to perform request {:message=>"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target", :exception=>Manticore::ClientProtocolException, :cause=>#<Java::JavaxNetSsl::SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target>}
[WARN ] 2022-11-02 10:24:30.878 [Agent thread] licensereader - Marking url as dead. Last error: [LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [[https://localhost:9200/_xpack]Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target {:url=>https://elastic:xxxxxx@localhost:9200/, :error_message=>"Elasticsearch Unreachable: [[https://localhost:9200/_xpack]Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target", :error_class=>"LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError"}
[WARN ] 2022-11-02 10:24:30.886 [Agent thread] licensereader - Attempt to validate Elasticsearch license failed. Sleeping for 0.02 {:fail_count=>1, :exception=>"Elasticsearch Unreachable: [[https://localhost:9200/_xpack]Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"}
[ERROR] 2022-11-02 10:24:30.909 [Agent thread] licensereader - Unable to retrieve license information from license server {:message=>"No Available connections"}
[ERROR] 2022-11-02 10:24:30.932 [Agent thread] modulelicensechecker - Failed to fetch X-Pack information from Elasticsearch. This is likely due to failure to reach a live Elasticsearch cluster.
[WARN ] 2022-11-02 10:24:30.932 [Agent thread] modulescommon - The arcsight module is not enabled. Please check the logs for additional information.
[ERROR] 2022-11-02 10:24:30.936 [Agent thread] sourceloader - No configuration found in the configured sources .
[INFO ] 2022-11-02 10:24:31.001 [LogStash::Runner] runner - Logstash shut down.
`
Just to confirm I was able to run a test pipeline from the /etc/logstash/conf.d without any issues so I can confirm Logstash and ES are able to contact each other securely.
Below is a confirmation that I could curl ES on HTTPS which would confirm ES is running despite the errors indicating that the instance is unreachable
curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic https://localhost:9200 Enter host password for user 'elastic': { "name" : "HOSTNAME", "cluster_name" : "elasticsearch", "cluster_uuid" : "Hpk7yGniS1S8aASBpe9ebA", "version" : { "number" : "8.4.3", "build_flavor" : "default", "build_type" : "rpm", "build_hash" : "42f05b9372a9a4a470db3b52817899b99a76ee73", "build_date" : "2022-10-04T07:17:24.662462378Z", "build_snapshot" : false, "lucene_version" : "9.3.0", "minimum_wire_compatibility_version" : "7.17.0", "minimum_index_compatibility_version" : "7.0.0" }, "tagline" : "You Know, for Search" }
Any help on how to resolve this will be highly appreciated!