Hi All ,
This is the first time I am building up an ELK stack for my customer . After enabling minimal , basic and basic+HTTPs security for Elastic Stack , I am unable to run pipelines due to certification related issues . I tried all the steps and even solved blog methods but still in a soup 
.
Need urgent help to solve this . Any help will be really appreciated .
Getting the following error ::
<Manticore::UnknownException: Host name 'bld20690001' does not match the certificate subject provided by the peer (CN=bld20690001)
Logstash conf::
input {
tcp {
host => "bld20690004"
port => 61002
type => "syslog"
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
target => "syslog_timestamp"
}
}
}
output{
elasticsearch{
hosts => ["https://bld20690001:61000"]
ssl => true
cacert => "/etc/logstash/config/certs/elastic-certificates-ca_1.pem"
user => "elastic"
password => "security9"
#index => "syslog-received-on-tcp"
#index => "logstashtest-%{+YYYY.MM.dd}"
}
stdout {
codec => "rubydebug"
}
}
Logstash.yml -->
X-Pack Monitoring
Monitoring Logstash with APIs | Logstash Reference [7.13] | Elastic
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.elasticsearch.password: xxxxxxxx
#xpack.monitoring.elasticsearch.proxy: ["http://proxy:port"]
xpack.monitoring.elasticsearch.hosts: ["https://bld20690001:61000"]
an alternative to hosts + username/password settings is to use cloud_id/cloud_auth
#xpack.monitoring.elasticsearch.cloud_id: monitoring_cluster_id:xxxxxxxxxx
#xpack.monitoring.elasticsearch.cloud_auth: logstash_system:password
another authentication alternative is to use an Elasticsearch API key
#xpack.monitoring.elasticsearch.api_key: "id:api_key"
xpack.monitoring.elasticsearch.ssl.certificate_authority: /etc/logstash/config/certs/elastic-certificates-ca_1.pem
#xpack.monitoring.elasticsearch.ssl.truststore.path: path/to/file
#xpack.monitoring.elasticsearch.ssl.truststore.password: password
#xpack.monitoring.elasticsearch.ssl.keystore.path: /path/to/file
#xpack.monitoring.elasticsearch.ssl.keystore.password: password
#xpack.monitoring.elasticsearch.ssl.verification_mode: certificate
#xpack.monitoring.elasticsearch.sniffing: false
#xpack.monitoring.collection.interval: 10s
#xpack.monitoring.collection.pipeline.details.enabled: true
Also note I have tried to created pem files from following ::
- Tried with pem file created from elastic-certificates.p12 generated from Basic security
- Tried with pem file created from http.12 (from basic+https step) :: elasticsearch-ca.pem
- Tried with pem file created from (./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 --ip 10.50.139.2,10.50.139.3,10.50.139.4,10.50.139.5,10.50.139.6,10.50.139.8,10.50.139.9 --dns bld20690001,bld20690002,bld20690003,bld20690004,bld20690005,bld20690007,bld20690008) :: elastic-certificates-ca_1.pem
None worked . I may be missing some integral part .
PFB detailed log file ::
Using bundled JDK: /usr/share/logstash/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2021-07-10 09:24:47.390 [main] runner - Starting Logstash {"logstash.version"=>"7.13.2", "jruby.version"=>"jruby 9.2.16.0 (2.5.7) 2021-03-03 f82228dc32 OpenJDK 64-Bit Server VM 11.0.11+9 on 11.0.11+9 +indy +jit [linux-x86_64]"}
[WARN ] 2021-07-10 09:24:47.595 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2021-07-10 09:24:48.091 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2021-07-10 09:24:48.740 [Converge PipelineAction::Create] Reflections - Reflections took 29 ms to scan 1 urls, producing 24 keys and 48 values
[WARN ] 2021-07-10 09:24:49.565 [Converge PipelineAction::Create] elasticsearch - Relying on default value of pipeline.ecs_compatibility, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[INFO ] 2021-07-10 09:24:49.731 [[main]-pipeline-manager] elasticsearch - New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["https://bld20690001:61000"]}
[INFO ] 2021-07-10 09:24:49.988 [[main]-pipeline-manager] elasticsearch - Elasticsearch pool URLs updated {:changes=>{:removed=>, :added=>[https://elastic:xxxxxx@bld20690001:61000/]}}
[ERROR] 2021-07-10 09:24:50.220 [[main]-pipeline-manager] javapipeline - Pipeline error {:pipeline_id=>"main", :exception=>#<Manticore::UnknownException: Host name 'bld20690001' does not match the certificate subject provided by the peer (CN=bld20690001)>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.7.0-java/lib/manticore/response.rb:37:in block in initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.7.0-java/lib/manticore/response.rb:79:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb:74:in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:306:in perform_request_to_url'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:258:in health_check_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:265:in block in healthcheck!'", "org/jruby/RubyHash.java:1415:in each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:263:in healthcheck!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:354:in update_urls'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:86:in update_initial_urls'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:80:in start'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client.rb:340:in build_pool'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client.rb:62:in initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:106:in create_http_client'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:102:in build'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/plugin_mixins/elasticsearch/common.rb:34:in build_client'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch.rb:275:in register'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:131:in register'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:68:in register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:228:in block in register_plugins'", "org/jruby/RubyArray.java:1809:in each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:227:in register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:585:in maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:240:in start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:185:in run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:137:in block in start'"], "pipeline.sources"=>["/etc/logstash/conf.d/logstash_syslog_test.conf"], :thread=>"#<Thread:0x6210c6e3 run>"}
[INFO ] 2021-07-10 09:24:50.222 [[main]-pipeline-manager] javapipeline - Pipeline terminated {"pipeline.id"=>"main"}
[ERROR] 2021-07-10 09:24:50.237 [Converge PipelineAction::Create] agent - Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create, action_result: false", :backtrace=>nil}
[INFO ] 2021-07-10 09:24:50.300 [LogStash::Runner] runner - Logstash shut down.