Hey Everyone,
I am currently trying to use the elasticsearch filter plugin for logstash to do some calculations/aggregations and it seems to be having issues connecting to our elasticsearch node.
based on the documentation here for Elasticsearch filter plugin for 6.8 https://www.elastic.co/guide/en/logstash/6.8/plugins-filters-elasticsearch.html it indicates that the only authentication configuration settings areuser
, password
and ca_file
which i have passed the correct information for all parameters in my configuration below
elasticsearch {
index => "<index>"
user => "<elastic_user>"
password => "<elastic_password>"
hosts => ["https://<elasticsearch_node_ip>:9200"]
ca_file => '/path/to/ca_cert/elk_ca.cert.pem'
query => "metric_type:metric AND file_name:%{[file_name]}"
fields => {
"pickup_time" => "pickup_time"
}
}
But i am getting the following error when logstash attemtps to start up with the above pipeline cofigurations
[2020-11-30T21:31:07,918][ERROR][logstash.pipeline ] Pipeline aborted due to error {:pipeline_id=>"input_json_mt_metrics", :exception=>#<Manticore::ClientProtocolException: Received fatal alert: bad_certificate>
Which is strange because i am referencing the same ca_cert in the elasticsearch output plugin which works fine
output {
elasticsearch {
index => "<index>"
user => "<elastic_user>"
password => "<elastic_password>"
hosts => ["https://<elasticsearch_node_ip>:9200"]
ssl => true
ssl_certificate_verification => true
cacert => '/path/to/ca_cert/elk_ca.cert.pem'
keystore => '/path/to/keyfile/logstash.p12'
keystore_password => "<elastic_password>"
}
}
So Im not sure why it the ca file is accepted in the elasticserach output plugin but not the elasticsearch filter plugin.
On my elasticnode these are the security setting in my elasticsearch.yml
file
xpack.security.enabled: true
xpack.security.fips_mode.enabled: true
xpack.security.authc.accept_default_password: false
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.key: /path/to/key/file.key.rsa.pem
xpack.security.transport.ssl.certificate: /path/to/cert/file.combined.pem
xpack.security.transport.ssl.certificate_authorities: ["/path/to/ca_cert/elk_ca.cert.pem"]
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.verification_mode: certificate
xpack.security.http.ssl.client_authentication: required
xpack.security.http.ssl.key: /path/to/key/file.key.rsa.pem
xpack.security.http.ssl.certificate: /path/to/cert/file.combined.pem
xpack.security.http.ssl.certificate_authorities: ["/path/to/ca_cert/elk_ca.cert.pem"]
xpack.security.authc.realms.native.type: native
xpack.security.authc.realms.native.order: 0
xpack.security.authc.realms.pki1.type: pki
xpack.security.authc.realms.pki1.order: 1
xpack.security.authc.password_hashing.algorithm: PBKDF2
any ideas?