Watcher with ES 2.4 shield -- SSL handshake

Elastic Stack Elasticsearch
1

Hi,
I am trying to integrate pagerduty with es watcher. My ES 2.4 has shield enabled. While watcher get and event it is not able to trigger action event.While checking status of watcher history, i get below response:

While checking history of watcher actions using below curl-

curl -u esadmin:Y291c3RlYXUK -k -XGET 'https://localhost:9200/.watch_history*/_search?pretty' -d '{
"query" : {
"bool" : {
"must" : [
{ "match" : { "result.condition.met" : true }},
{ "range" : { "result.execution_time" : { "from" : "now-20s"}}}
]
}
}
}'

I get below payload:

condition" : {
"type" : "compare",
"status" : "success",
"met" : true,
"compare" : {
"resolved_values" : {
"ctx.payload.hits.total" : 3 } } },
"actions" : [ {
"id" : "notify-pagerduty",
"type" : "pagerduty",
"status" : "failure",
"reason" : "SSLHandshakeException[sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested t
arget]; nested: ValidatorException[PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderExce
ption: unable to find valid certification path to requested target]; nested: SunCertPathBuilderException[unable to
find valid certification path to requested target]; "
} ] } } } ] } }

If i disable the shield everything works fine.
In my current elastic search yml I have provided truststore and key store as below:

watcher.http.ssl.keystore.path: /etc/elasticsearch/shield/keystore.jks
watcher.http.ssl.keystore.password: *****
watcher.http.ssl.truststore.path: /etc/elasticsearch/shield/truststore.jks
watcher.http.ssl.truststore.password: ****
watcher.http.ssl.verification_mode: certificate
shield.enabled: true
script.inline: on
script.indexed: on
shield.ssl.keystore.path: /etc/elasticsearch/shield/keystore.jks
shield.ssl.keystore.password: ****
shield.transport.ssl: true
shield.http.ssl: true
shield.ssl.truststore.path: /etc/elasticsearch/shield/truststore.jks
shield.ssl.truststore.password: ***
shield.ssl.hostname_verification: false
shield.ssl.hostname_verification.resolve_name: false
watcher.actions.pagerduty.service:
account:
my_pagerduty_account:
service_api_key: ******
event_defaults:
description: "Watch notification"
#incident_key: "error_watch"
#client: "_watcher/watch/error_watch"
#client_url: "https://localhost:9200/"
#event_type: trigger
#attach_payload: true

2

Hey,

as the pagerduty action has this issue, it seems as if there is no valid cert to connect to pagerduty. A few more questions:

  • What exact Elasticsearch version are you using?
  • What JVM are you using? Please share the output of java -version. Is this JVM installed from a distributor as a debian or RPM package?
  • Can you share the full watch?

--Alex

3

Thanks for the help, Here is required detail.
Java version= "1.8.0_131"
Elasticsearch version =2.4
So JVM is installed from debian .

4

Below is json for my watcher:

curl -u usename:password -k 'https://localhost:9200/_watcher/watch/error_watch?pretty' -XPUT -d '{
"trigger":{
"schedule":{
"interval":"5s"
}
},
"input":{
"search":{
"request":{
"indices":[
"logs"
],
"body":{
"query":{
"match":{
"message":"error"
}
}
}
}
}
},
"condition":{
"compare":{
"ctx.payload.hits.total":{
"gt":0
}
}
},
"actions" : {
"notify-pagerduty" : {
"throttle_period" : "5m",
"pagerduty" : {
"description" : "Main system down, please check! Happened at {{ctx.execution_time}}"

}

}
}
}'

5

output of java -version is below:

java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

6

Elasticsearch with x-pack installed allows a limited set of ciphers for secure connections. It might be that the list of ciphers does not match with pagerduty. You can use a tool like sslscan to get a list of supported ciphers on the pagerduty side.

The default supported ciphers on 2.4 are TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA.

You can change this with the shield.ssl.ciphers setting.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.