# Enable Alerts and Actions **URL:** \ **Category:** Kibana\ **Tags:** elastic-stack-security, elastic-stack-alerting\ **Created:** [January 14, 2021, 8:57pm UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180 "2021-01-14T20:57:19Z")\ **Posts on this page:** 18\ **Page:** 1
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [January 14, 2021, 8:57pm UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/1 "2021-01-14T20:57:19Z")
Is there a blog site or is there some one might have wrote step by step instruction on how to turn on Alerts and Actions? I have read few document and trying to follow this but got completely lost [https://www.elastic.co/guide/en/kibana/current/configuring-tls.html#configuring-tls-kib-es](https://www.elastic.co/guide/en/kibana/current/configuring-tls.html#configuring-tls-kib-es) I am using CA certificate between elastic cluster and using user/password from kibana but kibana is still using http Created certificated with following command ``` bin/elasticsearch-certutil cert --ca elastic-certificates.p12 ``` then copy over that to all elasticcluster nodes and each of them using ``` xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/elastic-certificates.p12 ``` this is working fine. now I am trying to turn on Kibana Alerting. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [January 15, 2021, 1:24am UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/2 "2021-01-15T01:24:42Z")
Assuming Kibana and Elasticsearch up and running. See [here](https://www.elastic.co/guide/en/kibana/current/alerting-getting-started.html). Navigate to Stack Management / Alerts and Actions is that not there? ![Screen Shot 2021-01-14 at 5.24.00 PM](https://us1.discourse-cdn.com/elastic/original/3X/d/8/d8ac553f509840fd3be77098fd9606778623e401.png) ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [January 15, 2021, 2:44pm UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/3 "2021-01-15T14:44:32Z")
it is there but trying to setup that security is where I am having difficulty ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/1/711380d5bce618efcbfab8b5e043daeea572521b.png) ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [January 16, 2021, 2:12am UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/4 "2021-01-16T02:12:46Z")
To enable security for both elasitcsearch and kibana follow [this tutorial](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-getting-started.html) That message indicates that the communication between Kibana and Elasticsearch is not TLS plus you need to enable SSL on kibana have you followed [these steps?](https://www.elastic.co/guide/en/kibana/current/using-kibana-with-security.html) I wrote a short how to secure a single node cluster a while back perhaps that will [help](https://github.com/bvader/howtos/blob/master/basic-security-elasticsearch/README.md) ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [February 1, 2021, 6:20pm UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/5 "2021-02-01T18:20:07Z")
What is this error means I am able to setup ssl certificate for elasticsearch communiction. this is from kibana log. I can use [https://localhost:5601](https://localhost:5601) but it just gives me message "kibana server is not ready yet" `{"type":"log","@timestamp":"2021-02-01T10:22:17Z","tags":["error","elasticsearch","data"],"pid":16047,"message":"[ConnectionError]: self signed certificate in certificate chain"}` and this message `{"type":"log","@timestamp":"2021-02-01T10:24:52Z","tags":["warning","elasticsearch","monitoring"],"pid":16047,"message":"Unable to revive connection: https://perelkm01:9200/"}` this tells me missing some certificate. but then I can connect to cluster from command line ` curl -u elastic:changeme -k --cacert /etc/elasticsearch/config/elasticsearch-ca.pem https://localhost:9200` What did I miss ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [February 2, 2021, 4:23am UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/6 "2021-02-02T04:23:37Z")
Hard to tell from only snippets of the logs But if you refer to ES by [https://perelkm01:9200/](https://perelkm01:9200/) Then you needed to add that to the certificate when you created it... Certs are certs you got to get it right You might also turn off ssl verification mode `elasticsearch.ssl.verificationMode` ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [February 2, 2021, 3:20pm UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/7 "2021-02-02T15:20:18Z")
> [@stephenb](#): > > Then you needed to add that to the certificate when you created it... This I have not done. is this means every host in cluster who is kibana needs to be added. got it. Let me test it out today. ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [February 2, 2021, 11:09pm UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/8 "2021-02-02T23:09:50Z")
made little more progress. now kibana is not dying but i have following error messages. and https:// gives me only message called "Kibana server is not ready yet" ``` {"type":"log","@timestamp":"2021-02-02T15:01:35Z","tags":["warning","config","deprecation"],"pid":8940,"message":"Config key [monitoring.cluster_alerts.email_notifications.email_address] will be required for email notifications to work in 8.0.\""} {"type":"log","@timestamp":"2021-02-02T15:01:35Z","tags":["warning","plugins","security","config"],"pid":8940,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"} {"type":"log","@timestamp":"2021-02-02T15:01:35Z","tags":["warning","plugins","reporting","config"],"pid":8940,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.reporting.encryptionKey in kibana.yml"} {"type":"log","@timestamp":"2021-02-02T15:01:35Z","tags":["warning","plugins","reporting","config"],"pid":8940,"message":"Chromium sandbox provides an additional layer of protection, but is not supported for Linux Red Hat Linux 7.5 OS. Automatically setting 'xpack.reporting.capture.browser.chromium.disableSandbox: true'."} {"type":"log","@timestamp":"2021-02-02T15:01:35Z","tags":["info","plugins","monitoring","monitoring"],"pid":8940,"message":"config sourced from: production cluster"} {"type":"log","@timestamp":"2021-02-02T15:01:36Z","tags":["info","savedobjects-service"],"pid":8940,"message":"Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations..."} {"type":"log","@timestamp":"2021-02-02T15:01:36Z","tags":["error","elasticsearch","monitoring"],"pid":8940,"message":"Request error, retrying\nGET https://perelkm01:9200/_xpack => self signed certificate in certificate chain"} {"type":"log","@timestamp":"2021-02-02T15:01:36Z","tags":["warning","elasticsearch","monitoring"],"pid":8940,"message":"Unable to revive connection: https://perelkm01:9200/"} {"type":"log","@timestamp":"2021-02-02T15:01:36Z","tags":["warning","elasticsearch","monitoring"],"pid":8940,"message":"No living connections"} {"type":"log","@timestamp":"2021-02-02T15:01:36Z","tags":["warning","plugins","licensing"],"pid":8940,"message":"License information could not be obtained from Elasticsearch due to Error: No Living connections error"} {"type":"log","@timestamp":"2021-02-02T15:01:36Z","tags":["warning","plugins","monitoring","monitoring"],"pid":8940,"message":"X-Pack Monitoring Cluster Alerts will not be available: No Living connections"} {"type":"log","@timestamp":"2021-02-02T15:01:36Z","tags":["error","elasticsearch","data"],"pid":8940,"message":"[ConnectionError]: self signed certificate in certificate chain"} {"type":"log","@timestamp":"2021-02-02T15:01:36Z","tags":["error","savedobjects-service"],"pid":8940,"message":"Unable to retrieve version information from Elasticsearch nodes."} {"type":"log","@timestamp":"2021-02-02T15:01:38Z","tags":["error","elasticsearch","data"],"pid":8940,"message":"[ConnectionError]: self signed certificate in certificate chain"} {"type":"log","@timestamp":"2021-02-02T15:01:41Z","tags":["error","elasticsearch","data"],"pid":8940,"message":"[ConnectionError]: self signed certificate in certificate chain"} {"type":"log","@timestamp":"2021-02-02T15:01:43Z","tags":["error","elasticsearch","data"],"pid":8940,"message":"[ConnectionError]: self signed certificate in certificate chain"} {"type":"log","@timestamp":"2021-02-02T15:01:46Z","tags":["error","elasticsearch","data"],"pid":8940,"message":"[ConnectionError]: self signed certificate in certificate chain"} ``` this is what my config looks like kibana.yml ``` server.ssl.enabled: true server.ssl.certificate: /etc/kibana/config/kibana.pem server.ssl.key: /etc/kibana/config/kibana.key # Optional setting that enables you to specify a path to the PEM file for the certificate # authority for your Elasticsearch instance. elasticsearch.ssl.certificateAuthorities: ["/etc/kibana/config/selfca.pem"] ``` This is what my elasticsearch.yml file looks like ``` xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/config/perelkm01.p12 xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/config/perelkm01.p12 # SSL setup xpack.security.http.ssl.enabled: true xpack.security.http.ssl.keystore.path: /etc/elasticsearch/config/perelkm01.p12 xpack.security.http.ssl.truststore.path: /etc/elasticsearch/config/perelkm01.p12 ``` ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [February 2, 2021, 11:34pm UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/9 "2021-02-02T23:34:23Z")
> [@elasticforme](#): > > `https://perelkm01:9200/` can you just curl this ``` curl -u "elastic:myawesomepassword" --cacert certs/selfca.pem https://perelkm01:9200/ ``` does that work? Debugging certs can be painful elastic follows all the standards so you just need to get it right. First I would make sure the cluster is up .. this is a single node only right. If you followed my walkthrough it should work... ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [February 3, 2021, 12:01am UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/10 "2021-02-03T00:01:12Z")
> [@stephenb](#): > > `curl -u "elastic:myawesomepassword" --cacert certs/selfca.pem https://perelkm01:9200/` yes it works but I had to use -k option as insuc ``` curl -k -u elastic:changeme --cacert selfca.pem https://perelkm01:9200/ { "name" : "perelkm01", "cluster_name" : "test", "cluster_uuid" : "1WjGacI4ToCbNly7U3RMEw", "version" : { "number" : "7.10.1", "build_flavor" : "default", "build_type" : "rpm", "build_hash" : "1c34507e66d7db1211f66f3513706fdf548736aa", "build_date" : "2020-12-05T01:00:33.671820Z", "build_snapshot" : false, "lucene_version" : "8.7.0", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" } ``` ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [February 3, 2021, 1:31am UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/11 "2021-02-03T01:31:23Z")
Well then something is not right with the because when I build up following my directions i run without the `-k` ``` curl -u elastic:password --cacert ./certs/selfca.pem https://localhost:9200 curl -u elastic:password --cacert ./certs/selfca.pem https://stephenb-metrics-test-es1:9200 curl -u elastic:password --cacert ./certs/selfca.pem https://10.168.0.81:9200 ``` All 3 work.. ``` { "name" : "stephenb-metrics-test-es1", "cluster_name" : "elasticsearch", "cluster_uuid" : "sIFMd-G-Spi80z4Roaauqw", "version" : { "number" : "7.7.0", "build_flavor" : "default", "build_type" : "deb", "build_hash" : "81a1e9eda8e6183f5237786246f6dced26a10eaf", "build_date" : "2020-05-10T19:02:51.169154Z", "build_snapshot" : false, "lucene_version" : "8.5.1", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" } ``` I set up my cert with all three names / ips You can see in my instructions ``` ./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 --ip 10.168.0.71,127.0.0.1 --dns hostname,localhost openssl pkcs12 -in elastic-stack-ca.p12 -out selfca.pem -clcerts -nokeys ``` where my `hostname` is `stephenb-metrics-test-es1` and yours should be `perelkm01` [Detailed Instructions here](https://github.com/bvader/howtos/tree/master/basic-security-elasticsearch) _We will be [setting up TLS on the node](https://www.elastic.co/guide/en/elasticsearch/reference/7.5/ssl-tls.html). Setup the certs, we will [creating self signed certs](https://www.elastic.co/guide/en/elasticsearch/reference/7.5/configuring-tls.html#node-certificates) using the `elasticsearch-certutil` tool. Note you will want to create the cert with the internal address, and loopback and a dns entry for `localhost` . If you have an addition external IP you would add to the ip list as well. We also added `localhost` and the `hostname` for the DNS section. We are also going to extract / create the ca for later use, you will need it. We are going to create them and then put them in a certs directory in the elasticsearch config directory._ _Just take the defaults for all the question and skip the passwords for the .p12s, you can try that later if you like._ ``` sudo -i cd /usr/share/elasticsearch ./bin/elasticsearch-certutil ca ./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 --ip 10.168.0.71,127.0.0.1 --dns hostname,localhost openssl pkcs12 -in elastic-stack-ca.p12 -out selfca.pem -clcerts -nokeys mkdir /etc/elasticsearch/certs chmod 755 /etc/elasticsearch/certs mv *.p12 /etc/elasticsearch/certs/. mv *.pem /etc/elasticsearch/certs/. chmod 664 /etc/elasticsearch/certs/* ``` ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [February 3, 2021, 1:37am UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/12 "2021-02-03T01:37:53Z")
yes I have been following your post this is three node cluster and I believe ca certificate is working as I can connect to elasticcluster from all node -k is just because it is new curl ``` curl message if I don't use -k If you'd like to turn off curl's verification of the certificate, use the -k (or --insecure) option. ``` but yes I have not put 127.0.0.1 address when I created individual kibana.key and cert I only use name, full name and IP address. Let me test it out this tomorrow morning. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [February 3, 2021, 2:05am UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/13 "2021-02-03T02:05:05Z")
Can we just say it together... self signed certs are painful ☹ Good luck you will find it. ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [February 3, 2021, 1:40pm UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/14 "2021-02-03T13:40:48Z")
Absolutely agree. it is main pain in..... had spend so much time already trying to figure out. main reason I want to do this is to activate alert. once successful will have to do in bigger cluster. I will defiantly put document like you did here for someone else to understand. LOL ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [February 10, 2021, 2:13pm UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/15 "2021-02-10T14:13:33Z")
Just to update this thread. Last night I have successfully turn on TLS and HTTPS. This is three node cluster that I was testing. and I have repetedly done this to make sure procedure works. @stephenb Thank you for your blog post. it is pretty much using that and some other post. I will post full document in day or two. ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [February 10, 2021, 4:12pm UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/16 "2021-02-10T16:12:12Z")
Full document on setting TLS/HTTPS and SSL this happens in few steps 1. create internal password 2. create instance file 3. create keys/certs 4. update elasticsearch.yml file 5. restart all 6. check if it worked 7. copy require kesy/certs to kibana locations 8. update kibana.yml file 9. start kiabna and test. /usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive I am using all short name. ``` instances: - name: "node01" ip: - "10.82.1.61" - "127.0.0.1" dns: - "node01" - "localhost" - name: "node02" ip: - "10.82.1.62" - "127.0.0.1" dns: - "node02" - "localhost" - name: "node03" ip: - "10.82.1.63" - "127.0.0.1" dns: - "node03" - "localhost" ``` `/usr/share/elasticsearch/bin/elasticsearch-certutil cert --keep-ca-key --pem --in /root/instance.yml --out /etc/elasticsearch/sachin_certs/certs.zip` `unzip certs.zip` Copy all node keys/certs to ca dir for simplicity ``` rsync -av node01/* ca/ rsync -av node02/* ca/ rsync -av node03/* ca/ ``` copy this ca dir to two more node ``` rsync -av * node02:/etc/elasticsearch/sachin_certs/ rsync -av * node03:/etc/elasticsearch/sachin_certs/ ``` Now setup elasticsearch.yml file, do this on all three node, replace node01 to other node name ``` xpack.security.enabled: true xpack.security.http.ssl.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.http.ssl.key: /etc/elasticsearch/sachin_certs/ca/node01.key xpack.security.http.ssl.certificate: /etc/elasticsearch/sachin_certs/ca/node01.crt xpack.security.http.ssl.certificate_authorities: /etc/elasticsearch/sachin_certs/ca/ca.crt xpack.security.transport.ssl.key: /etc/elasticsearch/sachin_certs/ca/node01.key xpack.security.transport.ssl.certificate: /etc/elasticsearch/sachin_certs/ca/node01.crt xpack.security.transport.ssl.certificate_authorities: /etc/elasticsearch/sachin_certs/ca/ca.crt xpack.security.transport.ssl.verification_mode: certificate ``` Start elasticsearch on all node test if you have connection. check log files etc.. if this didn't work then go back and check everything. mostly probably you will have typo or something. `curl --cacert /etc/elasticsearch/sachin_certs/ca/ca.crt -u elastic:changeme https://node01:9200/_cat/nodes?v` Now to kibana. it is easy copy whole certs to kibana config dir. `rsync -av * /etc/kibana/sachin_certs/` change kiaban.yml ``` server.ssl.enabled: true server.ssl.certificate: /etc/kibana/sachin_certs/ca/node01.crt server.ssl.key: /etc/kibana/sachin_certs/ca/node01.key elasticsearch.hosts: ["https://node01:9200"] elasticsearch.ssl.certificateAuthorities: ["/etc/kibana/sachin_certs/ca/ca.crt"] xpack.encryptedSavedObjects.encryptionKey: "something_at_least_32_characters" elasticsearch.username: "kibana_system" elasticsearch.password: "kibana_system" ``` Start kibana and you should be able to connect `https://node01:5601` ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [February 10, 2021, 4:25pm UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/17 "2021-02-10T16:25:51Z")
@elasticforme Very Nice Thanks! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 10, 2021, 4:26pm UTC](https://discuss.elastic.co/t/enable-alerts-and-actions/261180/18 "2021-03-10T16:26:21Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.