Watcher Question -Inserting log message contents into text and email body

[Johnnie843]

I set up a Watcher watch, It is running successfully but I'm trying to insert some contents of the log message into the action text and eventually an email body, is this possible? If so what is the syntax. Thank you for all the help!

[spinscale (Alexander Reelsen)]

hey,

please provide the watch you wrote. and what part you would like to modify (I suppose the email action, but it helps a lot to see the initial watch - stripped of any sensitive data of course).

--Alex

[Johnnie843]

Highlighted in bold is the area where I would like to insert some of the "hit" data returned from the watch (ex. message):

{
"trigger": {
"schedule": {
"interval": "24h"
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"logstash*"
],
"types": ,
"body": {
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "message: "Transport Connection to:") AND (message:"java.io.EOFException""
}
},
{
"range": {
"@timestamp": {
"gte": "now-7d"
}
}
}
]
}
},
"_source": [
"message"
],
"sort": [
{
"@timestamp": {
"order": "desc"
}
}
]
}
}
}
},
"condition": {
"compare": {
"ctx.payload.hits.total": {
"gt": 0
}
}
},
"actions": {
"log": {
"logging": {
"level": "info",
"text": "{{ctx.payload.hits.total}} Errors have occured in the logs:{{#ctx.payload.hits.hits}}{{_id}}:{{/ctx.payload.hits.hits}}"

  }
}

},
"throttle_period_in_millis": 900000
}

I haven't included the email action yet but when I do I would like to edit the body of it, which should have the same syntax as inserting the message into the text, if it is possible.

[Johnnie843]

Sorry about the structure of the JSON and the white space, when I copied the watch in it took out my tabs

[spinscale (Alexander Reelsen)]

please use markdown to properly format code snippets. This is really hard to read.

the code sample looks good regarding looping through the results. You can use {{_source.foo}} to access fields within the JSON being returned.

Can you be more detailed what is not working, I may have just missed it.

[Johnnie843]

**EDITED It was working I just couldn't find any documentation to insert log messages that got "hits" into the "text" and eventually they email body once I get that up and running. Can you provide a code snippet of what you mean. From what I understand if I want to put the actual log message in the "text" I can do "{{_source.message}}"

[Johnnie843]

Also if i have multiple hits it will loop through each hit and display each message?

[Johnnie843]

Got it to work with you help! Thank you again!! One more quick question, are we able to insert new lines between each loop?

Here is the text tag I have working, Bold is what I would like to have a new line inbetween each message:

"text": "{{ctx.payload.hits.total}} Errors have occured in the logs:{{#ctx.payload.hits.hits}}{{_id}}:{{/ctx.payload.hits.hits}} Result Message:{{#ctx.payload.hits.hits}}{{_source.message}}:{{/ctx.payload.hits.hits}}"

[spinscale (Alexander Reelsen)]

hey, you should be able to add \n within the loop and it should work. In a HTML based email you may need to use an HTML tag though.

[Johnnie843]

Thank you got it the alert like I wanted....One more quick question on configuring the email account,
I found two different links which explain how to do the configuration, which one should I use?

First Link:

https://www.elastic.co/guide/en/watcher/current/email-services.html

watcher.actions.email.service.account:
outlook_account:
profile: outlook
smtp:
auth: true
starttls.enable: true
host: smtp-mail.outlook.com
port: 587
user:
password:

Second Link:
https://www.elastic.co/guide/en/x-pack/current/actions-email.html#configuring-email

xpack.notification.email.account:
outlook_account:
profile: outlook
smtp:
auth: true
starttls.enable: true
host: smtp-mail.outlook.com
port: 587
user:
password:

[Johnnie843]

Here is my elasticsearch.yml. Everytime I restart the stack the Kibana fails to come up.

# Token is required by SAML
xpack.security.authc.token.enabled: true
xpack.security.authc.realms.native1:
   type: native
   order: 0
xpack.security.authc.realms.saml1:
   type: saml
   order: 1
   idp.metadata.path: saml/idp-external.xml
   idp.entity_id: ""
   sp.entity_id:  ""
   sp.acs: ""
   sp.logout: ""
   attributes.principal: "nameid:persistent"
   attributes.groups: "roles"
   signing.certificate: saml/saml-sign.crt
   signing.key: saml/saml-sign.key

xpack.notification.email.account:
    exchange_account:
        profile: outlook
        email_defaults:
            from: "example@outlook.com"
        smtp:
            auth: false
            starttls.enable: false
            host: "ip address of host"
            port: 587
            user: "example@outlook.com"