Watcher migration Invalid Address Error

Elastic Stack Elasticsearch
1

I have managed to configure a one node cluster to send out email alerts, with the same configuration on a three node cluster I get the following error:

"type": "messaging_exception",
"reason": "failed to send email with subject [Report test] via account [smtp_account]",
"caused_by": {
        "type": "s_m_t_p_send_failed_exception",
        "reason": "501 5.1.7 Invalid address\n",
        "caused_by": {
          "type": "s_m_t_p_sender_failed_exception",
          "reason": "501 5.1.7 Invalid address\n"
        }
}

This is strange given that I changed the account name within each of the cluster configs to smtp_account_dev. The only meaningful difference between the two version is that the one that worked utilises elasticsearch 6.1.2 and is a one node cluster whereas the three node cluster utilise 6.1.1

This is the meaningful settings within the config:

xpack.security.enabled: false
xpack.watcher.enabled: true
xpack.notification.email.default_account: smtp_account_dev
xpack.notification.email.account.smtp_account_dev.profile: standard
xpack.notification.email.account.smtp_account_dev.smtp.host: smtp.???.??.com

Where am I going wrong?

2

can you share the full output of the Execute Watch API for this watch?

Thank you!

3 
{
"_id": "vima_log_errors_ea0e2ea4-609a-4960-9ea3-dee5438b19c8-2018-01-24T09:58:44.400Z",
"watch_record": {
"watch_id": "blahblah",
"node": "oDplAK8gSQue3dcFZKxGmA",
"state": "executed",
"status": {
  "state": {
    "active": true,
    "timestamp": "2018-01-23T16:45:08.803Z"
  },
  "last_checked": "2018-01-24T09:58:44.400Z",
  "last_met_condition": "2018-01-24T09:58:44.400Z",
  "actions": {
    "send_email": {
      "ack": {
        "timestamp": "2018-01-23T16:45:08.803Z",
        "state": "awaits_successful_execution"
      },
      "last_execution": {
        "timestamp": "2018-01-24T09:58:44.400Z",
        "successful": false,
        "reason": ""
      }
    }
  },
  "execution_state": "executed",
  "version": 1026
},
"trigger_event": {
  "type": "manual",
  "triggered_time": "2018-01-24T09:58:44.399Z",
  "manual": {
    "schedule": {
      "scheduled_time": "2018-01-24T09:58:44.399Z"
    }
  }
},
"input": {
  "search": {
    "request": {
      "search_type": "query_then_fetch",
      "indices": [
        "indexes"
      ],
      "types": [],
      "body": {
        "query": {
          "range": {
            "Ticket_total": {
              "gte": 10
            }
          }
        }
      }
    }
  }
},
"condition": {
  "compare": {
    "ctx.payload.hits.total": {
      "gt": 0
    }
  }
},
"metadata": {
  "color": "yellow"
},
"result": {
  "execution_time": "2018-01-24T09:58:44.400Z",
  "execution_duration": 90,
  "input": {
    "type": "search",
    "status": "success",
    "payload": {
      "_shards": {
        "total": 5,
        "failed": 0,
        "successful": 5,
        "skipped": 0
      },
      "hits": {
        "hits": [
          ""50 RESULTS""
        ],
        "total": 50,
        "max_score": 1
      },
      "took": 3,
      "timed_out": false
    },
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "algosec_stats"
        ],
        "types": [],
        "body": {
          "query": {
            "range": {
              "Ticket_total": {
                "gte": 10
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "type": "compare",
    "status": "success",
    "met": true,
    "compare": {
      "resolved_values": {
        "ctx.payload.hits.total": 50
      }
    }
  },
  "actions": [
    {
      "id": "send_email",
      "type": "email",
      "status": "failure",
      "error": {
        "root_cause": [
          {
            "type": "messaging_exception",
            "reason": "failed to send email with subject [Report test] via account [smtp_account]"
          }
        ],
        "type": "messaging_exception",
        "reason": "failed to send email with subject [Report test] via account [smtp_account]",
        "caused_by": {
          "type": "s_m_t_p_send_failed_exception",
          "reason": "501 5.1.7 Invalid address\n",
          "caused_by": {
            "type": "s_m_t_p_sender_failed_exception",
            "reason": "501 5.1.7 Invalid address\n"
          }
        }
      }
    }
  ]
},
"messages": []
}
}
4

hey,

sorry can you include the full watch including the send_email action, that one is still missing from the above entry. Thank you so much!

--Alex

5

Apologies.

PUT _xpack/watcher/watch/vima_log_errors
{
"metadata" : {
"color" : "yellow"
},
"trigger" : {
"schedule" : {
  "interval" : "1m"
 }
},
"input" : {
"search" : {
  "request" : {
    "indices" : [ "???" ],
    "body" : {
      "query": {
         "range" : {
            "Ticket_total" : {
                 "gte" : 10
             }
         }
      }
    }
  }
}
},
"condition" : {
"compare" : { "ctx.payload.hits.total" : { "gt" : 0 }}
},
"actions" : {
"send_email" : {
  "email" : {
    "to" : "???@???.com",
    "from" : "???@??.com",
    "subject" : "Report test",
    "body" : "Hello World"
  }
 }
 }
}
6

Hey,

did you replace the email address by adding question marks or is that some non ASCII email address?

7

I didn't want to show my email address, it is a valid email address.

8

I found the following being logged in elasticsearch, I'm presuming my error is caused by "from:Testing" though I can't find these settings in elasticsearch.yml, where could these settings be?

[o.e.c.s.ClusterSettings  ] [hostname] updating [xpack.notification.email.] 
from [{"account":{"smtp_account":{"profile":"standard","smtp":{"host":"smtp.??.??.com"}}},
"default_account":"smtp_account"}] 
to [{"account":{"smtp_account":{"profile":"standard","smtp":
{"host":"smtp.??.??.com","user":"root@hostname.localdomain","from":"Testing"}}},
"default_account":"smtp_account"}]
9

unless you come up with concrete values it is gonna be super hard to debug this issue, as it seems that you provided an invalid email address somewhere.

10

This is the log message with full email address:

    [o.e.c.s.ClusterSettings  ] [blt] updating [xpack.notification.email.] from [{"account":{"smtp_account":{"profile":"standard","smtp":
    {"host":"smtp.intra..com"}}},"default_account":"smtp_account"}] to 
     [{"account":{"smtp_account":{"profile":"standard","smtp":
    {"host":"smtp.intra..com","user":"root@3.localdomain",
    "from":"AnalyseIT_Testing"}}},"default_account":"smtp_account"}]

This is the email address from the action:

"actions" : {
"send_email" : {
  "email" : {
    "to" : "",
    "from" : "",
    "subject" : "Report test 1",
    "body" : "Hello World"
  }
}
}

The only invalid email I can see is the "from":"AnalyseIT_Testing"}}}," though I can't find this setting anywhere within the config so I'm wondering if it persisted across upgrades and restarts.

11

you can remove the from address by simply setting it to null when you update the settings dynamically.

Do you have specified a password for SMTP auth for the root@ user? Have you tried without SMTP auth against a local SMTP server to see if that might be the culprit?

12

Should I remove from through elasticsearch.yml? I can't seem to find the setting "from":"AnalyseIT_Testing"}}}," anywhere so not where this was set.

13

It seems someone set the "from":"AnalyseIT_Testing"}}}," in cluster settings, now I need to find a way to delete these cluster settings.

14

You can use the cluster update settings API for that

https://www.elastic.co/guide/en/elasticsearch/reference/6.1/cluster-update-settings.html

15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.