Issues with Wildcard Watcher

Elastic Stack Kibana
1

I am writing a watcher to monitor failures on a website of XML files imported that fail due to invalid character(s) in the XML. This is what I have right now, but for some reason it isn't working. Can anyone help me out?

The watcher looks at windows logs and searches for any logs with a message like
"The following error occurred while converting Filename.XML to the standard xml format for CLIENT

Invalid character in the given encoding. Line 12, position 37"

Watcher:

    {
  "trigger": {
    "schedule": {
      "interval": "10s"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "*"
        ],
        "rest_total_hits_as_int": true,
        "body": {
          "query": {
            "bool": {
              "must": [
                {
                  "wildcard": {
                    "message": {
                      "value": "The following error occurred while converting *.XML to the standard xml format for CLIENT*Invalid character in the given encoding. Line *, position *"
                    }
                  }
                }
              ],
              "filter": {
                "range": {
                  "@timestamp": {
                    "gte": "now-20m"
                  }
                }
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gt": 0
      }
    }
  },
  "actions": {
    "email_admin": {
      "email": {
        "profile": "standard",
        "to": [
          "monitoring@mydomain.com"
        ],
        "subject": "{{ ctx.payload.hits.total }} CLIENT Files Failed to Import",
        "body": {
          "text": "Elasticsearch has flagged {{ ctx.payload.hits.total }} error(s) indicating a CLIENT Files(s) failed to import for invalid XML. Please ignore this email if you believe it was sent in error or that the issue has been resolved."
        }
      }
    }
  }
}

Any more information needed please let me know.

2

Hi @LiamSwe

You might want to use a [match_phrase](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/query-dsl-match-query-phrase.html) rather than a wildcard query.

For example:

{
  "query": {
    "match_phrase": {
      "message": {
        "query": "Invalid character in the given encoding"
      }
    }
  }
}

Wildcard queries can be tricky. Here's a great article about writing useful queries.

1 Like
3

Thank you! This is much simpler and appears to be working perfectly!

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.