Any way of specifying a tag within a watch?

Elastic Stack Elasticsearch
1

ES 5.6 with x-pack
I'm trying to extract a field that matches a specific tag in my watch. My watch looks like this:

PUT _xpack/watcher/watch/error_logs
{
  "trigger" : {
    "schedule" : { "interval" : "20s" } 
  },
  "input" : {
    "search" : {
      "request" : {
        "indices" : [ "<logstash-{now/d}>" ],
        "body" : {
          "query" : {
            "match" : { "error_message": "Not able to find userId" }
          }
        }
      }
    }
  },
  "condition" : {
    "compare" : { "ctx.payload.hits.total" : { "gt" : 0 }} 
  },
  "actions" : {
    "send_email" : {
      "email" : {
        "to" : "email@email.com",
        "subject" : "Encountered {{ctx.payload.hits.total}} Errors",
        "body" : "Too many errors found",
        "attachments" : {
          "attached_data" : {
            "data" : {
              "format" : "json"
            }
          }
        }
 
     }
   }
  }
}

The output of my watch looks like this:

"ctx" : {
    "metadata" : null,
    "watch_id" : "error_logs",
    "payload" : {
      "_shards" : {
        "total" : 5,
        "failed" : 0,
        "successful" : 5,
        "skipped" : 0
      },
      "hits" : {
        "hits" : [
          {
            "_index" : "logstash-2019.05.21",
            "_type" : "linux-logs",
            "_source" : {
              "severity" : "*WARN*",
              "error_message" : "Not able to find user for userId",
              "offset" : 68548,
              "method" : "GET",
              "ip" : "10.x.x.x",
              "prospector" : {
                "type" : "log"
              },
              "source" : "/var/log/aem/error.log",
              "message" : "21.05.2019 00:00:28.126 *WARN* [10.43.32.119 [1558396828123] GET /content/regent.html HTTP/1.1] com.adobe.fd.core.security.internal.CurrentUserServiceImpl Not able to find user for userId [anonymous]",
              "type" : "linux-logs",
              "version" : "HTTP/1.1",
              "tags" : [
                "aemlogs",
                "aemlogs",
                "fglam",
                "beats_input_codec_plain_applied",
                "_grokparsefailure",
                "_fglamparsefailure"
              ],
              "input" : {
                "type" : "log"
              },
              "@timestamp" : "2019-05-21T00:00:29.984Z",

Any ideas? Thanks.

2

Can you be more specific with your question? What field are you exactly trying to extract? If you want to access the field of the search first hit, you would go with something like ctx.payload.hits.hits.0._source.MY_FIELD

hope that helps.

--Alex

3

I'm trying to limit my search to events that have the tag "aemlogs". In many cases I have the same error_message from multiple pipelines but I only need those tagged as "aemlogs". Thanks

4

this means you need to use a bool query with two must clauses. One the match query from above and one a match query for the tags (there are other ways to solve this, but I think this is the easiest one).

See https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-bool-query.html

--Alex

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.