Watcher, hits array empty

Hello, I am trying to use ctx fields in watcher action but some of them is empty

My watcher is:

{
  "trigger": {
    "schedule": {
      "interval": "1m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "*"
        ],
        "rest_total_hits_as_int": true,
        "body": {
          "size": 0,
          "query": {
            "bool": {
              "must": [
                {
                  "match": {
                    "Msg": "No need to reconnect."
                  }
                },
                {
                  "range": {
                    "DateTime": {
                      "gte": "now-5m"
                    }
                  }
                }
              ]
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gte": 1
      }
    }
  },
  "actions": {
    "opegenie": {
      "max_iterations": 100,
      "webhook": {
        "scheme": "https",
        "host": "api.eu.opsgenie.com",
        "port": 443,
        "method": "post",
        "path": "/v2/alerts",
        "params": {},
        "headers": {
          "Authorization": "GenieKey ...",
          "Content-Type": "application/json"
        },
        "body": """{"message": "watcher", "alias": "watcher", "description":"hits.total={{ctx.payload.hits.total}},\n _shards.successful={{ctx.payload._shards.successful}},\n hits.hits[0]._source.Component={{ctx.payload.hits.hits[0]._source.Component}},\n execution_time={{ctx.execution_time}}", "entity":"ctx={{ctx}}, var={{var}}", "priority":"P1"}"""
      }
    }
  }
}

And it works, but ctx.payload.hits.hits[0]._source.Component field is empty.
If I use ctx.payload.hits.hits.0._source.Component it fail with "reason": "Failed to get value for ctx.payload.hits.hits.0._source.Component @[query-template:1]",

I understand that this errors because hits array is empty, then I print {{ctx}} I receive that:

{
metadata={
        xpack={type=json}
        },
watch_id=d938f2ea-ead0-4788-b5d9-dcd3b2180423,
payload={
        _shards={total=39, failed=0, successful=39, skipped=0},
        hits={hits=[], total=24, max_score=null},
        took=14,
        timed_out=false
        },
        id=d938f2ea-ead0-4788-b5d9-dcd3b2180423_ed0a6c33-dd04-44a7-8d34-8313a41ac74d-2021-03-30T17:08:02.358297161Z,
        trigger={triggered_time=2021-03-30T17:08:02.358Z, scheduled_time=2021-03-30T17:08:02.270Z},
        vars={},
        execution_time=2021-03-30T17:08:02.358297161Z
}

I have tried to use foreach like this:

...
  "actions": {
    "opegenie": {
      "foreach": "ctx.payload.hits.hits",
      "max_iterations": 100,
      "webhook": {
...

And it fail with "reason": "foreach object [ctx.payload.hits.hits] was an empty list, could not run any action"

But if I make query from Dev Tools like this:

GET _search
{
    "query": {
    "bool": {
      "must": [
        {
          "match": {
            "Msg": "No need to reconnect."
          }
        },
        {
          "range": {
            "DateTime": {
              "gte": "now-5m"
            }
          }
        }
      ]
    }
  }
}

I receive not empty hits array:

{
  "took" : 9,
  "timed_out" : false,
  "_shards" : {
    "total" : 40,
    "successful" : 40,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 29,
      "relation" : "eq"
    },
    "max_score" : 18.151394,
    "hits" : [
      {
        "_index" : "logs-2021-03-31",
        "_type" : "_doc",
        "_id" : "601c9f7da7884736b1ef29f54e23d2f9",
        "_score" : 18.151394,
        "_source" : {
          "DateTime" : "2021-03-31T12:15:37.6742108Z",
          "Level" : "info",
          "Component" : "Adapter"
        }
      },
      {
        "_index" : "logs-2021-03-31",
        "_type" : "_doc",
        "_id" : "56e3a92b061d44249f79c26ce1171a3e",
        "_score" : 18.151394,
        "_source" : {
          "DateTime" : "2021-03-31T12:15:48.3257299Z",
...

Why my hits array is empty in the watch - may be some security parameters at elastic search nodes?
help me please - I have no options

I have read similar issues here like: watcher-parse-an-array, hits-hits-array-empty-even-thought-total-1, watcher-sometimes-input-search-hits-total-return-0 and ctx-payload-hits-hits-0-source-field and don't understand what I did wrong

My configuration is

elasticsearch docker-compose.yml

version: '2.2'
services:
  esdc01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.11.1
    container_name: esdc01
    environment:
      - node.name=esdc01
      - cluster.name=es-docker-cluster
      - cluster.initial_master_nodes=esdc01
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - AZURE_ACCOUNT=${AZURE_ACCOUNT}
      - AZURE_KEY=${AZURE_KEY}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data01-7111:/usr/share/elasticsearch/data
      - ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - ./httpx.key:/usr/share/elasticsearch/config/httpx.key
      - ./httpx.crt:/usr/share/elasticsearch/config/httpx.crt
      - ./httpx-ca.crt:/usr/share/elasticsearch/config/httpx-ca.crt
    ports:
      - 9200:9200
    networks:
      - elastic

volumes:
  data01-7111:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /ext/elastic/esdc01-7111

networks:
  elastic:
    driver: bridge

elasticsearch.yml

cluster.name: "docker-cluster"
network.host: 0.0.0.0
indices.query.bool.max_clause_count: "2000"
xpack.security.enabled: true
xpack.security.authc:
  anonymous:
    username: anonymous_user
    roles: role1, role2
    authz_exception: false
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key:  httpx.key
xpack.security.http.ssl.certificate: httpx.crt
xpack.security.http.ssl.certificate_authorities: [ "httpx-ca.crt" ]

kibana docker-compose.yaml

version: '2'
services:
    kibana:
      image: docker.elastic.co/kibana/kibana:7.11.1
      container_name: kibana
      ports:
         - 5601:5601
      volumes:
         - ./kibana.yml:/usr/share/kibana/config/kibana.yml
         - ./elasticsearch-ca.pem:/usr/share/kibana/config/elasticsearch-ca.pem

kibana.yml

server.name: kibana
server.host: "0"
elasticsearch.hosts: [ "https://example.com:9200" ]
elasticsearch.shardTimeout: 300000
elasticsearch.requestTimeout: 300001
xpack.security.enabled: true
monitoring.cluster_alerts.email_notifications.email_address: email@example.com
xpack.encryptedSavedObjects.encryptionKey: "..."
xpack.security.encryptionKey: "..."
xpack.reporting.encryptionKey: "..."
xpack.reporting.kibanaServer.hostname: 0.0.0.0
xpack.reporting.capture.browser.chromium.disableSandbox: false
elasticsearch.username: elastic
elasticsearch.password: ...
1 Like

Your hits array is empty because you told Elasticsearch not to return any hits, when you set "size": 0,. The default size is 10.

1 Like

Great! Thank you so much! It works! :slight_smile: