Watcher Automated Report Failing

Elastic Stack Kibana
1

I am trying to create a watcher that generates and attaches a CSV file to an email that gets sent out twice weekly. I have previously done this with a different index and it is working for that watcher. I am getting an error that I cannot identify when executing as a simulation.

The POST URL is generated from a saved Discover view. The error I am receiving is:

"actions": [
      {
        "id": "email_admin",
        "type": "email",
        "status": "failure",
        "error": {
          "root_cause": [
            {
              "type": "exception",
              "reason": "Watch[_inlined_] reporting[user_data.csv] Error when polling pdf from host[fc346d3a79794e518b08a42b0e51f1bc.eastus2.azure.elastic-cloud.com], port[9243], method[POST], path[/api/reporting/generate/csv], status[500], body[{\"statusCode\":500,\"error\":\"Internal Server Error\",\"message\":\"Reporting generation failed: Error: Expected _scroll_id in the following Elasticsearch response: {\\\"took\\\":1,\\\"timed_out\\\":false,\\\"_shards\\\":{\\\"total\\\":0,\\\"successful\\\":0,\\\"skipped\\\":0,\\\"failed\\\":0},\\\"hits\\\":{\\\"total\\\":{\\\"value\\\":0,\\\"relation\\\":\\\"eq\\\"},\\\"max_score\\\":0,\\\"hits\\\":[]}}\"}]"
            }
          ],
          "type": "exception",
          "reason": "Watch[_inlined_] reporting[user_data.csv] Error when polling pdf from host[fc346d3a79794e518b08a42b0e51f1bc.eastus2.azure.elastic-cloud.com], port[9243], method[POST], path[/api/reporting/generate/csv], status[500], body[{\"statusCode\":500,\"error\":\"Internal Server Error\",\"message\":\"Reporting generation failed: Error: Expected _scroll_id in the following Elasticsearch response: {\\\"took\\\":1,\\\"timed_out\\\":false,\\\"_shards\\\":{\\\"total\\\":0,\\\"successful\\\":0,\\\"skipped\\\":0,\\\"failed\\\":0},\\\"hits\\\":{\\\"total\\\":{\\\"value\\\":0,\\\"relation\\\":\\\"eq\\\"},\\\"max_score\\\":0,\\\"hits\\\":[]}}\"}]"
        }
      }
    ]

Here is the JSON for the watcher:

{
  "trigger": {
    "schedule": {
      "weekly": [
        {
          "on": [
            "MON",
            "THU"
          ],
          "at": [
            "13:00"
          ]
        }
      ]
    }
  },
  "input": {
    "none": {}
  },
  "condition": {
    "always": {}
  },
  "actions": {
    "email_admin": {
      "email": {
        "profile": "standard",
        "attachments": {
          "user_data.csv": {
            "reporting": {
              "url": "https://fc346d3a79794e518b08a42b0e51f1bc.eastus2.azure.elastic-cloud.com:9243/api/reporting/generate/csv?jobParams=%28browserTimezone%3AAmerica%2FNew_York%2CconflictedTypesFields%3A%21%28%29%2Cfields%3A%21%28%27%40timestamp%27%2C_id%2C_index%2C_score%2C_type%2Cfields.agent%2Cfields.email%2Cfields.environment%2Cfields.firstName%2Cfields.ip%2Cfields.lastName%2Cfields.log.action%2Cfields.log.endDate%2Cfields.log.facilityKey%2Cfields.log.groupId%2Cfields.log.historyEntryType%2Cfields.log.module%2Cfields.log.startDate%2Cfields.orgId%2Cfields.organization%2Cfields.platform%2Cfields.userId%2Cgeoip.city_name%2Cgeoip.continent_name%2Cgeoip.country_iso_code%2Cgeoip.location%2Cgeoip.region_iso_code%2Cgeoip.region_name%2Clevel%2Cmessage%2CmessageTemplate%2Cuser_agent.device.name%2Cuser_agent.name%2Cuser_agent.original%2Cuser_agent.os.full%2Cuser_agent.os.name%2Cuser_agent.os.version%2Cuser_agent.version%29%2CindexPatternId%3A%274eb9bbc0-b0ac-11ea-92d0-c908c91449e6%27%2CmetaFields%3A%21%28_source%2C_id%2C_type%2C_index%2C_score%29%2CobjectType%3Asearch%2CsearchRequest%3A%28body%3A%28_source%3A%28excludes%3A%21%28%29%29%2Cdocvalue_fields%3A%21%28%28field%3A%27%40timestamp%27%2Cformat%3Adate_time%29%29%2Cquery%3A%28bool%3A%28filter%3A%21%28%28match_all%3A%28%29%29%2C%28match_phrase%3A%28fields.environment.keyword%3AProduction%29%29%2C%28range%3A%28%27%40timestamp%27%3A%28format%3Astrict_date_optional_time%2Cgte%3A%272020-09-16T13%3A41%3A36.815Z%27%2Clte%3A%272020-09-23T13%3A41%3A36.815Z%27%29%29%29%29%2Cmust%3A%21%28%29%2Cmust_not%3A%21%28%28match_phrase%3A%28fields.organization.keyword%3ADemo%29%29%2C%28match_phrase%3A%28fields.email.keyword%3A%27%2A%40careprepare.com%27%29%29%29%2Cshould%3A%21%28%29%29%29%2Cscript_fields%3A%28%29%2Csort%3A%21%28%28%27%40timestamp%27%3A%28order%3Adesc%2Cunmapped_type%3Aboolean%29%29%29%2Cstored_fields%3A%21%28%27%2A%27%29%2Cversion%3A%21t%29%2Cindex%3A%27cpplogs%2A%27%29%2Ctitle%3A%27CPP%20User%20Tracking%27%29",
              "auth": {
                "basic": {
                  "username": "watcher",
                  "password": "::es:redacted::"
                }
              }
            }
          }
        },
        "to": [
          "'INFO REDACTED <placeholder@gmail.com>'"
        ],
        "subject": "INFO REDACTED"
      }
    }
  }
}
2

Seems that your search return no documents!
make sure your new index contain documents

{
   "took":1,
   "timed_out":false,
   "_shards":{
      "total":0,
      "successful":0,
      "skipped":0,
      "failed":0
   },
   "hits":{
      "total":{
         "value":0,
         "relation":"eq"
      },
      "max_score":0,
      "hits":[
         
      ]
   }
}
3

@ylasri Thanks for the reply. I'm not sure how that could be the case. There are 43 results returned from the query when using the Discovery tool and from there I copied the POST URL and pasted into the watcher attachment url field.

4

Check the time range

5

It ended up being a role didn't have permissions to access the index. Your post made me think to check that! Thank you @ylasri.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.