Automated Daily PDF Reports Failing with Error: http_host_connect_exception

I'm using Kibana and Elasticsearch 6.3, and after generating a report PDF from a visualization, I have tried to automate it inside of a watcher. The watcher always fires / 1 day, but its execution is failing with the below error message:

  • I've tried it with and without the connection_timeout
  • I've tried it with "scheme:http" and "scheme:https"
  • I've tried speeding up the watcher to fire / 5 minutes for testing
  • I've successfully sent emails with other watchers
  • I duplicated the watcher code presented in:
    You Get A Report! You Get a Report!
  • I dropped in the search parameters and triggering conditions from a good, working watcher.
  • I'm able to generate the report manually with the Visualization:Report:Generate button
  • The IP Address that the reporting URL references is the same machine running the Kibana instance, and the 3rd node (master, non-data) in our elasticsearch cluster

What am I doing wrong?

Watcher Execution [Error] Result:
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
{
"watch_id": "Daily_Report_Part_Events_per_Product",
"node": "CmPiz4-kQ1moZQsiDNuAnA",
"state": "executed",
"status": {
"state": {
"active": true,
"timestamp": "2019-06-04T14:17:59.245Z"
},
"last_checked": "2019-06-04T14:20:38.048Z",
"last_met_condition": "2019-06-04T14:20:38.048Z",
"actions": {
"Email_Daily_Report": {
"ack": {
"timestamp": "2019-06-04T14:14:30.297Z",
"state": "awaits_successful_execution"
},
"last_execution": {
"timestamp": "2019-06-04T14:20:38.048Z",
"successful": false,
"reason": ""
}
}
},
"execution_state": "executed",
"version": -1
},
"trigger_event": {
"type": "schedule",
"triggered_time": "2019-06-04T14:20:38.048Z",
"schedule": {
"scheduled_time": "2019-06-04T14:20:37.788Z"
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"monitoring-loading-last-file-state*",
"logstash*",
"monitoring-loading-processed*",
"monitoring-loading-raw*",
"monitoring-part-event*"
],
"types": ,
"body": {
"size": 0,
"query": {
"bool": {
"filter": {
"range": {
"@timestamp": {
"gte": "{{ctx.trigger.scheduled_time}}||-1d",
"lte": "{{ctx.trigger.scheduled_time}}",
"format": "strict_date_optional_time||epoch_millis"
}
}
}
}
}
}
}
}
},
"condition": {
"script": {
"source": "if (ctx.payload.hits.total > params.threshold) { return true; } return false;",
"lang": "painless",
"params": {
"threshold": -1
}
}
},
"metadata": {
"name": "[Daily Report] Part Events / Product / Hour"
},
"result": {
"execution_time": "2019-06-04T14:20:38.048Z",
"execution_duration": 127246,
"input": {
"type": "search",
"status": "success",
"payload": {
"_shards": {
"total": 1410,
"failed": 0,
"successful": 1410,
"skipped": 1380
},
"hits": {
"hits": ,
"total": 60256,
"max_score": 0
},
"took": 66,
"timed_out": false
},
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"monitoring-loading-last-file-state*",
"logstash*",
"monitoring-loading-processed*",
"monitoring-loading-raw*",
"monitoring-part-event*"
],
"types": ,
"body": {
"size": 0,
"query": {
"bool": {
"filter": {
"range": {
"@timestamp": {
"gte": "2019-06-04T14:20:37.788Z||-1d",
"lte": "2019-06-04T14:20:37.788Z",
"format": "strict_date_optional_time||epoch_millis"
}
}
}
}
}
}
}
}
},
"condition": {
"type": "script",
"status": "success",
"met": true
},
"transform": {
"type": "script",
"status": "success",
"payload": {
"result": 60256
}
},
"actions": [
{
"id": "Email_Daily_Report",
"type": "email",
"status": "failure",
"error": {
"root_cause": [
{
"type": "http_host_connect_exception",
"reason": "Connect to 10.4.2.26:5601 [/10.4.2.26] failed: Connection timed out (Connection timed out)"
}
],
"type": "http_host_connect_exception",
"reason": "Connect to 10.4.2.26:5601 [/10.4.2.26] failed: Connection timed out (Connection timed out)",
"caused_by": {
"type": "connect_exception",
"reason": "Connection timed out (Connection timed out)"
}
}
}
]
},
"messages":
}

Watcher Code:
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
PUT _xpack/watcher/watch/Daily_Report_Part_Events_per_Product
{
  "trigger": {
    "schedule": {
      "interval": "1d"
    }
  },
    "input": {
      "search": {
        "request": {
          "search_type": "query_then_fetch",
          "indices": [
            "monitoring-loading-last-file-state*",
            "logstash*",
            "monitoring-loading-processed*",
            "monitoring-loading-raw*",
            "monitoring-part-event*"
          ],
          "types": [],
          "body": {
            "size": 0,
            "query": {
              "bool": {
                "filter": {
                  "range": {
                    "@timestamp": {
                      "gte": "{{ctx.trigger.scheduled_time}}||-1d",
                      "lte": "{{ctx.trigger.scheduled_time}}",
                      "format": "strict_date_optional_time||epoch_millis"
                    }
                  }
                }
              }
            }
          }
        }
      }
    },
    "condition": {
      "script": {
        "source": "if (ctx.payload.hits.total > params.threshold) { return true; } return false;",
        "lang": "painless",
        "params": {
          "threshold": -1
        }
      }
    },
    "transform": {
      "script": {
        "source": "HashMap result = new HashMap(); result.result = ctx.payload.hits.total; return result;",
        "lang": "painless",
        "params": {
          "threshold": -1
        }
      }
    },
  "actions": {
    "Email_Daily_Report": {
      "email": {
        "to": "'EMPLOYEE NAME <EMPLOYEE.NAME@COMPANY.com>'",
        "subject" : "[Daily Report] COMPANY Elastic Part Events / Product / Hour",
        "body" : "Attached you will find your automatically generated daily report on COMPANY Part Events / Product / Hour.",
        "attachments": {
          "Part_Events_per_Product_Hour.pdf": {
            "http": {
              "content_type": "application/pdf",
              "request": {
                "method": "POST",
                "headers": {
                  "kbn-xsrf": "reporting"
                },
                "connection_timeout": "300s",
                "read_timeout": "300s",
                "scheme": "https",
                "url": "http://123.123.123.123:5601/api/reporting/generate/printablePdf?jobParams=(browserTimezone:Atlantic%2FReykjavik,layout:(dimensions:(height:785.9732666015625,width:1655.4285888671875),id:preserve_layout),objectType:visualization,relativeUrls:!(%27%2Fapp%2Fkibana%23%2Fvisualize%2Fedit%2F25b2d7a0-830a-11e9-9164-43eefc31c86f%3F_g%3D(refreshInterval:(display:Off,pause:!!f,value:0),time:(from:now-2d,mode:relative,to:now))%26_a%3D(filters:!!((!%27$state!%27:(store:appState),exists:(field:PartEvents.Uuid),meta:(alias:!%27Part%2BEvents!%27,disabled:!!f,index:!%279748b6e0-2ec9-11e9-b8d5-a118b862a945!%27,key:PartEvents.Uuid,negate:!!f,type:exists,value:exists)),(!%27$state!%27:(store:appState),meta:(alias:!!n,disabled:!!f,index:!%279748b6e0-2ec9-11e9-b8d5-a118b862a945!%27,key:LoadingId,negate:!!t,params:(query:0,type:phrase),type:phrase,value:0),query:(match:(LoadingId:(query:0,type:phrase)))),(!%27$state!%27:(store:appState),meta:(alias:!!n,disabled:!!f,index:!%279db343c0-338c-11e9-80e1-1f13c2097a0d!%27,key:PartEvents.EventStartTime,negate:!!f,params:(gte:now-2d,lt:now),type:range,value:!%27now-2d%2Bto%2Bnow!%27),range:(PartEvents.EventStartTime:(gte:now-2d,lt:now)))),linked:!!f,query:(language:lucene,query:!%27!%27),uiState:(spy:!!n,vis:(colors:(!%27CAMERA%2BCAR%2B3.5!%27:%25231F78C1,!%27CAMERA%2BCAR%2B4!%27:%25237EB26D,!%27Part%2BEvents!%27:%2523EF843C),legendOpen:!!t)),vis:(aggs:!!((enabled:!!t,id:!%271!%27,params:(customLabel:!%27Part%2BEvents%2B%2B%252F%2B%2BProduct!%27,json:!%27%257B%2522title%2522:%2522Part%2BEvents%2B%2B%252F%2B%2BProduct%2B%2B%252F%2B%2BHour%2B%2B%255B48%25E2%2580%2593Hours%255D%2522%257D!%27),schema:metric,type:count),(enabled:!!t,id:!%272!%27,params:(customInterval:!%272h!%27,customLabel:Hours,extended_bounds:(),field:PartEvents.EventStartTime,interval:h,min_doc_count:1),schema:segment,type:date_histogram),(enabled:!!t,id:!%273!%27,params:(field:Product,json:!%27!%27,missingBucket:!!f,missingBucketLabel:Missing,order:desc,orderBy:!%271!%27,otherBucket:!!f,otherBucketLabel:Other,size:5),schema:group,type:terms)),params:(addLegend:!!t,addTimeMarker:!!f,addTooltip:!!t,categoryAxes:!!((id:CategoryAxis-1,labels:(filter:!!f,rotate:0,show:!!t,truncate:13),position:bottom,scale:(type:linear),show:!!t,style:(),title:(),type:category)),grid:(categoryLines:!!t,style:(color:%2523eee),valueAxis:ValueAxis-1),legendPosition:top,seriesParams:!!((data:(id:!%271!%27,label:!%27Part%2BEvents%2B%2B%252F%2B%2BProduct!%27),drawLinesBetweenPoints:!!t,mode:stacked,show:true,showCircles:!!t,type:histogram,valueAxis:ValueAxis-1)),times:!!(),type:histogram,valueAxes:!!((id:ValueAxis-1,labels:(filter:!!f,rotate:0,show:!!t,truncate:100),name:LeftAxis-1,position:left,scale:(mode:normal,type:linear),show:!!t,style:(),title:(text:!%27Part%2BEvents%2B%2B%252F%2B%2BProduct!%27),type:value))),title:!%27Part%2BEvents%2B%2B%252F%2B%2BProduct%2B%2B%252F%2BHour%2B%2B%255B48%25E2%2580%2593hour%2BHistory%255D!%27,type:histogram))%27),title:%27Part%20Events%20%20%2F%20%20Product%20%20%2F%20Hour%20%20%5B48%E2%80%93hour%20History%5D%27)&sync"
              }
            }
          }
        }
      }
    }
  },
    "metadata": {
      "name": "[Daily Report]  Part Events / Product / Hour"
    }
}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.