Fix kibana dashboards for filebeat aws module

I am very excited about the new dashboards feature for filebeat. I am trying to use custom indicies to differentiate between vpc flow logs, elb access logs, and cloudtrail logs. The issue is that the dashboards do not load properly. I am referencing the dashboards in my filebeat config like so

  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
  index.number_of_shards: 1 "localhost:5601"
setup.kibana.protocol: "https"
setup.kibana.username: "username"
setup.kibana.password: "password"
setup.dashboards.enabled: true ${path.config}/kibana
setup.ilm.enabled: false
logging.level: debug
  hosts: ["localhost:9243"]
  protocol: "https"
  username: "username"
  password: "password"
  bulk_max_size: 3200
  worker: 16
  - index: "cloudtrail-%{[agent.version]}-%{+yyyy.MM.dd}"
      event.dataset: "aws.cloudtrail"
  - index: "elb-%{[agent.version]}-%{+yyyy.MM.dd}"
      event.dataset: "aws.elb"
  - index: "vpc-%{[agent.version]}-%{+yyyy.MM.dd}"
      event.dataset: "aws.vpc"
  - add_fields:
      target: SIEM
        environment: ${ENVIRONMENT} 

In the ${path.config}/kibana directory I have custom json files for building the dashboards. Here is an example of a vpc flow log visualization.

            "attributes": {
                "description": "",
                "kibanaSavedObjectMeta": {
                    "searchSourceJSON": "{\"filter\": [], \"query\": {\"query\": \"\", \"language\": \"kuery\"}}"
                "title": "VPC Flow Top IP Addresses [Filebeat AWS]",
                "uiStateJSON": "{}",
                "version": 1,
                "visState": "{\"aggs\": [], \"type\": \"metrics\", \"title\": \"VPC Flow Top IP Addresses [Filebeat AWS]\", \"params\": {\"show_legend\": 1, \"gauge_style\": \"half\", \"gauge_width\": 10, \"default_index_pattern\": \"vpc-*\", \"id\": \"61ca57f0-469d-11e7-af02-69e470af7417\", \"series\": [{\"type\": \"timeseries\", \"split_mode\": \"terms\", \"label\": \"IP address\", \"stacked\": \"none\", \"series_index_pattern\": \"vpc-*\", \"override_index_pattern\": 1, \"chart_type\": \"line\", \"axis_position\": \"right\", \"terms_field\": \"source.ip\", \"filter\": {\"query\": \" : \\\"vpcflow\\\" \", \"language\": \"kuery\"}, \"fill\": 0.5, \"id\": \"61ca57f1-469d-11e7-af02-69e470af7417\", \"series_drop_last_bucket\": 0, \"color\": \"rgba(115,216,255,1)\", \"separate_axis\": 0, \"point_size\": 1, \"metrics\": [{\"type\": \"count\", \"id\": \"61ca57f2-469d-11e7-af02-69e470af7417\"}, {\"field\": \"61ca57f2-469d-11e7-af02-69e470af7417\", \"id\": \"40c52370-3e87-11ea-9067-cf383a4ea3b3\", \"type\": \"cumulative_sum\"}], \"formatter\": \"number\", \"line_width\": 1, \"color_rules\": [{\"id\": \"42e14220-3e86-11ea-9067-cf383a4ea3b3\"}], \"terms_order_by\": \"61ca57f2-469d-11e7-af02-69e470af7417\"}], \"default_timefield\": \"@timestamp\", \"axis_formatter\": \"number\", \"gauge_color_rules\": [{\"id\": \"2b29c940-3e86-11ea-9067-cf383a4ea3b3\"}], \"interval\": \"\", \"isModelInvalid\": false, \"bar_color_rules\": [{\"id\": \"cc6d5070-3e85-11ea-9067-cf383a4ea3b3\"}], \"legend_position\": \"bottom\", \"axis_scale\": \"normal\", \"axis_min\": \"0\", \"time_field\": \"\", \"gauge_inner_width\": 10, \"type\": \"top_n\", \"background_color_rules\": [{\"id\": \"29527130-3e86-11ea-9067-cf383a4ea3b3\"}], \"axis_position\": \"left\", \"pivot_id\": \"user_agent.original\", \"show_grid\": 1, \"index_pattern\": \"vpc-*\", \"pivot_type\": \"string\"}}"
            "id": "75853f20-4484-11ea-ad63-791a5dc86f10",
            "migrationVersion": {
                "visualization": "7.3.1"
            "references": [],
            "type": "visualization",
            "updated_at": "2020-02-03T16:42:58.806Z",
            "version": "WzYzNzcsMV0="

The issue is that when the dashboards are generated I run into the following issue

I would love to know how to point the dashboards to the custom indicies. Any help would be greatly appreciated.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.