Hi there,
I am trying to setup metricbeat inside a local docker environment to write the collected data into an AWS Elastic Search Service. The Service runs and authentication is available via Basic Authentication. Here is my docker setup:
docker-stack.yml
metricbeat:
image: docker.elastic.co/beats/metricbeat:6.4.0
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- path/to/metricbeat.yml:/usr/share/metricbeat/metricbeat.yml
- path/to/docker.yml:/usr/share/metricbeat/modules.d/docker.yml
user: root
metricbeat.yml
metricbeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.dashboards.enabled: true
setup.template.settings:
index.number_of_shards: 1
index.codec: best_compression
setup.kibana:
ssl.enabled: true
protocol: "https"
host: "host.on.amazon.com:443"
path: "/_plugin/kibana"
username: "yyy"
password: "xxx"
output.elasticsearch:
hosts: ["https://host.on.amazon.com:443"]
username: "yyy"
password: "xxx"
docker.yml
- module: docker
metricsets:
- "container"
- "cpu"
- "diskio"
- "healthcheck"
- "info"
- "memory"
- "network"
hosts: ["unix:///var/run/docker.sock"]
period: 10s
enabled: true
When starting the docker container, I get the following logs:
docker | 2020-11-19T12:10:28.317Z INFO instance/beat.go:544 Home path: [/usr/share/metricbeat] Config path: [/usr/share/metricbeat] Data path: [/usr/share/metricbeat/data] Logs path: [/usr/share/metricbeat/logs]
docker | 2020-11-19T12:10:28.321Z INFO instance/beat.go:551 Beat UUID: ae7c955a-c679-4c3b-af0f-95c0ba3a9be1
docker | 2020-11-19T12:10:28.321Z INFO [seccomp] seccomp/seccomp.go:116 Syscall filter successfully installed
docker | 2020-11-19T12:10:28.321Z INFO [beat] instance/beat.go:768 Beat info {"system_info": ...}
docker | 2020-11-19T12:10:28.321Z INFO [beat] instance/beat.go:777 Build info {"system_info": ...}
docker | 2020-11-19T12:10:28.321Z INFO [beat] instance/beat.go:780 Go runtime info {"system_info": ...}
docker | 2020-11-19T12:10:28.322Z INFO [beat] instance/beat.go:784 Host info {"system_info": ...}
docker | 2020-11-19T12:10:28.322Z INFO [beat] instance/beat.go:813 Process info {"system_info": ...}
docker | 2020-11-19T12:10:28.322Z INFO instance/beat.go:273 Setup Beat: metricbeat; Version: 6.4.0
docker | 2020-11-19T12:10:28.322Z INFO elasticsearch/client.go:163 Elasticsearch url: https://host.on.amazon.com:443
docker | 2020-11-19T12:10:28.322Z INFO pipeline/module.go:98 Beat name: f9f8815c3442
docker | 2020-11-19T12:10:28.323Z INFO [monitoring] log/log.go:114 Starting metrics logging every 30s
docker | 2020-11-19T12:10:28.323Z INFO elasticsearch/client.go:163 Elasticsearch url: https://host.on.amazon.com:443
docker | 2020-11-19T12:10:40.612Z INFO elasticsearch/client.go:708 Connected to Elasticsearch version 7.8.0
docker | 2020-11-19T12:10:40.612Z INFO kibana/client.go:113 Kibana url: https://host.on.amazon.com:443/_plugin/kibana
docker | 2020-11-19T12:10:47.202Z INFO [monitoring] log/log.go:149 Total non-zero metrics {"monitoring": ...}
docker | 2020-11-19T12:10:47.202Z INFO [monitoring] log/log.go:150 Uptime: 18.900561226s
docker | 2020-11-19T12:10:47.202Z INFO [monitoring] log/log.go:127 Stopping metrics logging.
docker | 2020-11-19T12:10:47.203Z INFO instance/beat.go:358 metricbeat stopped.
docker | 2020-11-19T12:10:47.203Z ERROR instance/beat.go:743 Exiting: Error importing Kibana dashboards: fail to import the dashboards in Kibana: Error importing directory /usr/share/metricbeat/kibana: Failed to import index-pattern: Failed to load directory /usr/share/metricbeat/kibana/6/index-pattern:
docker | error loading /usr/share/metricbeat/kibana/6/index-pattern/metricbeat.json: parsing kibana response: unexpected end of JSON input. Response:
docker | 2020-11-19T12:10:09.031Z INFO instance/beat.go:544 Home path: [/usr/share/metricbeat] Config path: [/usr/share/metricbeat] Data path: [/usr/share/metricbeat/data] Logs path: [/usr/share/metricbeat/logs]
docker | 2020-11-19T12:10:09.037Z INFO instance/beat.go:551 Beat UUID: 59aeaafb-82f9-4b29-b791-de3276b6eba5
docker | 2020-11-19T12:10:09.038Z INFO [seccomp] seccomp/seccomp.go:116 Syscall filter successfully installed
docker | Exiting: Error importing Kibana dashboards: fail to import the dashboards in Kibana: Error importing directory /usr/share/metricbeat/kibana: Failed to import index-pattern: Failed to load directory /usr/share/metricbeat/kibana/6/index-pattern:
docker | error loading /usr/share/metricbeat/kibana/6/index-pattern/metricbeat.json: parsing kibana response: unexpected end of JSON input. Response:
docker | 2020-11-19T12:10:09.038Z INFO [beat] instance/beat.go:768 Beat info {"system_info": ...}
docker | 2020-11-19T12:10:09.038Z INFO [beat] instance/beat.go:777 Build info {"system_info": ...}
docker | 2020-11-19T12:10:09.038Z INFO [beat] instance/beat.go:780 Go runtime info {"system_info": ...}
docker | 2020-11-19T12:10:09.038Z INFO [beat] instance/beat.go:784 Host info {"system_info": ...}
docker | 2020-11-19T12:10:09.039Z INFO [beat] instance/beat.go:813 Process info {"system_info": ...}
docker | 2020-11-19T12:10:09.039Z INFO instance/beat.go:273 Setup Beat: metricbeat; Version: 6.4.0
docker | 2020-11-19T12:10:09.039Z INFO elasticsearch/client.go:163 Elasticsearch url: https://host.on.amazon.com:443
docker | 2020-11-19T12:10:09.039Z INFO pipeline/module.go:98 Beat name: a6c488013e48
docker | 2020-11-19T12:10:09.039Z INFO [monitoring] log/log.go:114 Starting metrics logging every 30s
docker | 2020-11-19T12:10:09.039Z INFO elasticsearch/client.go:163 Elasticsearch url: https://host.on.amazon.com:443
docker | 2020-11-19T12:10:09.220Z INFO elasticsearch/client.go:708 Connected to Elasticsearch version 7.8.0
docker | 2020-11-19T12:10:09.220Z INFO kibana/client.go:113 Kibana url: https://host.on.amazon.com:443/_plugin/kibana
docker | 2020-11-19T12:10:22.004Z INFO [monitoring] log/log.go:149 Total non-zero metrics {"monitoring": ...}
docker | 2020-11-19T12:10:22.004Z INFO [monitoring] log/log.go:150 Uptime: 12.990061471s
docker | 2020-11-19T12:10:22.004Z INFO [monitoring] log/log.go:127 Stopping metrics logging.
docker | 2020-11-19T12:10:22.004Z INFO instance/beat.go:358 metricbeat stopped.
docker | 2020-11-19T12:10:22.004Z ERROR instance/beat.go:743 Exiting: Error importing Kibana dashboards: fail to import the dashboards in Kibana: Error importing directory /usr/share/metricbeat/kibana: Failed to import index-pattern: Failed to load directory /usr/share/metricbeat/kibana/6/index-pattern:
docker | error loading /usr/share/metricbeat/kibana/6/index-pattern/metricbeat.json: parsing kibana response: unexpected end of JSON input. Response:
docker | Exiting: Error importing Kibana dashboards: fail to import the dashboards in Kibana: Error importing directory /usr/share/metricbeat/kibana: Failed to import index-pattern: Failed to load directory /usr/share/metricbeat/kibana/6/index-pattern:
docker | error loading /usr/share/metricbeat/kibana/6/index-pattern/metricbeat.json: parsing kibana response: unexpected end of JSON input. Response:
Setting read_only_allow_delete to false did not ne trick. What else can I do?
Thanks for any help!