When cadvisor feeds ELK with data I consistently get a strack trace when I want to visualize the data in e.g. a linechart.
Version used: Elasticsearch 5.0.0, Logstash 5.0.0, and Kibana 5.0.0.
See https://github.com/spujadas/elk-docker/issues/90
...
elk | ==> /var/log/elasticsearch/elasticsearch.log <==
elk | [2016-11-20 19:33:22,009][DEBUG][action.fieldstats ] [Steve Rogers] [.kibana][0], node[5RAVyDN7QfyI67NdmbtO1g], [P], v[2], s[STARTED], a[id=cu_9QYBCQYSKsuKprDwzwg]: failed to execute [org.elasticsearch.action.fieldstats.FieldStatsRequest@68fe0de5]
elk | RemoteTransportException[[Steve Rogers][172.19.0.5:9300][indices:data/read/field_stats[s]]]; nested: IllegalArgumentException[field [container_stats.timestamp] doesn't exist];
elk | Caused by: java.lang.IllegalArgumentException: field [container_stats.timestamp] doesn't exist
elk | at org.elasticsearch.action.fieldstats.TransportFieldStatsTransportAction.shardOperation(TransportFieldStatsTransportAction.java:166)
elk | at org.elasticsearch.action.fieldstats.TransportFieldStatsTransportAction.shardOperation(TransportFieldStatsTransportAction.java:54)
elk | at org.elasticsearch.action.support.broadcast.TransportBroadcastAction$ShardTransportHandler.messageReceived(TransportBroadcastAction.java:282)
elk | at org.elasticsearch.action.support.broadcast.TransportBroadcastAction$ShardTransportHandler.messageReceived(TransportBroadcastAction.java:278)
elk | at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:77)
elk | at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
elk | at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
elk | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
elk | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
elk | at java.lang.Thread.run(Thread.java:745)
elk |
...
I can see that the data is received by ELK. In Kibana I created an index for container_stats.timestamp.
Also, need to use sebp/elk:es241_l240_k461, since cadvisor does not work with latest.
Docker compose files:
cadvisor-elastic.yml
version: '2'
services:
cadvisor:
image: google/cadvisor
container_name: cadvisor
ports:
- "8180:8080"
volumes:
- /:/rootfs:ro
- /var/run:/var/run:rw
- /sys:/sys:ro
- /var/lib/docker/:/var/lib/docker:ro
restart: always
command: -storage_driver="elasticsearch" -storage_driver_es_host="http://10.10.10.11:9200"
elk.yml
version: '2'
services:
elk:
image: sebp/elk:es241_l240_k461
container_name: elk
ports:
- "5601:5601"
- "9200:9200"
- "5044:5044"