Cadvisor <=> Elk timestamp issue


(Tom van den Berg) #1

When cadvisor feeds ELK with data I consistently get a strack trace when I want to visualize the data in e.g. a linechart.
Version used: Elasticsearch 5.0.0, Logstash 5.0.0, and Kibana 5.0.0.
See https://github.com/spujadas/elk-docker/issues/90

...
elk    | ==> /var/log/elasticsearch/elasticsearch.log <==
elk    | [2016-11-20 19:33:22,009][DEBUG][action.fieldstats        ] [Steve Rogers] [.kibana][0], node[5RAVyDN7QfyI67NdmbtO1g], [P], v[2], s[STARTED], a[id=cu_9QYBCQYSKsuKprDwzwg]: failed to execute [org.elasticsearch.action.fieldstats.FieldStatsRequest@68fe0de5]
elk    | RemoteTransportException[[Steve Rogers][172.19.0.5:9300][indices:data/read/field_stats[s]]]; nested: IllegalArgumentException[field [container_stats.timestamp] doesn't exist];
elk    | Caused by: java.lang.IllegalArgumentException: field [container_stats.timestamp] doesn't exist
elk    |        at org.elasticsearch.action.fieldstats.TransportFieldStatsTransportAction.shardOperation(TransportFieldStatsTransportAction.java:166)
elk    |        at org.elasticsearch.action.fieldstats.TransportFieldStatsTransportAction.shardOperation(TransportFieldStatsTransportAction.java:54)
elk    |        at org.elasticsearch.action.support.broadcast.TransportBroadcastAction$ShardTransportHandler.messageReceived(TransportBroadcastAction.java:282)
elk    |        at org.elasticsearch.action.support.broadcast.TransportBroadcastAction$ShardTransportHandler.messageReceived(TransportBroadcastAction.java:278)
elk    |        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:77)
elk    |        at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
elk    |        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
elk    |        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
elk    |        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
elk    |        at java.lang.Thread.run(Thread.java:745)
elk    |
...

I can see that the data is received by ELK. In Kibana I created an index for container_stats.timestamp.
Also, need to use sebp/elk:es241_l240_k461, since cadvisor does not work with latest.

Docker compose files:

cadvisor-elastic.yml

version: '2'

services:
 cadvisor:
  image: google/cadvisor
  container_name: cadvisor
  ports:
  - "8180:8080"
  volumes:
  - /:/rootfs:ro
  - /var/run:/var/run:rw
  - /sys:/sys:ro
  - /var/lib/docker/:/var/lib/docker:ro
  restart: always
  command: -storage_driver="elasticsearch" -storage_driver_es_host="http://10.10.10.11:9200"

elk.yml

version: '2'

services:
 elk:
  image: sebp/elk:es241_l240_k461
  container_name: elk
  ports:
  - "5601:5601"
  - "9200:9200"
  - "5044:5044"

(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.