Configure output Index name with kubernetes metadata field


I would like to have the kubernetes namespace in the index name, so I tried like this :

  hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']  
  index: "%{[kubernetes.namespace]}-filebeat-%{[beat.version]}-%{+yyyy.MM.dd}"

With this configuration, no index at all is created even if the metadata are present in the documents :

018-06-19T21:30:39.424Z	DEBUG	[publish]	pipeline/processor.go:275	Publish event: {
  "@timestamp": "2018-06-19T21:30:38.483Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "doc",
    "version": "6.2.4"
  "message": "      }",
  "source": "/var/lib/docker/containers/af869caadf53af083955fee5e506fd2ab6658ea7e285c12a8c724462b347494c/af869caadf53af083955fee5e506fd2ab6658ea7e285c12a8c724462b347494c-json.log",
  "offset": 535365,
  "prospector": {
    "type": "docker"
  "docker": {
    "container": {
      "id": "af869caadf53af083955fee5e506fd2ab6658ea7e285c12a8c724462b347494c",
      "name": "k8s_filebeat_filebeat-958s5_default_01d6839d-7408-11e8-9939-42010a8e0061_0",
      "image": "",
      "labels": {
        "": "filebeat-958s5",
        "org.label-schema.schema-version": "= 1.0 Base Image     org.label-schema.vendor=CentOS     org.label-schema.license=GPLv2",
        "": "2237901ac21fcc16fb9bc2b69ccbb43dfe3b22916860bd0267da007c649ae478",
        "": "/dev/termination-log",
        "": "File",
        "": "30",
        "": "filebeat",
        "io.kubernetes.container.logpath": "/var/log/pods/01d6839d-7408-11e8-9939-42010a8e0061/filebeat_0.log",
        "io.kubernetes.pod.namespace": "default",
        "io.kubernetes.pod.uid": "01d6839d-7408-11e8-9939-42010a8e0061",
        "": "629c5c7d",
        "io.kubernetes.docker.type": "container",
        "": "0"
  "kubernetes": {
    "pod": {
      "name": "filebeat-958s5"
    "node": {
      "name": "gke-cluster-1-default-pool-8f121b79-qrxv"
    "namespace": "default",
    "labels": {
      "pod-template-generation": "2",
      "app": "filebeat",
      "controller-revision-hash": "3319485886"
    "container": {
      "name": "filebeat"
  "beat": {
    "name": "filebeat-958s5",
    "hostname": "filebeat-958s5",
    "version": "6.2.4"
  "stream": "stderr"

Hi @orgoz,

Your config looks good to me. Did you enable automatic index creation in Elasticsearch? Do you get any errors? Log output would be interesting here.

Best regards

Hello @exekias,

After trying two more times to send you logs, I just manage to get it running... I don't why, I have probably fix an error in my config file...

Sorry for the inconvenience

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.