# Creating indices of older days too

**URL:** <https://discuss.elastic.co/t/creating-indices-of-older-days-too/266492>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [March 7, 2021, 11:15pm UTC](https://discuss.elastic.co/t/creating-indices-of-older-days-too/266492 "2021-03-07T23:15:16Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![souji](https://avatars.discourse-cdn.com/v4/letter/s/65b543/32.png) [@souji](https://discuss.elastic.co/u/souji)\
**Post date:** [March 7, 2021, 11:15pm UTC](https://discuss.elastic.co/t/creating-indices-of-older-days-too/266492/1 "2021-03-07T23:15:16Z")

</div>

Hi

```auto
I have deployed filebeat and logstash helm chart in eks.

I am sending logs to amazon es.

I am facing a similar issue https://github.com/uken/fluent-plugin-elasticsearch/issues/482, but in filebeat do we have option like fulentd

Here is the filebeat and logstash configs.

filebeatConfig:
    filebeat.yml: |
      filebeat.inputs:   
      - type: container
        paths:
          - /var/log/containers/*.log
        processors:
        - add_kubernetes_metadata:
            host: ${NODE_NAME}
            matchers:
            - logs_path:
                logs_path: "/var/log/containers/"
        - drop_event.when:
            or:
            - equals:
                kubernetes.container.name: "filebeat"
            - equals:  
                kubernetes.container.name: "logstash"        
      output.logstash:
        host: '${NODE_NAME}'
        hosts: ["logstash.logstash.svc.cluster.local:5044"]
  

Logstash configuration
`
image: "xxxx/devops/logstash-kubernetes/logstash"
imageTag: "7.11.1" 
fullnameOverride: "logstash"
podLabels:
  service: "logstash"
rbac:
  create: true  
  serviceAccountName: logstash 
service:
  type: ClusterIP
  ports:
    - name: beat-me
      port: 5044
      targetPort: 5044
      protocol: TCP
    - name: http
      port: 8080
      targetPort: 8080
      protocol: TCP
persistence:
  enabled: true

logstashPipeline:
  logstash.conf: |
    input {
      beats {
        port => 5044
      }
    }
    filter {
      dissect {
        mapping => {
            "message" => "%{timestamp} | %{level} | %{thread} | %{category} | %{message}"
        }
      }
      mutate {
        remove_field => ["[kubernetes][annotations]", "headers", "ecs", "[agent][ephemeral_id]", "[agent][id]", "[agent][version]", "[container][id]","[container][runtime]","[input][type]", "[kubernetes][pod][uid]", "[log][file][path]", "[log][offset]","tags","[agent][hostname]","[agent][name]","[agent][type]","[host][name]","[kubernetes][labels][pod-template-hash]","[kubernetes][labels][service_istio_io/canonical-revision]","[kubernetes][labels][pod-template-hash]","[kubernetes][labels][istio_io/rev]","[kubernetes][container][image]","[kubernetes][labels][security_istio_io/tlsMode]","[kubernetes][labels][service_istio_io/canonical-name]","[kubernetes][labels][heritage]" ]
      }
      mutate {
        add_field => {
          "cluster_name" => "xxx"
          "environment" => "xxx"
          "region" => "us-west-2"
        }
      }
    }
    output {
      amazon_es {
        hosts => ["vpc-xxxxx"]
        region => "us-east-1"
        port => 443
        protocol => "https"
        index => "%{cluster_name}-%{region}-%{environment}-%{[kubernetes][labels][app]}-%{+YYYY.MM.dd}"
      }
      stdout { codec => rubydebug }
    }

I am creating daily indices, If I install the filebeat and logstash on mar 07 2021, it's still creating the indices of older dates up to week back dates. 
green open xxxx_ddog-agent_2021.03.05 CHCWjCmKRYq1uoKcpmdEuQ 1 2 61550 0 70.1mb 23.4mb
green open xxx_ddog-agent_2021.03.04 vF7J2MeWS8m-wFrZc4iydg 1 2 121822 0 136.8mb 45.7mb
green open rokumesh-merge-request-test_merge-request-test_ddog-agent_2021.03.07 YeHpPqubTii_OS_vlwOqvg 1 2 11247 0 14.8mb 5.1mb
green open xxxx_ddog-agent_2021.03.06 gYxCtPiMSfi7CDxqsxcLGw 1 2 11963 0 14.9mb 5mb
green open xxxx_ddog-agent_2021.03.03 FnscWP4GTjq7y4_p8CgdNg 1 2 122359 0 134.8mb 45mb
green open xxx_ddog-agent_2021.03.02 wtpJeht_RnOlgSKP5LdOKQ 1 2 32883 0 33.4mb 11.2mb

```

Thanks in adavce.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 4, 2021, 11:15pm UTC](https://discuss.elastic.co/t/creating-indices-of-older-days-too/266492/2 "2021-04-04T23:15:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.