Prevent filebeat-version-yyyy-mm-dd index from ingesting

Elastic Stack Elasticsearch
1

On daily basis i am seeing indexes with name filebeat-7.17.7-yyyy-mm-dd are creating. This is eating lot of space i have to delete them manually.

  1. I Have seen options like to create entry in ES which will not allow auto index => if i enable this then ILM will not work.

Is there any better option to stop this entirely?

2

Can you stop them at the source?

3

actually i am not seeing anything about it on filebeat.yml file.

4

What does your filebeat.yml looks like?

Is this the only filebeat you have?

If you do not want the data you need to stop it in the source.

5

here is the file.

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/*/*/*/*.log
  fields:
          type: demo_logs
# ---------------------------- Elasticsearch Output ----------------------------
#output.elasticsearch:
  # Array of hosts to connect to.
  #hosts: ["ip:9200"]

# ------------------------------ Logstash Output -------------------------------
output.logstash:
  # The Logstash hosts
        hosts: ["ip:5044"]
# ================================= Processors =================================
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~
  - drop_event:
          when:
                  contains:
                          message: INFO

rest of fields are commented out.

6

You are sending your logs to Logstash, so you need to look check and share your logstash configuration.

Also, you didn't say if you are running other filebeats or just this one.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.