Filebeat custom index name without logstash

Elastic Stack Beats
1

I want to change name of the index.

https://www.elastic.co/guide/en/beats/filebeat/current/elasticsearch-output.html#index-option-es
I'm referring above link. following is the filebeat.yml file. I'm not seeing any index in kibana.

 filebeat.inputs:
    - type: log
      paths:
        - /var/www/myapp/**/runtime/debug/*.data
      fields:
        log_type: debugdata

      tags: ["myapp-logs"]
      json.keys_under_root: true
      json.add_error_key: true
      json.message_key: log

    setup.template.name: "debugdata"
    setup.template.pattern: "debugdata-*"

    cloud.id: ${ELASTIC_CLOUD_ID}
    cloud.auth: ${ELASTIC_CLOUD_AUTH}

    output.elasticsearch:
      hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
      username: ${ELASTICSEARCH_USERNAME}
      password: ${ELASTICSEARCH_PASSWORD}
      index: "%{[fields.log_type]}-%{[beat.version]}-%{+yyyy.MM.dd}"

Attached is the list of indices.

34%20PM
Screen Shot 2018-12-21 at 11.48.34 PM.png531×505 26.4 KB

what changes do i need to make to create custom index name.?
Expecting debugdata-7.0.0-alpha1-2018-20-21

2

Wouldn't you need to specify the index name under the ES output?

output.elasticsearch:
hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
username: ${ELASTICSEARCH_USERNAME}
password: ${ELASTICSEARCH_PASSWORD}
index: "debugdata-%{[beat.version]}-%{+yyyy.MM.dd}"

3

I couldn't get what you are trying to explain. It's under ES output, right? Can you show the example?

4

Sorry, I misread your filebeat.yml file. I now see you have added a custom field and are calling that field to be used for the index name. To me everything looks good in the config, not sure if you need to set it to overwrite the existing template

setup.template.overwrite: true

5

@rayi7786 tried setting setup.template.overwrite: true, no luck with it. Is there a way to debug the issue?

6

Any suggestions?

7

@Christian_Dahlqvist sir, could you please help here?

8

The indentation looks off.

filebeat.inputs:
    - type: log
      paths:
        - /var/www/myapp/**/runtime/debug/*.data
      fields:
        log_type: debugdata

      tags: ["myapp-logs"]
      
      json.keys_under_root: true
      json.add_error_key: true
      json.message_key: log

setup.template.name: "debugdata"
setup.template.pattern: "debugdata-*"

cloud.id: ${ELASTIC_CLOUD_ID}
cloud.auth: ${ELASTIC_CLOUD_AUTH}

output.elasticsearch:
  hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
  username: ${ELASTICSEARCH_USERNAME}
  password: ${ELASTICSEARCH_PASSWORD}
  index: "%{[fields.log_type]}-%{[beat.version]}-%{+yyyy.MM.dd}"

fixing the indentation I think it should work.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.