Cannon simple change index name in filebeat

[patsevanton (Anton Patsev)]

ELK - 7.0.1
Filebeat - 7.0.1

cat /etc/filebeat/filebeat.yml

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - '/var/lib/docker/containers/*/*.log'
  json.keys_under_root: true
  json.message_key: log
  encoding: utf-8
  document_type: docker

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1

setup.kibana:
  host: "localhost:5601"

output.elasticsearch:
  hosts: ["localhost:9200"]
  protocol: "http"
  index: "api-access-%{+yyyy.MM.dd}"

setup.template:
  name: 'api-access'
  pattern: 'api-access-*'
  enabled: false

processors:
- decode_json_fields:
    fields: ["log"]
    target: ""
    overwrite_keys: true
- add_docker_metadata: ~

filebeat -c /etc/filebeat/filebeat.yml export config

filebeat:
  config:
    modules:
      path: /etc/filebeat/modules.d/*.yml
      reload:
        enabled: false
  inputs:
  - document_type: docker
    enabled: true
    encoding: utf-8
    json:
      keys_under_root: true
      message_key: log
    paths:
    - /var/lib/docker/containers/*/*.log
    type: log
output:
  elasticsearch:
    hosts:
    - localhost:9200
    index: api-access-%{+yyyy.MM.dd}
    protocol: http
path:
  config: /etc/filebeat
  data: /var/lib/filebeat
  home: /usr/share/filebeat
  logs: /var/log/filebeat
processors:
- decode_json_fields:
    fields:
    - log
    overwrite_keys: true
    target: ""
- add_docker_metadata: null
setup:
  kibana:
    host: localhost:5601
  template:
    enabled: false
    name: api-access
    pattern: api-access-*
    settings:
      index:
        number_of_shards: 1

But in Index management create filebeat-7.1.0-2019.05.21-000001.

How simple change index name in filebeat?

[jsoriano (Jaime Soriano)]

Hi @patsevanton,

If ILM (Index Lifecycle Management) is used, the index cannot be customized. This is a known limitation and there is an open issue to document it better.

In your configuration you are not enabling ILM, but it is activated by default if your Elasticsearch cluster supports it.

To confirm that it is what is happening in your case, please check logs for messages from index-management like this one:

INFO [index-management] idxmgmt/std.go:361 Set setup.template.name to '{filebeat-7.0.1 {now/d}-000001}' as ILM is enabled.

You can read more about ILM configuration here: https://www.elastic.co/guide/en/beats/filebeat/7.0/ilm.html

[patsevanton (Anton Patsev)]

Thanks!

Worked config:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - '/var/lib/docker/containers/*/*.log'
  json.keys_under_root: true
  json.message_key: log
  encoding: utf-8
  document_type: docker

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1

setup.kibana:
  host: "localhost:5601"

output.elasticsearch:
  hosts: ["localhost:9200"]
  protocol: "http"
  index: "api-access-%{+yyyy.MM.dd}"

setup.template:
  name: 'api-access'
  pattern: 'api-access-*'
  enabled: false

processors:
- decode_json_fields:
    fields: ["log"]
    target: ""
    overwrite_keys: true
- add_docker_metadata: ~

setup.ilm.enabled: false