Hi Everyone.
I need to set ILM for some beat services.
As you know saving beats docs from the index changed to Datastream in ELK version 8.
I could enable index rollup for heartbeat From Stack Management > Index Lifecycle Policies by these settings:
-
Maximum primary shard size: 10GB
-
Maximum age: 1d
Now heartbeat indices are separated daily.
I applied the same settings and parameters for filebeat and metricbeat but it didn't work.
This is the filebeat yml configuration:
###################### Filebeat Configuration Example #########################
# ============================== Filebeat inputs ===============================
filebeat.inputs:
- type: filestream
# Unique ID among all inputs, an ID is required.
id: my-filestream-id
# Change to true to enable this input configuration.
enabled: false
# Paths that should be crawled and fetched. Glob based paths.
paths:
- /var/log/*.log
#- c:\programdata\elasticsearch\logs\*
# ============================== Filebeat modules ==============================
filebeat.config.modules:
# Glob pattern for configuration loading
path: ${path.config}/modules.d/*.yml
# Set to true to enable config reloading
reload.enabled: false
# Period on which files under path should be checked for changes
#reload.period: 10s
# ======================= Elasticsearch template setting =======================
setup.template.settings:
index.number_of_shards: 1
#index.codec: best_compression
#_source.enabled: false
setup.dashboards.index: "filebeat-*"
#setup.template.enabled: false
output.elasticsearch.index: "filebeat-gameplatform-%{[agent.version]}"
setup.template.name: "filebeat-gameplatform-%{[agent.version]}"
setup.template.pattern: "filebeat-gameplatform-%{[agent.version]}"
setup.template.overwrite: false
setup.template.fields: "/etc/filebeat/fields-gameplatform.yml"
#setup.ilm.enabled: true
# ================================== KIbana===================================
setup.kibana:
host: "https://********:443"
# ================================== Outputs ===================================
# Configure what output to use when sending the data collected by the beat.
# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["https://************:443"]
# Protocol - either `http` (default) or `https`.
#protocol: "https"
# Authentication credentials - either API key or username/password.
#api_key: "id:api_key"
username: "*******"
password: "********"
# ================================= Processors =================================
processors:
- add_host_metadata:
when.not.contains.tags: forwarded
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
and the same Elasticsearch template setting for metricbeat.
could these settings cause a conflict or overwrite on ILM settings??