Delay in logs in filebeat for only one index

Elastic Stack Beats
1

Hi!
I have filebeat with multiple indexes and one particular index is causing the issue and other indexes are sending logs without delayi.e., Sending logs to Kibana is getting delayed by 6 hours for the particular index.

`filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /mnt/test*.log
  json.message_key: msg
  json.keys_under_root: true
  json.ignore_decoding_error: true
  document_type: api
  fields:
    type: test-api
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /mnt/test1*.log
  json.message_key: msg
  json.keys_under_root: true
  json.ignore_decoding_error: true
  document_type: api
  fields:
    type: test1-api
output.elasticsearch:
  bulk_max_size: 7000
  workers: 10
  protocol: "https"
  template.name: filebeat
  template.path: filebeat.template.json
  index: "ecs-%{[fields.type]:other}-{+yyyy.MM.dd}"
setup.template.name: filebeat
setup.template.pattern: "filebeat-*"
setup.ilm.enabled: false
setup.ilm.check_exists: false
setup.template.settings:
  index.number_of_shards: 1
  index.number_of_replicas: 1
  index.refresh_interval: "30s"
`

test1-api is generating large sized logs whereas test-api is relatively smaller. any suggestions to resolve the issue
3

test1-api is generating large sized logs whereas test-api is relatively smaller. any suggestions to resolve the issue

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.