# Filebeat sends logs with delay

**URL:** <https://discuss.elastic.co/t/filebeat-sends-logs-with-delay/311038>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [July 30, 2022, 11:16am UTC](https://discuss.elastic.co/t/filebeat-sends-logs-with-delay/311038 "2022-07-30T11:16:03Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![behzad\_alipoor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/behzad_alipoor/32/109054_2.png) [@behzad\_alipoor](https://discuss.elastic.co/u/behzad_alipoor)\
**Post date:** [July 30, 2022, 11:16am UTC](https://discuss.elastic.co/t/filebeat-sends-logs-with-delay/311038/1 "2022-07-30T11:16:03Z")

</div>

I send logs from filebeat to elasticsearch directly . the service that i use generates logs every second . most of the time it works properly but sometimes the logs reach to elastic after 15 minutes or later and I don't know why .  
this is my config in filebet :

filebeat. inputs:  
-type: log  
id: my-filestream-id  
enabled: true  
backoff: 0.5s  
scan frequency: 15  
close inactive: 10m  
ignore older: 30m  
flush.min events: 0  
paths:  
- /var/log/\*.log

---

<div class="post-metadata">

**Author:** ![TiagoQueiroz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tiagoqueiroz/32/107061_2.png) [@TiagoQueiroz](https://discuss.elastic.co/u/TiagoQueiroz)\
**Post date:** [August 1, 2022, 8:51am UTC](https://discuss.elastic.co/t/filebeat-sends-logs-with-delay/311038/2 "2022-08-01T08:51:00Z")

</div>

Hi @behzad_alipoor,

There are some issues with your configuration, aside the fact it lost its indentation, so it's a bit hard to reason about it. Whenever posting configuration/code blocks, use the "Preformatted Text" option.

Logs delayed to be searchable in Kibana/Elasticsearch might indicate the Elasticsearch nodes are overloaded and not being able to index the new documents quick enough. Do you have any idea of our throughput when you start noticing a delay on ingestion?

You mentioned the service generates logs almost every second, are those logs added to the same file? Is there any log rotation strategy?

Assuming your YAML config is:

```auto
filebeat. inputs:
  - type: log
    id: my-filestream-id
    enabled: true
    backoff: 0.5s
    scan frequency: 15
    close inactive: 10m
    ignore older: 30m
    flush.min events: 0
    paths:
      - /var/log/*.log

```

`close inactive` and `ignore older` are misspelled, the words are separated by a `_`, so the correct format is:

```auto
    close_inactive: 10m
    ignore_older: 30m

```

`flush.min events` is not part of the log input configuration, so it's being ignored. There is a [`flush.min_events`](https://github.com/elastic/beats/blob/main/filebeat/filebeat.reference.yml#L1296) configuration in the [memory queue](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-filestream.html), is that what you meant to set?

The default configurations generally provide a near-real-time ingestion of the files, unless you have very high throughput.

One last note, the `log` input has been deprecated a while ago, it's recommended to use the [`filestream`](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-filestream.html) input now, it has got more features and better performance.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 29, 2022, 10:51am UTC](https://discuss.elastic.co/t/filebeat-sends-logs-with-delay/311038/3 "2022-08-29T10:51:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.