# How to push logs to elasticsearch in filebeat?

**URL:** <https://discuss.elastic.co/t/how-to-push-logs-to-elasticsearch-in-filebeat/218501>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 10, 2020, 2:55am UTC](https://discuss.elastic.co/t/how-to-push-logs-to-elasticsearch-in-filebeat/218501 "2020-02-10T02:55:03Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![111289](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111289/32/62327_2.png) [@111289](https://discuss.elastic.co/u/111289)\
**Post date:** [February 10, 2020, 2:55am UTC](https://discuss.elastic.co/t/how-to-push-logs-to-elasticsearch-in-filebeat/218501/1 "2020-02-10T02:55:03Z")

</div>

hear is my filebeat.yml

```auto
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - ../typescript/rate-limit-test/logs/*.log
  json.message_key: "message"
  json.keys_under_root: true
  json.overwrite_keys: true
  scan_frequency: 1s

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1

logging.level: debug

output.elasticsearch:
  hosts: ["34.97.108.113:9200"]
  index: "filebeat-%{+yyyy-MM-dd}"
setup.template:
  name: 'filebeat'
  pattern: 'filebeat-*'
  enabled: true
setup.template.overwrite: true
setup.template.append_fields:
- name: time
  type: date

processors:
  - drop_fields:
      fields: ["agent","host","ecs","input","log"]

setup.ilm.enabled: false`

```

I changed scan\_frequncy but elasticsearch could'nt get logs faster  
How can i get logs in elasticsearch instantly?  
Please help me..

---

<div class="post-metadata">

**Author:** ![faec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faec/32/46988_2.png) [@faec](https://discuss.elastic.co/u/faec)\
**Post date:** [February 10, 2020, 7:59pm UTC](https://discuss.elastic.co/t/how-to-push-logs-to-elasticsearch-in-filebeat/218501/2 "2020-02-10T19:59:57Z")

</div>

Ingestion latency is affected by a lot of factors, including CPU and memory limits of both the beat and elasticsearch nodes, the network connection between them, and so on. You'll never see logs in the search index _instantly_ but there may be ways to reduce the latency.

`scan_frequency` specifies how often filebeat should scan its input paths for new files, but it doesn't affect how fast the data in those files is processed ones they're being read. Some diagnostic questions to start with when troubleshooting ingestion speed are: how much delay are you observing between the initial logs and their appearance in elasticsearch? Is the delay steady, or does it vary depending on the time of day or other factors? How much log data (on average) are you trying to transmit? What is the network bandwidth between your beats and your elasticsearch server?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 9, 2020, 8:00pm UTC](https://discuss.elastic.co/t/how-to-push-logs-to-elasticsearch-in-filebeat/218501/3 "2020-03-09T20:00:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.