# Filebeat not sending Apache logs to Logstash

**URL:** <https://discuss.elastic.co/t/filebeat-not-sending-apache-logs-to-logstash/243118>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [July 29, 2020, 8:07pm UTC](https://discuss.elastic.co/t/filebeat-not-sending-apache-logs-to-logstash/243118 "2020-07-29T20:07:40Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kevin\_f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kevin_f/32/72456_2.png) [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\
**Post date:** [July 29, 2020, 8:07pm UTC](https://discuss.elastic.co/t/filebeat-not-sending-apache-logs-to-logstash/243118/1 "2020-07-29T20:07:40Z")

</div>

Hello, All,

I have an issue where I have filebeat installed on my apache server with the following config:

```auto
filebeat:
  prospectors:
    - enabled: true
      backoff: 1s
      backoff_factor: 2
      clean_inactive: 0
      clean_removed: true
      close_eof: false
      close_inactive: 5m
      close_removed: true
      close_renamed: false
      close_timeout: 0
      document_type: apache
      encoding: plain
      fields_under_root: true
      harvester_buffer_size: 16384
      input_type: log
      max_backoff: 10s
      max_bytes: 10485760
      paths:
        - /var/log/httpd/*
      scan_frequency: 10s
      tags:
        - apache
      tail_files: false

```

My filebeat.yml is:

```auto
fields_under_root: false
filebeat.config_dir: /etc/filebeat/conf.d
filebeat.idle_timeout: 5s
filebeat.publish_async: false
filebeat.registry_file: .filebeat
filebeat.shutdown_timeout: 0
filebeat.spool_size: 2048
max_procs: ~
name: myserver
output.logstash:
  hosts:
    - "logstash_server:5044"
queue_size: 1000

```

So I installed filebeat way later than when I configured my apache server. Logrotation is set for /var/log/httpd/ where files are zipped everyday at 3am. Currently my filebeat can only send logs starting from today but is not sending logs older than today. I understand that filebeat can't harvest .gz files, so I unzipped those files and restarted filebeat and still cant see older apache logs in kibana.

In filebeat log, I see that it says that it has started harvesting the unzipped log files but nothing more after that.

Any help would be much appreciated. Thanks.

---

<div class="post-metadata">

**Author:** ![Kaiyan\_Sheng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaiyan_sheng/32/38247_2.png) [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)\
**Post date:** [July 29, 2020, 9:43pm UTC](https://discuss.elastic.co/t/filebeat-not-sending-apache-logs-to-logstash/243118/2 "2020-07-29T21:43:08Z")

</div>

Hi! Are the old logs also under `/var/log/httpd/*` directory? Also do you have `ignore_older` parameter disabled?

---

<div class="post-metadata">

**Author:** ![Kevin\_f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kevin_f/32/72456_2.png) [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\
**Post date:** [July 30, 2020, 8:17am UTC](https://discuss.elastic.co/t/filebeat-not-sending-apache-logs-to-logstash/243118/3 "2020-07-30T08:17:55Z")

</div>

hi @Kaiyan_Sheng,

Thank you for your response.

Yes the logs are also in the same directory path as /var/log/httpd/\*. I have not set ignore\_older parameter anywhere.

I thought if ignore\_older is not set then by default it is set to disable ?

---

<div class="post-metadata">

**Author:** ![Kaiyan\_Sheng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaiyan_sheng/32/38247_2.png) [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)\
**Post date:** [July 30, 2020, 7:16pm UTC](https://discuss.elastic.co/t/filebeat-not-sending-apache-logs-to-logstash/243118/4 "2020-07-30T19:16:44Z")

</div>

Hmmm have you ingested these old logs before? Maybe remove the registry file will help!

---

<div class="post-metadata">

**Author:** ![Kevin\_f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kevin_f/32/72456_2.png) [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\
**Post date:** [July 30, 2020, 7:53pm UTC](https://discuss.elastic.co/t/filebeat-not-sending-apache-logs-to-logstash/243118/5 "2020-07-30T19:53:08Z")

</div>

These old logs were never ingested. I have removed the registry file again and restarted filebeat, still now joy.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 4, 2022, 8:08am UTC](https://discuss.elastic.co/t/filebeat-not-sending-apache-logs-to-logstash/243118/6 "2022-11-04T08:08:13Z")

</div>