# Filebeat is shipping old log lines

**URL:** <https://discuss.elastic.co/t/filebeat-is-shipping-old-log-lines/216242>\
**Category:** Beats\
**Created:** [January 23, 2020, 12:38pm UTC](https://discuss.elastic.co/t/filebeat-is-shipping-old-log-lines/216242 "2020-01-23T12:38:50Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![kwisatz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kwisatz/32/61277_2.png) [@kwisatz](https://discuss.elastic.co/u/kwisatz)\
**Post date:** [January 23, 2020, 12:38pm UTC](https://discuss.elastic.co/t/filebeat-is-shipping-old-log-lines/216242/1 "2020-01-23T12:38:50Z")

</div>

I've recently noticed that `filebeat` is shipping not only new lines in my logs, but apparently all the lines, all the time.

E.g. I'm shipping nginx logs that are being created very ordinarily:

```auto
  access_log /var/log/nginx/site-access.log combined;

```

My filebeat config says:

```auto
filebeat:
  prospectors:
  - document_type: log
    fields:
      gl2_source_collector: bd4ffc76-23b4-49c1-87df-f ******
    ignore_older: 0
    input_type: log
    paths:
    - /var/log/nginx/*.log
    scan_frequency: 10s
    tail_files: true
output:
  logstash:
    hosts:
    - graylog. *****.com:5044
path:
  data: /var/cache/graylog/collector-sidecar/filebeat/data
  logs: /var/log/graylog/collector-sidecar
tags:
- linux
- nginx
- phpfpm

```

And yet, I'm seeing log entries being shipped to my graylog instance that are several days old.  
Any clues to what could be causing this? To my knowledge, the log files are not truncated other than a logrotate each week.

E.g.

```auto
root@host2:~# stat /var/log/nginx/site-access.log
  File: ‘/var/log/nginx/site-access.log’
  Size: 6121515 Blocks: 11968 IO Block: 4096 regular file
Device: 801h/2049d Inode: 662327 Links: 1
Access: (0640/-rw-r-----) Uid: ( 33/www-data) Gid: ( 4/ adm)
Access: 2020-01-23 12:49:46.556082523 +0100
Modify: 2020-01-23 12:49:49.560126373 +0100
Change: 2020-01-23 12:49:49.560126373 +0100
 Birth: -

```

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [January 23, 2020, 3:49pm UTC](https://discuss.elastic.co/t/filebeat-is-shipping-old-log-lines/216242/2 "2020-01-23T15:49:58Z")

</div>

Hey @kwisatz,

The problem might be on the rotation of files, could you check the `log.source.path` of the old entries shipped? What is your logrotate configuration for these files?

You can check this guide with information about common issues with rotated files: [https://www.elastic.co/guide/en/beats/filebeat/7.5/file-log-rotation.html](https://www.elastic.co/guide/en/beats/filebeat/7.5/file-log-rotation.html)

Oh, and what version of filebeat are you using?

---

<div class="post-metadata">

**Author:** ![kwisatz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kwisatz/32/61277_2.png) [@kwisatz](https://discuss.elastic.co/u/kwisatz)\
**Post date:** [January 23, 2020, 4:53pm UTC](https://discuss.elastic.co/t/filebeat-is-shipping-old-log-lines/216242/3 "2020-01-23T16:53:22Z")

</div>

Thanks for your reply @jsoriano

In the end, I think the reason we were seeing old log entries was not related to filebeats at all, but rather to our graylog instance (yeah, I know) processing its disk journal after it got stuck on an index for a while.

So I guess this can be marked as closed/invalid.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 20, 2020, 7:05pm UTC](https://discuss.elastic.co/t/filebeat-is-shipping-old-log-lines/216242/4 "2020-02-20T19:05:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.