# I can't send older log

**URL:** <https://discuss.elastic.co/t/i-cant-send-older-log/151987>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 11, 2018, 6:24am UTC](https://discuss.elastic.co/t/i-cant-send-older-log/151987 "2018-10-11T06:24:09Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![OlivierPCN](https://avatars.discourse-cdn.com/v4/letter/o/41988e/32.png) [@OlivierPCN](https://discuss.elastic.co/u/OlivierPCN)\
**Post date:** [October 11, 2018, 6:24am UTC](https://discuss.elastic.co/t/i-cant-send-older-log/151987/1 "2018-10-11T06:24:09Z")

</div>

Hello,

I use filebeat to send slow log to logstash. I use the mysql module for that. But when I try to send an old file, filebeat send only the last slow query of the file. I already tried to delete the registry and I search for a missing configuration in my filebeat.yml, but I found nothing.

My filebeat configuration :

```auto
#=========================== Filebeat inputs =============================

filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

- type: log

  # Change to true to enable this input configuration.
  enabled: false

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/log/*.log
    #- c:\programdata\elasticsearch\logs\*

#============================= Filebeat modules ===============================

filebeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
  reload.enabled: false

  # Period on which files under path should be checked for changes
  #reload.period: 10s

#==================== Elasticsearch template setting ==========================

setup.template.settings:
  index.number_of_shards: 3
  #index.codec: best_compression
  #_source.enabled: false

#================================ General =====================================

# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
name: "filebeat-test"

# The tags of the shipper are included in their own field with each
# transaction published.
tags: ["filebeat-test"]

# Optional fields that you can specify to add additional information to the
# output.
#fields:
# env: staging

#================================ Outputs =====================================

#----------------------------- Logstash output --------------------------------
output.logstash:
  # The Logstash hosts
  hosts: ["logstashIP:5044"]

```

An extract of my filebeat log :

```auto
2018-10-11T02:10:32.375-0400	INFO	log/input.go:111	Configured paths: [/root/*.log]
2018-10-11T02:10:32.375-0400	INFO	crawler/crawler.go:82	Loading and starting Inputs completed. Enabled inputs: 0
2018-10-11T02:10:32.375-0400	INFO	cfgfile/reload.go:122	Config reloader started
2018-10-11T02:10:32.376-0400	INFO	log/input.go:111	Configured paths: [/var/log/mysql/error.log* /var/log/mysqld.log*]
2018-10-11T02:10:32.377-0400	INFO	log/input.go:111	Configured paths: [/root/*.log]
2018-10-11T02:10:32.377-0400	INFO	cfgfile/reload.go:253	Starting 1 runners ...
2018-10-11T02:10:32.377-0400	INFO	input/input.go:87	Starting input of type: log; ID: 18267570244159057816 
2018-10-11T02:10:32.377-0400	INFO	input/input.go:87	Starting input of type: log; ID: 15544384872414468624 
2018-10-11T02:10:32.377-0400	INFO	cfgfile/reload.go:214	Loading of config files completed.
2018-10-11T02:10:32.378-0400	INFO	log/harvester.go:228	Harvester started for file: /root/test.log
2018-10-11T02:11:02.376-0400	INFO	[monitoring]	log/log.go:124	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":10,"time":{"ms":13}},"total":{"ticks":20,"time":{"ms":24},"value":0},"user":{"ticks":10,"time":{"ms":11}}},"info":{"ephemeral_id":"a20f239b-72f3-4c4e-8071-30fdc78aea92","uptime":{"ms":30008}},"memstats":{"gc_next":5662672,"memory_alloc":3122544,"memory_total":6951832,"rss":15351808}},"filebeat":{"events":{"added":8,"done":8},"harvester":{"open_files":1,"running":1,"started":1}},"libbeat":{"config":{"module":{"running":1,"starts":1},"reloads":1},"output":{"events":{"acked":3,"batches":2,"total":3},"read":{"bytes":12},"type":"logstash","write":{"bytes":1026}},"pipeline":{"clients":4,"events":{"active":0,"filtered":5,"published":3,"retry":2,"total":8},"queue":{"acked":3}}},"registrar":{"states":{"cleanup":1,"current":1,"update":8},"writes":7},"system":{"cpu":{"cores":1},"load":{"1":0.06,"15":0.05,"5":0.03,"norm":{"1":0.06,"15":0.05,"5":0.03}}}}}}
2018-10-11T02:11:32.376-0400	INFO	[monitoring]	log/log.go:124	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":10,"time":{"ms":1}},"total":{"ticks":20,"time":{"ms":2},"value":20},"user":{"ticks":10,"time":{"ms":1}}},"info":{"ephemeral_id":"a20f239b-72f3-4c4e-8071-30fdc78aea92","uptime":{"ms":60008}},"memstats":{"gc_next":5662672,"memory_alloc":3290088,"memory_total":7119376}},"filebeat":{"harvester":{"open_files":1,"running":1}},"libbeat":{"config":{"module":{"running":1}},"pipeline":{"clients":4,"events":{"active":0}}},"registrar":{"states":{"current":1}},"system":{"load":{"1":0.04,"15":0.05,"5":0.03,"norm":{"1":0.04,"15":0.05,"5":0.03}}}}}}

```

Thanks in advance !  
Olivier

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [October 12, 2018, 11:32pm UTC](https://discuss.elastic.co/t/i-cant-send-older-log/151987/2 "2018-10-12T23:32:26Z")

</div>

What's the contents of your module configurations?

The filebeat log output suggest look for these files only: `/root/*.log`, `/var/log/mysql/error.log*`, `/var/log/mysqld.log*`. Is the slowlog fileset enabled?

---

<div class="post-metadata">

**Author:** ![OlivierPCN](https://avatars.discourse-cdn.com/v4/letter/o/41988e/32.png) [@OlivierPCN](https://discuss.elastic.co/u/OlivierPCN)\
**Post date:** [October 15, 2018, 5:57am UTC](https://discuss.elastic.co/t/i-cant-send-older-log/151987/3 "2018-10-15T05:57:55Z")

</div>

Hi,

In my mysql module, I only configure the path. To try an old log, I configured it to "/root/\*.log". My slowlog fileset is enable. I receive Only the first ans last query of the file.

If my slow log comes from another server, can this be a problem?

Sorry for my english if I make some mistakes(it is not my native language).

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [October 16, 2018, 8:53pm UTC](https://discuss.elastic.co/t/i-cant-send-older-log/151987/4 "2018-10-16T20:53:58Z")

</div>

> If my slow log comes from another server, can this be a problem?

Depends where/how the data is written and stored. Is the file local to filebeat or on another host? Is it shared via Network Shares (which often can be a problem)?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 13, 2018, 9:08pm UTC](https://discuss.elastic.co/t/i-cant-send-older-log/151987/5 "2018-11-13T21:08:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.