Large log entries getting truncated

Elastic Stack Beats
1

Using filebeat v1.0.0~rc2 on Ubuntu 12.04.3 LTS, our large php-error log entries are getting truncated. The log entries are typically between 3k and 8k, common ones are around 4k. The first 1024 bytes is what seems to be getting cut off.

At first thought it was network or logstash related, I took those out of the picture by using the file output in filebeat to send the log output directly to a file on the server where filebeat is installed and watching the php_error log.

Is this a known issue, how can we be sure the entire entry is sent without getting truncated?

filebeat config on problem host :

    filebeat:
  prospectors:
    -
  paths:
    - /mnt/log/php_errors.log
  encoding: utf-8
  input_type: log
  harvester_buffer_size: 16384
  registry_file: /var/lib/filebeat/registry
output:
  file:
    path: "/tmp/filebeat.local.log"
    filename: filebeat
    rotate_every_kb: 10000
    number_of_files: 7
shipper:
logging:
  files:
2

@Chris_Clifton This should of course not happen. The config file looks good. Any chance to provide me with a log file where the error happens? You can also share it privately.

3

Thanks, messaged you file links with the full entries, one of the raw php log, one of the filebeat output file that matches.

4

Thanks for reporting. I've managed to create an unit test reproducing this error. Problem has been some internal read buffer being reduced by accident.

You can follow the progress on github

5

This should now be fixed and will be part of the next release. In case you want to test it earlier you can use the nightlies. We recommend to use the nightlies only for testing: https://beats-nightlies.s3.amazonaws.com/index.html?prefix=filebeat/

6

Thank you!

7

I am seeing files being truncated, and I am using filebeat-5.0.0-alpha3-linux-x64

2016-06-15T00:05:20Z INFO File was truncated. Begin reading file from offset 0: /flow_base_dir/dump/flows-201606150000.dump
2016-06-15T00:10:20Z INFO File was truncated. Begin reading file from offset 0: /flow_base_dir/dump/flows-201606150005.dump
2016-06-15T00:15:20Z INFO File was truncated. Begin reading file from offset 0: /flow_base_dir/dump/flows-201606150010.dump
2016-06-15T00:20:21Z INFO File was truncated. Begin reading file from offset 0: /flow_base_dir/dump/flows-201606150015.dump
2016-06-15T00:25:21Z INFO File was truncated. Begin reading file from offset 0: /flow_base_dir/dump/flows-201606150020.dump
2016-06-15T00:30:21Z INFO File was truncated. Begin reading file from offset 0: /flow_base_dir/dump/flows-201606150025.dump
2016-06-15T00:35:11Z INFO File was truncated. Begin reading file from offset 0: /flow_base_dir/dump/flows-201606150030.dump
2016-06-15T00:40:12Z INFO File was truncated. Begin reading file from offset 0: /flow_base_dir/dump/flows-201606150035.dump
2016-06-15T00:45:12Z INFO File was truncated. Begin reading file from offset 0: /flow_base_dir/dump/flows-201606150040.dump
2016-06-15T00:50:12Z INFO File was truncated. Begin reading file from offset 0: /flow_base_dir/dump/flows-201606150045.dum

8

This is totally unrelated. Version v1.0.0-rc2 used to have a bug fixed by then. The log message you're reffering to is filebeat having detected the files being forwarded have been changed in size (new file size < old file size). Please create another topic discussing your problem.

9

Mi apologies, I didn't realize