Hi
I'm working on harvesting large csv files using multiline option and It looks like that default option of multiline.max_lines
is not overriden by value set in filebeat.yml
When filebeat is parsing record which takes more than 500 lines he stops on 500th line and incomplete record is published to output.
I am using filebeat 5.1.2 in docker on ubuntu image
Here are configuration and results of parsing sample file: https://gist.github.com/mjasion/235768fb87a7f708264669af755bd23d
Published record should have all lines from 0001
to 999
.
I also set multiline.timeout: 10s
but result was same