Hello 
Is there any possible method for shipping single-line logs without newline characters from Filebeat to Kafka?
I'm using 8.6.1 Stack, and in Filebeat, filestream (log) type.
The log is created just once every day.
And the setting (no newline) is strict, I can't change or modify it.
After reading this log, there is no error log left.
Harvester is working nomaly (pipeline open, close, etc..), but not shipping to Kafka.
I think Filebeat should wait the next log, even if the log is already ended, because there is no newline character.
cf) elasticGuide_filebeat_newlineCharacterRequired
If add the log newline character on purpose, it works well.
In that case, Kafka got all of the log line without leak.
So it's not problem with log size, lang or any other setting...
But I can't suggest adding newline character in the log to related department, I really need help to through this problem.
I tried some tricks for that.
-
'close' option - Using filestream type (and log type either)
I tried harvester close with all option (timeout, rename, removed, eof...).
But none of them works for me. Maybe 'close' means a real 'close'. Not 'shipping before close'. -
'multiline' and 'mutiline timeout' option
I write a pattern '*' and set multiline timeout.
The result is the same as above.
Any solutions of this?