I've been having difficulty with the logstash input filter sometimes picking up partial lines with my setup. I think it may be related to how I am using docker but I'm running out of ideas.
I have logstash in a docker container which monitors a file on the host system for new lines. When lines are added directly from the host system then these seem to be picked up correctly. If however these lines are written from inside a second docker container then most lines are correct but Logstash always seems to pick up some partial lines.
I know there's some complex logic going on with inodes here and watching files but unsure what might be the problem here, or how I might even start fixing. Should I be looking at different docker storage drivers?
Thanks in advance!