File input not working on OSX

Elastic Stack Logstash
1

I'm using the file input on my mac, and every time I update the file, it's replaying all messages from the beginning of the log.

I've tried explicitly setting the sincedb_path and start_position vars, but no luck.

.sincedb is created, but I noticed the inode didn't match the file I was updating.

On further investigation, (and I could be wrong), it looks like OSX changes the inode every time the file is updated, not just when a new file is created.

I assume this is the reason it resends all as Logstash thinks it's a new file, but wondering if anyone else has seen this, and/or has a workaround.

2

Indeed, it sounds like the file is rewritten in a new file. Logstash assumes that log files are appended to. What program is updating the file and is this an actual use case or just you testing?

3

Hi, it's just me testing, so using vi to add a new line to the file.

We don't use Mac's in production :slight_smile: so it's not a big issue, but it's nicer to do my development on a Mac so would be good if it could be resolved.

4

vi probably creates a new file and renaming it to replace the original file, thus resulting in a new inode for the file every time. Make sure you append to the file instead, e.g. with echo "test log message" >> logfile.log.

5

Good point, didn't even occur to me that might be the issue - thanks for pointer.