Filebeat can't fetch the noeol(no end-of-line) file

Elastic Stack Beats
1

Forgive my poor English,I'm a chinese.I use the filebeat 5.2.0,our system is Linux 2.6.32-431.el6.x86_64.My colleague keep generating many new files by the Spring.All those file have only one line of data(json).
When vi those file,the file is marked as noeol.I use the filebeat fetch the file and sending the text to kafka,the registry file :
[
{
"source": "/home/hadoop/baiRong/AccountChangeU/aaa.txt",
"offset": 0,
"FileStateOS": {
"inode": 27187051,
"device": 2053
},
"timestamp": "2017-06-23T11:00:10.615905605+08:00",
"ttl": -1
}
]
I vi a new file ,then fetch this file,the text can be sent to kafka,the registry file:
[
{
"source": "/home/hadoop/baiRong/AccountChangeU/aaa.txt",
"offset": 0,
"FileStateOS": {
"inode": 27187051,
"device": 2053
},
"timestamp": "2017-06-23T11:00:10.615905605+08:00",
"ttl": -2
},
{
"source": "/home/hadoop/baiRong/AccountChangeU/a.txt",
"offset": 13,
"FileStateOS": {
"inode": 27191424,
"device": 2053
},
"timestamp": "2017-06-23T11:28:25.088876899+08:00",
"ttl": -1
}
]

2

filebeat splits events when it sees the line separator \n. With this one being missing, filebeat can not tell, if more content is to come or not. Can any of you add the final '\n' ? (e.g. echo >> )

1 Like
3

Thanks for your replying.We have using flume in our product environment.I only want to try the filebeat.I think it will be better if the filebeat can solve this or print out the log.I use -d "*", i can't see any relevant log.It will be rude to say that,thanks again and hope your company will do better.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.