I have four nodes cluster from the two ones are data and master nodes, one dedicated master node and one coordinating node. I collect logs from windows and linux hosts with winlogbeat and filebeat which are sent to logstash at coordinatiing node.
Recently I updated cluster to version 7.4 and after this I have the big problem. Linux logs sent from coordinating node let's name it as DB05 still have field agent.version 7.3.2. For the begining I tried to reinstall filebeat on that host, nothing helped. Then I erased completely all files of filebeat and installed again, no result again. I switched off DB05 but logs were uploading to index. So I find out that the source of that logs in index wasn't DB05 despite field agent.hostname was DB05. I switched off all elastic nodes besides one. And there was new logs. I restricted connections to nodes with firewall only between nodes but I still get these fantom logs.
The source for these logs are /var/log/messages and /var/log/secure. I tried to look through different servers but no one server has such a logs.
Finally I got stuck because only one node gets logs somewhere but there is no any source except itself. And this node doesn't have this info which is uploaded to index.
Any help appreciated....
I solved the problem.
Who or what was sending fake logs I couldn't find out. I just deleted hole index with these logs and that was all.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.