FileBeat not picking up the updated file asap

Elastic Stack Beats
1

I have installed Elastic Search, FileBeat and Kibana on my Windows 7 machine.
As of now my log path is C:\LogFiles\ and it contains 5 files(Log File 1.txt...2.txt....). I created it new and have some contents in it.
I edited the yaml file to pick the logs from "- C:\LogFiles*" and have set "input_type: file". No more changes.

I started all the 3(ES, Kibana and FB) from Windows Services. When tried searching for a text in Kibana, no results where found.

Also, I edited the yaml file to pick the logs from "- C:\LogFiles*" and have set "input_type: log". Again no results.

What am I missing here ? Also, do I need to restart FB, ES or KB services if I have any changes to my log files?

2

input_type:file does not exist. What was your intention with this change?

Did you restart filebeat after config changes? You only need to restart filebeat if you change the config, not if files change.

I think also your path to pick up should be C:\LogFiles\*

To get some more information what happens, your filebeat with -e -d "*" and it will print out some more debugging information.

3

I have my logs saved in TXT format. Hence, I tried using input_type: file. I read an article on that.
But now I have used input_type: log and corrected my log paths to -
paths:
- C:\LogFiles*
- C:\Log Files 1*
- C:\Log Files 2*

This works fine. But it doesnt works when I add a shared server path

  • \ServerName\Shared\ServiceLogs*

Is there anything that I am missing here ?

4

Can you link to the article where it was stated that input_type: file exist? It would be good to notifiy the author that this is incorrect. Otherwise other people will follow the same example.

What do you mean by a Shared server path? Shared drive? Is ServiceLogs the prefix of your files or a directory?

5

Sorry, I don't have that link now. I found that during my initial analysis. Will try to search for it.

By shared path I meant, the path where logs on my servers are stored. I have shared the logs folder on all my servers with a group of people.
Sorry for the typo above

  • \ServerName\Shared\ServiceLogs*

Cant I use a shared location in the config file ?

6

Path for remote drives should work as there is nothing really different about it from a filebeat perspective. But an important note here is that we don't recommend to do this. See https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-network-volumes.html

Try to install filebeat as close to the source if possible.