Not able to create index for an unchanged log file

[Ritwik_Sharma]

Elasticsearch version (5.1.1):

JVM version (1.8.0_77):

Description of the problem including expected versus actual behavior:
With proper Filebeat and Logstash configuration, index should be created in Elasticsearch, but it is not able to create a new index when the log file is unchanged. It is obvious that it should not override the existing Index if there is no change in the log file but it is also not able to create a new index for an unchanged log file. If we create a small change in the log file then it is able to create a new index.

Steps to reproduce:

1. Install Filebeat, logstash and Elasticsearch
2. Set filebeat configurations to read from a log file and send it to logstash.
3. Set logstash configuration to create new index in Elasticsearch with input as beats.
4. Once index is created delete it.
5. no new index will be created now until there is a change in log file.
6. Change index name in Logstash configuration and create new index.
7. Even when the index name is changed it will not be able to create new index.
8. Make any changes in the log file, Index will be created.

[Christian_Dahlqvist]

Filebeat tails files, so unless you append data or create a new file it will not process any data and thus not create any new index. The behaviour you describe therefore sounds like what is expected.

[Ritwik_Sharma]

That is understood when we are creating same index. But if want to create new index with same file then it should do so.

[Christian_Dahlqvist]

No. It will only read the file once, no matter where you send it, so if you expect the file to be reprocessed you will need to modify the registry file where Filebeat keeps track of files processed.

[Ritwik_Sharma]

Alright, I got that now. Thanks a lot for your help @Christian_Dahlqvist