# Logstash file-input doesn't pick up new files

**URL:** https://discuss.elastic.co/t/logstash-file-input-doesnt-pick-up-new-files/206026
**Category:** Logstash
**Created:** [October 31, 2019, 10:51am UTC](https://discuss.elastic.co/t/logstash-file-input-doesnt-pick-up-new-files/206026 "2019-10-31T10:51:20Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Gordon\_Thompson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gordon_thompson/32/56927_2.png) [@Gordon\_Thompson](https://discuss.elastic.co/u/Gordon_Thompson)
#### Post date: [October 31, 2019, 10:51am UTC](https://discuss.elastic.co/t/logstash-file-input-doesnt-pick-up-new-files/206026/1 "2019-10-31T10:51:20Z")

</div>

We have a simple logstash 7.1.1 installation on a hardened Redhat 7.7

**pipelines.yml**

```
- pipeline.id: synonyms-ingest
  pipeline.workers: 1
  path.config: "/etc/logstash/CDE/synonym-logstash-csv-to-es.conf"

```

**synonym-logstash-csv-to-es.conf**

```
input 
{
	file 
	{
		stat_interval => "10 seconds"
		start_position => "beginning"
		path => "/Data/Synonyms/synonyms.tsv"
		sincedb_path => "/dev/null"
		mode => "read"
		file_completed_action => "delete"
	}
}
filter { ....... }
output {
	amazon_es {
		manage_template => false
		document_id => "%{[@metadata][generated_id]}"
		hosts => ["${ES_HOST}"]
		index => "synonym-english"
	}
}

```

The first time logstash runs it ingests correctly into the endpoint.

If the synonyms.tsv is deleted and re-added (it's got a different inode value and a different last modified date) then logstash doesn't re-run the ingestion until logstash is restarted.

This problem is happening on a hardened AWS instance provided by the client. I don't know the extent of the hardening but feel that it has something to do with the issue. My next test will be to run it on a standard image and see if that resolves the issue.

Can anybody offer any suggestions about how to diagnose this?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [October 31, 2019, 2:15pm UTC](https://discuss.elastic.co/t/logstash-file-input-doesnt-pick-up-new-files/206026/2 "2019-10-31T14:15:53Z")

</div>

> [@Gordon\_Thompson](#):
>
> Can anybody offer any suggestions about how to diagnose this?

If you set log level to TRACE then filewatch should log enough for you do see if it thinks it has seen the file before. It it reads the file again after a restart then with sincedb\_path set to /dev/null that suggests to me that that is the problem.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 28, 2019, 2:15pm UTC](https://discuss.elastic.co/t/logstash-file-input-doesnt-pick-up-new-files/206026/3 "2019-11-28T14:15:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
