# File input, from NetApp CIFS share, not reading single file
**URL:** https://discuss.elastic.co/t/file-input-from-netapp-cifs-share-not-reading-single-file/195391
**Category:** Logstash
**Created:** [August 15, 2019, 8:23pm UTC](https://discuss.elastic.co/t/file-input-from-netapp-cifs-share-not-reading-single-file/195391 "2019-08-15T20:23:40Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@eclipsed450](https://discuss.elastic.co/u/eclipsed450)
#### Post date: [August 15, 2019, 8:23pm UTC](https://discuss.elastic.co/t/file-input-from-netapp-cifs-share-not-reading-single-file/195391/1 "2019-08-15T20:23:40Z")
Hi,
I've made quite a few other posts, and have successfully ingested more than 1 billion messages in the last couple weeks, but we've come to the conclusion that Logstash just isn't keeping up with the backlog of messages. The reason I say this is that Logstash will seemingly lose its place in the ingestion, and start over, randomly.
What we have is NetApp CIFS audit logs turned on, and writing to a CIFS share on itself (not in the audit logs). There are 10,001 files in this directory - a single current file, and 10,000 previous files; each 100MB in size; or approximately 77,000 lines. I had the input configured to `/mnt/cifs/*.xml`, and this was working, for the most part like I said above - hundreds of millions of messages have been ingested so far. But since it wasn't catching up, I opted to change the input to the single current file. I've tried different configurations of `start_position => "beginning"`, `sincedb_path => "/dev/null"`, `stat_interval => 1`, `mode => read`, and others, and it seems that Logstash will read the file when it starts, and only that one time, and never again, no matter what, until I restart Logstash.
I've read [this page](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#_reading_from_remote_network_volumes), but I'm not seeing where a specific problem is. Can someone break this down for me? Thank you in advance.
---
### Author:  [@eclipsed450](https://discuss.elastic.co/u/eclipsed450)
#### Post date: [August 15, 2019, 9:43pm UTC](https://discuss.elastic.co/t/file-input-from-netapp-cifs-share-not-reading-single-file/195391/2 "2019-08-15T21:43:31Z")
@badger since you've been so helpful so far 🙂
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 16, 2019, 7:50pm UTC](https://discuss.elastic.co/t/file-input-from-netapp-cifs-share-not-reading-single-file/195391/3 "2019-08-16T19:50:44Z")
sincedb\_path =\> "/dev/null" does not prevent the file input managing the sincedb. It prevents it persisting that db across restarts.
Why do expect it to get read more than once?
If you enable '--log.level trace' what does filewatch have to say?
---
### Author:  [@eclipsed450](https://discuss.elastic.co/u/eclipsed450)
#### Post date: [August 16, 2019, 8:34pm UTC](https://discuss.elastic.co/t/file-input-from-netapp-cifs-share-not-reading-single-file/195391/4 "2019-08-16T20:34:37Z")
> [@Badger](#):
>
> sincedb\_path =\> "/dev/null" does not prevent the file input managing the sincedb. It prevents it persisting that db across restarts.
Okay, good to know, thank you.
> [@Badger](#):
>
> Why do expect it to get read more than once?
It's not documented anywhere that I've found, but as I understand it, when the file gets to 100MB, it renames with a timestamp in the name, then starts a new `latest` file. Additionally, once Logstash has read that latest file on first startup, it doesn't read any lines that are added to it, even when it's that same file.
> [@Badger](#):
>
> If you enable '--log.level trace' what does filewatch have to say?
I'm sorry, where do I put this when I'm running Logstash as a service?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 16, 2019, 8:43pm UTC](https://discuss.elastic.co/t/file-input-from-netapp-cifs-share-not-reading-single-file/195391/5 "2019-08-16T20:43:58Z")
You can set the log.level in logstash.yml
---
### Author:  [@eclipsed450](https://discuss.elastic.co/u/eclipsed450)
#### Post date: [August 16, 2019, 9:21pm UTC](https://discuss.elastic.co/t/file-input-from-netapp-cifs-share-not-reading-single-file/195391/6 "2019-08-16T21:21:06Z")
> [@Badger](#):
>
> You can set the log.level in logstash.yml
If I run the single config via the command line, it seems to read the file as I expect it to; however if I put that config parameter in the yml and restart the service, it will read everything inthe current file once, then stop reading any new messages. In both, I get the following:
```auto
[DEBUG] 2019-08-16 13:54:58.339 [pool-3-thread-2] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2019-08-16 13:54:58.339 [pool-3-thread-2] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[TRACE] 2019-08-16 13:54:58.595 [[main]""}
```
and it looks like the `current_size` and `last_stat_size` don't change.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 17, 2019, 12:48pm UTC](https://discuss.elastic.co/t/file-input-from-netapp-cifs-share-not-reading-single-file/195391/7 "2019-08-17T12:48:10Z")
Does it work as expected with a local file?
---
### Author:  [@eclipsed450](https://discuss.elastic.co/u/eclipsed450)
#### Post date: [August 17, 2019, 2:11pm UTC](https://discuss.elastic.co/t/file-input-from-netapp-cifs-share-not-reading-single-file/195391/8 "2019-08-17T14:11:02Z")
> [@Badger](#):
>
> Does it work as expected with a local file?
Even if it did, NetApp CIFS audit logs don't have an option to write to anything that wouldn't be a remote share to logstash. I could try to figure out something to sync the files from the CIFS share to a local filesystem, maybe, if this is determined to not work for me in this method.
---
### Author:  [@eclipsed450](https://discuss.elastic.co/u/eclipsed450)
#### Post date: [August 19, 2019, 4:41pm UTC](https://discuss.elastic.co/t/file-input-from-netapp-cifs-share-not-reading-single-file/195391/9 "2019-08-19T16:41:00Z")
FYI, I tried using [lsyncd](https://github.com/axkibe/lsyncd) to keep the single file in sync to the local system, but it didn't seem to be able to do that repeatedly. I'm going to try again with the rsync.ssh config, but am not optimistic. Also, when I checked this morning, the running config has been pulling in messages, but only once every hour-ish.
---
### Author:  [@eclipsed450](https://discuss.elastic.co/u/eclipsed450)
#### Post date: [August 19, 2019, 6:23pm UTC](https://discuss.elastic.co/t/file-input-from-netapp-cifs-share-not-reading-single-file/195391/10 "2019-08-19T18:23:28Z")
I found this morning if I run `watch ls /mnt/cifs_audit/`, the config runs fine and ingests as I'd expect it to. Until I can figure out a better solution, I'm running a crontab to `ls` the directory on a loop, and it seems to be importing.
---
### Author:  [@eclipsed450](https://discuss.elastic.co/u/eclipsed450)
#### Post date: [August 19, 2019, 9:25pm UTC](https://discuss.elastic.co/t/file-input-from-netapp-cifs-share-not-reading-single-file/195391/11 "2019-08-19T21:25:30Z")
It seems adding the `close_older` (we set it to 5) option we were looking for, and doesn't require the cron job or lsyncd.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 16, 2019, 9:25pm UTC](https://discuss.elastic.co/t/file-input-from-netapp-cifs-share-not-reading-single-file/195391/12 "2019-09-16T21:25:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.