# Logstash won't read file?

**URL:** <https://discuss.elastic.co/t/logstash-wont-read-file/135813>\
**Category:** Logstash\
**Created:** [June 14, 2018, 12:55am UTC](https://discuss.elastic.co/t/logstash-wont-read-file/135813 "2018-06-14T00:55:53Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![reswob](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/reswob/32/38015_2.png) [@reswob](https://discuss.elastic.co/u/reswob)\
**Post date:** [June 14, 2018, 12:55am UTC](https://discuss.elastic.co/t/logstash-wont-read-file/135813/1 "2018-06-14T00:55:54Z")

</div>

I'm playing with the some logs files I ..... found on the internet, which comes in JSON files. So I'm using LS to ingest them and send them to ES. But LS won't read them and I'm having a really hard time diagnosing the problem.

My conf file:

```
input {
# stdin { codec => json }

    file {
            path => "/opt/BOTS/botsv1.iis.json"
            sincedb_path => "/dev/null"
            codec => json
    }
}

output {

    stdout { codec => rubydebug }

}

```

The only lines in logstash.yml that are active:

```
path.data: /var/lib/logstash
log.level: debug
path.logs: /var/log/logstash

```

Everything else is commented out.

When I run

/`usr/share/logstash/bin/logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/testtest.conf`

nothing prints to the screen.

The file has these permissions:

`-rwxr-xr-x. 1 logstash logstash 27M Jun 8 18:52 /opt/BOTS/botsv1.iis.json`

Here is the log file:

```
[2018-06-13T19:24:24,094][INFO][logstash.agent] Pipelines running {:count=>1, :pipelines=>["main"]}
[2018-06-13T19:24:24,102][DEBUG][logstash.inputs.file] _globbed_files: /opt/BOTS/botsv1.iis.json: glob is: ["/opt/BOTS/botsv1.iis.json"]
[2018-06-13T19:24:24,103][DEBUG][logstash.inputs.file] _discover_file: /opt/BOTS/botsv1.iis.json: new: /opt/BOTS/botsv1.iis.json (exclude is [])
[2018-06-13T19:24:24,196][DEBUG][logstash.inputs.file] _open_file: /opt/BOTS/botsv1.iis.json: opening
[2018-06-13T19:24:24,197][DEBUG][logstash.inputs.file] /opt/BOTS/botsv1.iis.json: initial create, no sincedb, seeking to end 28215738
[2018-06-13T19:24:24,199][DEBUG][logstash.inputs.file] each: file grew: /opt/BOTS/botsv1.iis.json: old size 0, new size 28215738
[2018-06-13T19:24:25,201][DEBUG][logstash.inputs.file] each: file grew: /opt/BOTS/botsv1.iis.json: old size 0, new size 28215738
[2018-06-13T19:24:26,203][DEBUG][logstash.inputs.file] each: file grew: /opt/BOTS/botsv1.iis.json: old size 0, new size 28215738
[2018-06-13T19:24:27,206][DEBUG][logstash.inputs.file] each: file grew: /opt/BOTS/botsv1.iis.json: old size 0, new size 28215738
[2018-06-13T19:24:27,977][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
[2018-06-13T19:24:27,979][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
[2018-06-13T19:24:28,209][DEBUG][logstash.inputs.file] each: file grew: /opt/BOTS/botsv1.iis.json: old size 0, new size 28215738
[2018-06-13T19:24:29,097][DEBUG][logstash.pipeline] Pushing flush onto pipeline {:pipeline_id=>"main", :thread=>"#<Thread:0x15f64d55@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:246 sleep>"}
[2018-06-13T19:24:29,211][DEBUG][logstash.inputs.file] each: file grew: /opt/BOTS/botsv1.iis.json: old size 0, new size 28215738
[2018-06-13T19:24:30,214][DEBUG][logstash.inputs.file] each: file grew: /opt/BOTS/botsv1.iis.json: old size 0, new size 28215738
[2018-06-13T19:24:31,215][DEBUG][logstash.inputs.file] each: file grew: /opt/BOTS/botsv1.iis.json: old size 0, new size 28215738
[2018-06-13T19:24:32,218][DEBUG][logstash.inputs.file] each: file grew: /opt/BOTS/botsv1.iis.json: old size 0, new size 28215738
--> [2018-06-13T19:24:32,990][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
[2018-06-13T19:24:32,991][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
[2018-06-13T19:24:33,220][DEBUG][logstash.inputs.file] each: file grew: /opt/BOTS/botsv1.iis.json: old size 0, new size 28215738
[2018-06-13T19:24:34,099][DEBUG][logstash.pipeline] Pushing flush onto pipeline {:pipeline_id=>"main", :thread=>"#<Thread:0x15f64d55@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:246 sleep>"}
[2018-06-13T19:24:34,221][DEBUG][logstash.inputs.file] each: file grew: /opt/BOTS/botsv1.iis.json: old size 0, new size 28215738
[2018-06-13T19:24:35,224][DEBUG][logstash.inputs.file] each: file grew: /opt/BOTS/botsv1.iis.json: old size 0, new size 28215738
[2018-06-13T19:24:36,227][DEBUG][logstash.inputs.file] each: file grew: /opt/BOTS/botsv1.iis.json: old size 0, new size 28215738
[2018-06-13T19:24:37,229][DEBUG][logstash.inputs.file] each: file grew: /opt/BOTS/botsv1.iis.json: old size 0, new size 28215738

```

The part from the --\> (my addition) repeats over and over again until I stop LS. But no data ever reads.

Does this mean that LS has marked the file as read and therefore won't reread it?  
I was told that using sincedb\_path =\> "/dev/null" would allow logstash to re-read the file over and over again, but it doesn't seem to be working.

Suggestions? Am I missing something? Did I misconfigure?

Thanks

(running CentOS 7 with 8 GB RAM & Quad core i3 processor.)

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 14, 2018, 5:47am UTC](https://discuss.elastic.co/t/logstash-wont-read-file/135813/2 "2018-06-14T05:47:10Z")

</div>

> I was told that using sincedb\_path =\> "/dev/null" would allow logstash to re-read the file over and over again, but it doesn't seem to be working.

You also need `start_position => "beginning"`.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 12, 2018, 5:47am UTC](https://discuss.elastic.co/t/logstash-wont-read-file/135813/3 "2018-07-12T05:47:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
