# Rereading a refreshed file

**URL:** <https://discuss.elastic.co/t/rereading-a-refreshed-file/169480>\
**Category:** Logstash\
**Created:** [February 21, 2019, 8:00pm UTC](https://discuss.elastic.co/t/rereading-a-refreshed-file/169480 "2019-02-21T20:00:25Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![lovecraft](https://avatars.discourse-cdn.com/v4/letter/l/a183cd/32.png) [@lovecraft](https://discuss.elastic.co/u/lovecraft)\
**Post date:** [February 21, 2019, 8:00pm UTC](https://discuss.elastic.co/t/rereading-a-refreshed-file/169480/1 "2019-02-21T20:00:25Z")

</div>

Hi All,

I am new to Logstash and trying to work out a config.

My scenario is that I have a utility which runs periodically and outputs a report as a multiline JSON object in a file. The filename remains the same across runs but the content is replaced with every run.

I want to take the content of the file and pass it into Elastic search as a single document. So there is a one document per run of the utility.

I have a config which does this BUT it only does it once at the startup of Logstash. When the content of the file is updated a reread of the file is not triggered.

I suspect the file is being 'unwatched' after the first run. Can someone suggest a better config please?

Thanks++

Here is my config:

```
input { 
    file {
        codec => json
        mode => read
        delimiter => "EOF"
        path => "/journals/journal.json"
    } 
} 
output { 
    stdout{
        codec => rubydebug
    }
    elasticsearch {
        index => "host-journals"
        document_type => "default"
        hosts => ["http://127.0.0.1:9200"]
    } 
}
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 21, 2019, 8:33pm UTC](https://discuss.elastic.co/t/rereading-a-refreshed-file/169480/2 "2019-02-21T20:33:03Z")

</div>

I don't think you can do that using a file input.

You could use an exec input to periodically cat the file, and then use a fingerprint filter to set the document\_id option on the elasticsearch output, so that if it re-reads the file with the same content it overwrites the same document in elasticsearch.

---

<div class="post-metadata">

**Author:** ![lovecraft](https://avatars.discourse-cdn.com/v4/letter/l/a183cd/32.png) [@lovecraft](https://discuss.elastic.co/u/lovecraft)\
**Post date:** [February 21, 2019, 9:53pm UTC](https://discuss.elastic.co/t/rereading-a-refreshed-file/169480/3 "2019-02-21T21:53:29Z")

</div>

Thanks for the tip!

Some shell slight of hand seems to work just fine with the exec input...

```
input {
    exec {
        command => "if [-f /journals/journal.json] ;then /bin/cat /journals/journal.json && mv /journals/journal.json /journals/journal.`date +%Y-%m-%d.%H:%M:%S` ; fi"
        interval => 30
    }
}

```

I used the drop filter to prevent Logstash from pushing anything to elasticsearch if the exec'd command produces no output...

```
filter {
  if [message] == "" {
    drop { }
  }
}

```

This all seems to do what I need it to do. Profuse thanks!

-Seàn

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 21, 2019, 10:07pm UTC](https://discuss.elastic.co/t/rereading-a-refreshed-file/169480/4 "2019-03-21T22:07:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.