# Logstash doesn't parse new files in directory

**URL:** <https://discuss.elastic.co/t/logstash-doesnt-parse-new-files-in-directory/345197>\
**Category:** Logstash\
**Created:** [October 17, 2023, 10:52am UTC](https://discuss.elastic.co/t/logstash-doesnt-parse-new-files-in-directory/345197 "2023-10-17T10:52:15Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Xhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xhar/32/126590_2.png) [@Xhar](https://discuss.elastic.co/u/Xhar)\
**Post date:** [October 17, 2023, 10:52am UTC](https://discuss.elastic.co/t/logstash-doesnt-parse-new-files-in-directory/345197/1 "2023-10-17T10:52:15Z")

</div>

Logstash doesn't parse new logs files in directory  
Here's my config

```auto
input {

  file{
    path => "/home/user/Documents/bdu/*.json"
    sincedb_path => "/dev/null"
    type => "bdu"
    codec => "json"
 }

}

output {

    if [type] == "bdu"{
	opensearch{  
	hosts => ["http:// *****"]
	index => "psdoi-bdu-%{+YYYY.MM.dd}"
	document_id => "%{ID}"
	action => "update"
	doc_as_upsert => true	
        }
      }

stdout { codec => rubydebug }
}

```

i need it to send every new file in bdu directory to the opensearch, but it works only after changing some files

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 17, 2023, 10:52am UTC](https://discuss.elastic.co/t/logstash-doesnt-parse-new-files-in-directory/345197/2 "2023-10-17T10:52:15Z")

</div>

OpenSearch/OpenDistro are AWS run products and differ from the original Elasticsearch and Kibana products that Elastic builds and maintains. You may need to contact them directly for further assistance.

(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: )

---

<div class="post-metadata">

**Author:** ![Matthias\_Brauchle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthias_brauchle/32/123249_2.png) [@Matthias\_Brauchle](https://discuss.elastic.co/u/Matthias_Brauchle)\
**Post date:** [October 17, 2023, 10:56am UTC](https://discuss.elastic.co/t/logstash-doesnt-parse-new-files-in-directory/345197/3 "2023-10-17T10:56:19Z")

</div>

Can you try to use "read" mode and start from "beginning"?

[Documentation](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html)

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [October 17, 2023, 11:05am UTC](https://discuss.elastic.co/t/logstash-doesnt-parse-new-files-in-directory/345197/4 "2023-10-17T11:05:56Z")

</div>

I don't see any need for `type => "bdu"` and remove from output

```auto
input {
  file {
    path => "/home/user/Documents/bdu/*.json" # is it documents or Documents?
    start_position => "beginning"
    sincedb_path => "/dev/null"
    codec => "json"
  }
}
output {
stdout { codec => rubydebug }
}

```

First start with debug, then go to opensearch.If this is multiline JSON, you have to use the multiline codec, check [the sample](https://discuss.elastic.co/t/still-facing-an-issue-with-json-multiline/344168/2).

---

<div class="post-metadata">

**Author:** ![Xhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xhar/32/126590_2.png) [@Xhar](https://discuss.elastic.co/u/Xhar)\
**Post date:** [October 17, 2023, 11:13am UTC](https://discuss.elastic.co/t/logstash-doesnt-parse-new-files-in-directory/345197/5 "2023-10-17T11:13:16Z")

</div>

that's really works, thank you

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 14, 2023, 11:13am UTC](https://discuss.elastic.co/t/logstash-doesnt-parse-new-files-in-directory/345197/6 "2023-11-14T11:13:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.