# Logstash file input not working for large file

**URL:** <https://discuss.elastic.co/t/logstash-file-input-not-working-for-large-file/133414>\
**Category:** Logstash\
**Created:** [May 27, 2018, 6:40am UTC](https://discuss.elastic.co/t/logstash-file-input-not-working-for-large-file/133414 "2018-05-27T06:40:44Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![muhammad](https://avatars.discourse-cdn.com/v4/letter/m/b487fb/32.png) [@muhammad](https://discuss.elastic.co/u/muhammad)\
**Post date:** [May 27, 2018, 6:40am UTC](https://discuss.elastic.co/t/logstash-file-input-not-working-for-large-file/133414/1 "2018-05-27T06:40:44Z")

</div>

Hello,  
I am using elk (v5.6.9-1) . I have a file of size 1.3G containing debug logs. When I take a small snippet of file(eg. create a file with 10 lines of the original) , then it correctly reads the file and indexes it into elasticsearch(as expected), but when I use the whole file(i.e as it is ,without trimming) in the logstash-file-input , it does nothing and the debug logs show something like:-

```
[2018-05-27T10:27:27,526][INFO][logstash.pipeline] Starting pipeline {"id"=>"main", "pipeline.workers"=>24, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>3000}
[2018-05-27T10:27:27,723][INFO][logstash.pipeline] Pipeline main started
[2018-05-27T10:27:27,733][DEBUG][logstash.agent] Starting puma
[2018-05-27T10:27:27,733][DEBUG][logstash.inputs.file] _globbed_files: /root/Downloads/debug: glob is: ["/root/Downloads/debug"]
[2018-05-27T10:27:27,735][DEBUG][logstash.agent] Trying to start WebServer {:port=>9600}
[2018-05-27T10:27:27,735][DEBUG][logstash.inputs.file] _discover_file: /root/Downloads/debug: new: /root/Downloads/debug (exclude is [])
[2018-05-27T10:27:27,736][DEBUG][logstash.api.service] [api-service] start
[2018-05-27T10:27:27,736][DEBUG][logstash.inputs.file] _open_file: /root/Downloads/debug: opening
[2018-05-27T10:27:27,737][DEBUG][logstash.inputs.file] /root/Downloads/debug: sincedb last value 1324520770, cur size 1324520770
[2018-05-27T10:27:27,737][DEBUG][logstash.inputs.file] /root/Downloads/debug: sincedb: seeking to 1324520770
[2018-05-27T10:27:27,758][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2018-05-27T10:27:32,725][DEBUG][logstash.pipeline] Pushing flush onto pipeline
[2018-05-27T10:27:37,725][DEBUG][logstash.pipeline] Pushing flush onto pipeline
[2018-05-27T10:27:41,752][DEBUG][logstash.inputs.file] _globbed_files: /root/Downloads/debug: glob is: ["/root/Downloads/debug"]
[2018-05-27T10:27:42,726][DEBUG][logstash.pipeline] Pushing flush onto pipeline
[2018-05-27T10:27:47,726][DEBUG][logstash.pipeline] Pushing flush onto pipeline
[2018-05-27T10:27:52,727][DEBUG][logstash.pipeline] Pushing flush onto pipeline
[2018-05-27T10:27:56,764][DEBUG][logstash.inputs.file] _globbed_files: /root/Downloads/debug: glob is: ["/root/Downloads/debug"]
[2018-05-27T10:27:57,728][DEBUG][logstash.pipeline] Pushing flush onto pipeline
[2018-05-27T10:28:02,728][DEBUG][logstash.pipeline] Pushing flush onto pipeline
[2018-05-27T10:28:07,728][DEBUG][logstash.pipeline] Pushing flush onto pipeline

```

My logstash configuration is:-

```
input {
file {    
        path => "/root/Downloads/debug"    
        type => "juniper_poc"
        start_position => "beginning"
    }
}
filter {
  if [type] == "juniper_poc" {
    grok {
          patterns_dir => "./patterns"
          match => { "message" => [ "%{DATESTAMP:log_date} \(%{NUMBER:pid}\): %{GREEDYDATA:message}"
                       ]} 
                     overwrite => ["message"]
    }
      date {
        timezone => "Asia/Karachi"
        match => ["log_date", "MM/dd/yy HH:mm:ss.SSS" , "ISO8601"]
    }
mutate {
    remove_field => ["@version" ,"log_date","path"]
}
}
}
output{
stdout { codec => rubydebug } 
elasticsearch { hosts => "http://192.168.100.10:9200"}
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 24, 2018, 6:40am UTC](https://discuss.elastic.co/t/logstash-file-input-not-working-for-large-file/133414/2 "2018-06-24T06:40:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.