# Unable to read logs files

**URL:** https://discuss.elastic.co/t/unable-to-read-logs-files/273891
**Category:** Logstash
**Created:** [May 25, 2021, 6:12am UTC](https://discuss.elastic.co/t/unable-to-read-logs-files/273891 "2021-05-25T06:12:48Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![legolas\_bilbao](https://avatars.discourse-cdn.com/v4/letter/l/3da27b/32.png) [@legolas\_bilbao](https://discuss.elastic.co/u/legolas_bilbao)
#### Post date: [May 25, 2021, 6:12am UTC](https://discuss.elastic.co/t/unable-to-read-logs-files/273891/1 "2021-05-25T06:12:48Z")

</div>

Good morning,

I'm trying to read log files from a concrete path. My configuration is the following

```
input {
        file {
              path => "/path/proxy/access2.log"
              start_position => "beginning"
        }
}
filter {
        grok {

match => ["message","%{NUMBER:timestamp}%{SPACE}%{NUMBER:duration}\s%{IP:client_address}\s%{WORD:cache_result}/%{POSINT:status_code}\s%{NUMBER:bytes}\s%{WORD:request_method}\s%{NOTSPACE:url}\s%{NOTSPACE:user}\s%{WORD:hierarchy_code}/(%{IP:dst_addrss})?%{NOTSPACE:server}\s%{NOTSPACE:content_type}"]
                }
        date {
                match => ["timestamp", "UNIX"]
                remove_field => ["timestamp"]
             }

        }

output {
  elasticsearch {
  "hosts" => "localhost:9200"
  "index" => "squid"
  }
stdout { }
}

```

I've set the debug log in logstash and appears the following log

```
[2021-05-25T08:07:02,420][INFO][logstash.javapipeline][squid] Pipeline Java execution initialization time {"seconds"=>0.74}
[2021-05-25T08:07:02,633][TRACE][logstash.inputs.file][squid] Registering file input {:path=>["/home/jallende/proxy/access2.log"]}
[2021-05-25T08:07:05,928][INFO][logstash.inputs.file][squid] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"/var/lib/logstash/plugins/inputs/file/.sincedb_5154ac82e8774c7923b7ced1fd141b67", :path=>["/home/jallende/proxy/access2.log"]}
[2021-05-25T08:07:05,938][INFO][logstash.javapipeline][squid] Pipeline started {"pipeline.id"=>"squid"}
[2021-05-25T08:07:05,940][DEBUG][org.logstash.execution.PeriodicFlush][squid] Pushing flush onto pipeline.
[2021-05-25T08:07:05,947][DEBUG][logstash.javapipeline] Pipeline started successfully {:pipeline_id=>"squid", :thread=>"#<Thread:0x6cf5c4d9 sleep>"}
[2021-05-25T08:07:05,950][TRACE][logstash.agent] Converge results {:success=>true, :failed_actions=>[], :successful_actions=>["id: reglas, action_type: LogStash::PipelineAction::Create", "id: vpn, action_type: LogStash::PipelineAction::Create", "id: arcsight, action_type: LogStash::PipelineAction::Create", "id: eps, action_type: LogStash::PipelineAction::Create", "id: squid, action_type: LogStash::PipelineAction::Create"]}
[2021-05-25T08:07:05,969][INFO][logstash.agent] Pipelines running {:count=>5, :running_pipelines=>[:reglas, :vpn, :arcsight, :eps, :squid], :non_running_pipelines=>[]}
[2021-05-25T08:07:05,973][INFO][filewatch.observingtail][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] START, creating Discoverer, Watch with file and sincedb collections
[2021-05-25T08:07:05,988][DEBUG][filewatch.sincedbcollection][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] open: reading from /var/lib/logstash/plugins/inputs/file/.sincedb_5154ac82e8774c7923b7ced1fd141b67
[2021-05-25T08:07:05,990][TRACE][filewatch.sincedbcollection][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] open: count of keys read: 0
[2021-05-25T08:07:06,006][TRACE][filewatch.discoverer][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] discover_files {:count=>0}
[2021-05-25T08:07:07,023][DEBUG][filewatch.sincedbcollection][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] writing sincedb (delta since last write = 1621922827)
[2021-05-25T08:07:07,027][TRACE][filewatch.sincedbcollection][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] sincedb_write: /var/lib/logstash/plugins/inputs/file/.sincedb_5154ac82e8774c7923b7ced1fd141b67 (time = 2021-05-25 08:07:07 +0200)
[2021-05-25T08:07:10,939][DEBUG][org.logstash.execution.PeriodicFlush][squid] Pushing flush onto pipeline.
[2021-05-25T08:07:15,939][DEBUG][org.logstash.execution.PeriodicFlush][squid] Pushing flush onto pipeline.
[2021-05-25T08:07:20,044][TRACE][filewatch.discoverer][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] discover_files {:count=>0}
[2021-05-25T08:07:20,939][DEBUG][org.logstash.execution.PeriodicFlush][squid] Pushing flush onto pipeline.
[2021-05-25T08:07:22,045][DEBUG][filewatch.sincedbcollection][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] writing sincedb (delta since last write = 15)
[2021-05-25T08:07:22,046][TRACE][filewatch.sincedbcollection][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] sincedb_write: /var/lib/logstash/plugins/inputs/file/.sincedb_5154ac82e8774c7923b7ced1fd141b67 (time = 2021-05-25 08:07:22 +0200)
[2021-05-25T08:07:25,939][DEBUG][org.logstash.execution.PeriodicFlush][squid] Pushing flush onto pipeline.
[2021-05-25T08:07:30,939][DEBUG][org.logstash.execution.PeriodicFlush][squid] Pushing flush onto pipeline.
[2021-05-25T08:07:35,049][TRACE][filewatch.discoverer][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] discover_files {:count=>0}
[2021-05-25T08:07:35,939][DEBUG][org.logstash.execution.PeriodicFlush][squid] Pushing flush onto pipeline.
[2021-05-25T08:07:37,050][DEBUG][filewatch.sincedbcollection][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] writing sincedb (delta since last write = 15)
[2021-05-25T08:07:37,050][TRACE][filewatch.sincedbcollection][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] sincedb_write: /var/lib/logstash/plugins/inputs/file/.sincedb_5154ac82e8774c7923b7ced1fd141b67 (time = 2021-05-25 08:07:37 +0200)
[2021-05-25T08:07:40,939][DEBUG][org.logstash.execution.PeriodicFlush][squid] Pushing flush onto pipeline.
[2021-05-25T08:07:45,939][DEBUG][org.logstash.execution.PeriodicFlush][squid] Pushing flush onto pipeline.
[2021-05-25T08:07:50,053][TRACE][filewatch.discoverer][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] discover_files {:count=>0}
[2021-05-25T08:07:50,939][DEBUG][org.logstash.execution.PeriodicFlush][squid] Pushing flush onto pipeline.
[2021-05-25T08:07:52,054][DEBUG][filewatch.sincedbcollection][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] writing sincedb (delta since last write = 15)
[2021-05-25T08:07:52,054][TRACE][filewatch.sincedbcollection][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] sincedb_write: /var/lib/logstash/plugins/inputs/file/.sincedb_5154ac82e8774c7923b7ced1fd141b67 (time = 2021-05-25 08:07:52 +0200)
[2021-05-25T08:07:55,939][DEBUG][org.logstash.execution.PeriodicFlush][squid] Pushing flush onto pipeline.
[2021-05-25T08:08:00,939][DEBUG][org.logstash.execution.PeriodicFlush][squid] Pushing flush onto pipeline.
[2021-05-25T08:08:05,057][TRACE][filewatch.discoverer][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] discover_files {:count=>0}
[2021-05-25T08:08:05,939][DEBUG][org.logstash.execution.PeriodicFlush][squid] Pushing flush onto pipeline.
[2021-05-25T08:08:07,058][DEBUG][filewatch.sincedbcollection][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] writing sincedb (delta since last write = 15)
[2021-05-25T08:08:07,058][TRACE][filewatch.sincedbcollection][squid][ce608fea0833a3776a6f3a4c4cd5d54e390a7df1f53b075797fe3227dd8c7923] sincedb_write: /var/lib/logstash/plugins/inputs/file/.sincedb_5154ac82e8774c7923b7ced1fd141b67 (time = 2021-05-25 08:08:07 +0200)

```

The sincedb file is empty and the index information in kibana is emtpy too , any idea?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 25, 2021, 2:12pm UTC](https://discuss.elastic.co/t/unable-to-read-logs-files/273891/2 "2021-05-25T14:12:51Z")

</div>

> [@legolas\_bilbao](#):
>
> `discover_files {:count=>0}`

The file input is not finding a file that matches `"/path/proxy/access2.log"`

---

<div class="post-metadata">

### Author: ![legolas\_bilbao](https://avatars.discourse-cdn.com/v4/letter/l/3da27b/32.png) [@legolas\_bilbao](https://discuss.elastic.co/u/legolas_bilbao)
#### Post date: [May 25, 2021, 2:15pm UTC](https://discuss.elastic.co/t/unable-to-read-logs-files/273891/3 "2021-05-25T14:15:11Z")

</div>

sorry, I've forget to answer, the problem resides in the permission

---

<div class="post-metadata">

### Author: ![legolas\_bilbao](https://avatars.discourse-cdn.com/v4/letter/l/3da27b/32.png) [@legolas\_bilbao](https://discuss.elastic.co/u/legolas_bilbao)
#### Post date: [May 25, 2021, 8:30pm UTC](https://discuss.elastic.co/t/unable-to-read-logs-files/273891/4 "2021-05-25T20:30:25Z")

</div>

I think the problem is in the permission but I've other problem cos if i run in this way /usr/share/logstash/bin/logstash --debug -f /etc/logstash/squid.conf its works but if i run in service logstash start the log file isnt read. Any idea?

regards

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 25, 2021, 8:51pm UTC](https://discuss.elastic.co/t/unable-to-read-logs-files/273891/5 "2021-05-25T20:51:28Z")

</div>

Use ps to see which user the service is running as. Check each element of the path to make sure that the user has x access to the directories and r access to the log.

---

<div class="post-metadata">

### Author: ![legolas\_bilbao](https://avatars.discourse-cdn.com/v4/letter/l/3da27b/32.png) [@legolas\_bilbao](https://discuss.elastic.co/u/legolas_bilbao)
#### Post date: [May 26, 2021, 5:48am UTC](https://discuss.elastic.co/t/unable-to-read-logs-files/273891/6 "2021-05-26T05:48:37Z")

</div>

Process as logstash  
folders and files has 777 permission  
drwxrwxrwx. 2 user user 24 may 26 07:04 proxy  
-rwxrwxrwx. 1 user user 43974656 may 26 07:03 access.log

any idea?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 23, 2021, 5:49am UTC](https://discuss.elastic.co/t/unable-to-read-logs-files/273891/7 "2021-06-23T05:49:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
