# Selecting "file" in logstash input does not send to elasticsearch
**URL:** https://discuss.elastic.co/t/selecting-file-in-logstash-input-does-not-send-to-elasticsearch/265602
**Category:** Logstash
**Created:** [February 26, 2021, 11:18am UTC](https://discuss.elastic.co/t/selecting-file-in-logstash-input-does-not-send-to-elasticsearch/265602 "2021-02-26T11:18:44Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)
#### Post date: [February 26, 2021, 11:18am UTC](https://discuss.elastic.co/t/selecting-file-in-logstash-input-does-not-send-to-elasticsearch/265602/1 "2021-02-26T11:18:44Z")
I am trying to send log files from logstash to elasticsearch.
I have completed the whole setup and the test to send the standard input to elasticsearch has passed.
Now, I want to send the file to elasticsearch by specifying the file, but nothing is sent to elasticsearch.
Logstash seems to be working fine.
(At least it's not outputting any errors at runtime.
What do I need to check and fix to send the file to elasticsearch?
Here is the logstash config file that I created.
logstash-sample.conf
```auto
input {
#stdin { } <-OK
file {
path => ["/usr/local/elastic/datasets/logstash-tutorial.log"]
}
}
output {
elasticsearch {
hosts => ["http:// ***.***. ***.*** :9200"]
index => "test-%{+YYYY.MM.dd}"
}
}
```
Then, run the following.
`/usr/share/logstash/bin/logstash -f /etc/logstash/logstash-sample.conf --config.reload.automatic`
---
### Author:  [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)
#### Post date: [February 26, 2021, 11:36am UTC](https://discuss.elastic.co/t/selecting-file-in-logstash-input-does-not-send-to-elasticsearch/265602/2 "2021-02-26T11:36:49Z")
Hi,
Is the file already complete or are you adding additional content after LogStash started?
By default, LogStash tails files and starts at the end if the files already exist: [File input plugin | Logstash Reference [7.11] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-start_position)
Depending on your usecase you might do the following:
1. If your files are always content complete you might want to switch the `mode` from `tail` to `read`: [File input plugin | Logstash Reference [7.11] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-mode)
2. If your files are not content complete(e.g. logfiles) but you do not want the current content to be lost switch the `start_position` of logstash to `beginning`: [File input plugin | Logstash Reference [7.11] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-start_position)
Note that the start position is only used if logstash finds a new file. After that LogStash will persist the current position in the [sincedb](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-sincedb_path).
Best regards
Wolfram
---
### Author:  [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)
#### Post date: [March 1, 2021, 1:20am UTC](https://discuss.elastic.co/t/selecting-file-in-logstash-input-does-not-send-to-elasticsearch/265602/3 "2021-03-01T01:20:27Z")
Thank you for your prompt reply.
In the above sample, I used an "already completed" file.
However, it is not complete because what I want to put into practical use in the future is the log file.
I have tried the two cases you gave me.
```auto
mode => "tail"
start_position => "beginning"
```
But in both cases, the result is the same: no logs are sent to elasticsearch.
Do you have any other ideas?
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [March 1, 2021, 1:39am UTC](https://discuss.elastic.co/t/selecting-file-in-logstash-input-does-not-send-to-elasticsearch/265602/4 "2021-03-01T01:39:13Z")
Logstash will not read that file again because it already had.
As @Wolfram_Haussig suggested you should read / understand
about the [sincedb here](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-sincedb_path)
To reread that file you need to clean out the sincedb data manually or while testing add
`sincedb_path => "/dev/null"`
In the file input section, this is good for testing then take out for production.
Depending on you system you may need to start Logstash with `sudo` when using `/dev/null`
---
### Author:  [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)
#### Post date: [March 1, 2021, 5:28am UTC](https://discuss.elastic.co/t/selecting-file-in-logstash-input-does-not-send-to-elasticsearch/265602/5 "2021-03-01T05:28:10Z")
@stephenb
Thanks for your comment.
You are right! I heard very good things!
I specified the sincedb\_path as you suggested and confirmed that it is registered in elasticsearch.
If I want to benefit from sincedb, what should I specify for sincedb\_path?
If I am running logstash for the first time, I don't think sincedb exists, do you?
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [March 1, 2021, 5:34am UTC](https://discuss.elastic.co/t/selecting-file-in-logstash-input-does-not-send-to-elasticsearch/265602/6 "2021-03-01T05:34:32Z")
sincedb will be automatically created in the right place if you do not set it ... The sincedb\_path will be shown in the Logstash logs during startup, you can set it if you want but really no need.
If I recall the sincedb is created upon starting Logstash the first time.
Remember /dev/null is really for dev and testing .
---
### Author:  [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)
#### Post date: [March 1, 2021, 5:45am UTC](https://discuss.elastic.co/t/selecting-file-in-logstash-input-does-not-send-to-elasticsearch/265602/7 "2021-03-01T05:45:54Z")
I kind of understood that sincedb is automatically created in the right place.
When I searched for sincedb, I found that in my environment
`/usr/share/logstash/data/plugins/inputs/file/.sincedb__ ****************** `
In my environment, there are several files called "sincedb".
I think some of them are from rewriting the config and running logstash several times.
However, this means that when I clean up sincedb to reload the files
I don't know which ones to delete.
Is there any way to determine these?
Or is it possible to name the sincedb for each type of log?
For example, if I want to run nginx, apache
`sincedb_nginx`, `sincedb_apache` or
`nginx/sincedb__ ****** `
`apache/sincedb_ ****** `
, etc.
---
### Author:  [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)
#### Post date: [March 1, 2021, 6:53am UTC](https://discuss.elastic.co/t/selecting-file-in-logstash-input-does-not-send-to-elasticsearch/265602/8 "2021-03-01T06:53:43Z")
> [@its-ogawa](#):
>
> Or is it possible to name the sincedb for each type of log?
Yes, as described in the documentation the config setting `sincedb_path`points to a file path for this specific file input sincedb. So, setting `sincedb_path` to `/nginx/sincedb` would work.
Otherwise, as @stephenb has pointed out you do not necessarily need to delete the sincedb. You could temporarily point your `sincedb_path` to `/dev/null` for reading the existing entries. after this is done you can reset this config change and restart LogStash. After this, LogStash will read all new entries as expected.
---
### Author:  [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)
#### Post date: [March 1, 2021, 8:04am UTC](https://discuss.elastic.co/t/selecting-file-in-logstash-input-does-not-send-to-elasticsearch/265602/9 "2021-03-01T08:04:24Z")
Thank you for your comment.
I have confirmed that sincedb can be placed in any directory.
I'll specify `/dev/null` during the test of file reading, and create sincedb in the given path during operation.
Finally, I will post the logstash-sample.conf that I created.
```auto
input {
file {
path => ["/var/log/nginx/*.log"]
start_position => "beginning"
#sincedb_path => "/dev/null"
sincedb_path => "/usr/local/elastic/logstash/nginx/sincedb" # Any file path
}
}
output {
elasticsearch {
hosts => ["http:// ***.***. ***.*** :9200"]
index => "test-%{+YYYY.MM.dd}"
}
}
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 29, 2021, 8:04am UTC](https://discuss.elastic.co/t/selecting-file-in-logstash-input-does-not-send-to-elasticsearch/265602/10 "2021-03-29T08:04:36Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.