# Error while uploading a Logfile using Logstash

**URL:** <https://discuss.elastic.co/t/error-while-uploading-a-logfile-using-logstash/135692>\
**Category:** Logstash\
**Created:** [June 13, 2018, 10:54am UTC](https://discuss.elastic.co/t/error-while-uploading-a-logfile-using-logstash/135692 "2018-06-13T10:54:43Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![helloWorld1](https://avatars.discourse-cdn.com/v4/letter/h/aca169/32.png) [@helloWorld1](https://discuss.elastic.co/u/helloWorld1)\
**Post date:** [June 13, 2018, 10:54am UTC](https://discuss.elastic.co/t/error-while-uploading-a-logfile-using-logstash/135692/1 "2018-06-13T10:54:43Z")

</div>

Hi All,

I have simple logs in the format:  
 \<[package.filename]\>

The problem I faced is that the Logfile sometimes fails to get uploaded and i have to make changes in the file to get it uploaded. Moreover capital letters are not allowed in naming of indexes which I'm not sure why.

The content of the script file I used:  
input {  
file {  
path =\> ["D:\logs\Log1.txt"]  
start\_position =\> "beginning"  
type=\> "log"  
}  
}

filter {

}

output {  
elasticsearch {  
action =\> "index"  
index =\> "app1-2018-06-11"  
hosts =\> ["localhost:9200"]  
document\_type =\> "log"

```
}

```

}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 13, 2018, 11:06am UTC](https://discuss.elastic.co/t/error-while-uploading-a-logfile-using-logstash/135692/2 "2018-06-13T11:06:36Z")

</div>

> The problem I faced is that the Logfile sometimes fails to get uploaded and i have to make changes in the file to get it uploaded.

Logstash remembers how far it has processed a file so if you need to reprocess a file you must clear its sincedb entry or set the file input's `sincedb_path` option to "nul" (WIndows) or "/dev/null" (non-Windows). See the file input documentation.

---

<div class="post-metadata">

**Author:** ![helloWorld1](https://avatars.discourse-cdn.com/v4/letter/h/aca169/32.png) [@helloWorld1](https://discuss.elastic.co/u/helloWorld1)\
**Post date:** [June 14, 2018, 8:36am UTC](https://discuss.elastic.co/t/error-while-uploading-a-logfile-using-logstash/135692/3 "2018-06-14T08:36:12Z")

</div>

Thanks for the reply.  
I just want to know that if I delete all the sincedb files will it make any difference apart from reparsing the log files again when I run logstash?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 14, 2018, 8:39am UTC](https://discuss.elastic.co/t/error-while-uploading-a-logfile-using-logstash/135692/4 "2018-06-14T08:39:18Z")

</div>

No, that's the only effect.

---

<div class="post-metadata">

**Author:** ![helloWorld1](https://avatars.discourse-cdn.com/v4/letter/h/aca169/32.png) [@helloWorld1](https://discuss.elastic.co/u/helloWorld1)\
**Post date:** [June 20, 2018, 8:36am UTC](https://discuss.elastic.co/t/error-while-uploading-a-logfile-using-logstash/135692/5 "2018-06-20T08:36:46Z")

</div>

Hey Magnus baeck,

Thanks for the help but I'm facing another issue i.e. the log file needs to be modified a little(anything like a simple newline or deleting a line) and only then does Logstash upload it.  
Any help regarding this?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 20, 2018, 9:46am UTC](https://discuss.elastic.co/t/error-while-uploading-a-logfile-using-logstash/135692/6 "2018-06-20T09:46:23Z")

</div>

It sounds like you didn't actually clear the sincedb state.

---

<div class="post-metadata">

**Author:** ![helloWorld1](https://avatars.discourse-cdn.com/v4/letter/h/aca169/32.png) [@helloWorld1](https://discuss.elastic.co/u/helloWorld1)\
**Post date:** [June 20, 2018, 10:14am UTC](https://discuss.elastic.co/t/error-while-uploading-a-logfile-using-logstash/135692/7 "2018-06-20T10:14:48Z")

</div>

I have added the sincedb\_path to null in the conf(script) file and removed the older sincedb files as well.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 20, 2018, 1:51pm UTC](https://discuss.elastic.co/t/error-while-uploading-a-logfile-using-logstash/135692/8 "2018-06-20T13:51:19Z")

</div>

> I have added the sincedb\_path to null

"null" or "nul"?

To debug what's going on when Logstash is reading file you can increase the loglevel.

---

<div class="post-metadata">

**Author:** ![helloWorld1](https://avatars.discourse-cdn.com/v4/letter/h/aca169/32.png) [@helloWorld1](https://discuss.elastic.co/u/helloWorld1)\
**Post date:** [June 21, 2018, 2:05pm UTC](https://discuss.elastic.co/t/error-while-uploading-a-logfile-using-logstash/135692/9 "2018-06-21T14:05:48Z")

</div>

this is the command written in input part of script:

sincedb\_path =\> "null"

the location of these files is:  
logstash-6.2.4\data\plugins\inputs\file

Is the location correct for the sincedb files?  
If yes then I've tried many times and no files are present in this folder.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 21, 2018, 2:09pm UTC](https://discuss.elastic.co/t/error-while-uploading-a-logfile-using-logstash/135692/10 "2018-06-21T14:09:54Z")

</div>

> sincedb\_path =\> "null"

This is wrong and will write the sincedb data to a file named "null". Use "nul" instead.

---

<div class="post-metadata">

**Author:** ![helloWorld1](https://avatars.discourse-cdn.com/v4/letter/h/aca169/32.png) [@helloWorld1](https://discuss.elastic.co/u/helloWorld1)\
**Post date:** [June 21, 2018, 2:27pm UTC](https://discuss.elastic.co/t/error-while-uploading-a-logfile-using-logstash/135692/11 "2018-06-21T14:27:41Z")

</div>

Oh!  
Thanks I'll try that.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 19, 2018, 2:27pm UTC](https://discuss.elastic.co/t/error-while-uploading-a-logfile-using-logstash/135692/12 "2018-07-19T14:27:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
