# How to have logstash update logs as they are coming in from logstash and send to kibana graphs automatically

**URL:** <https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383>\
**Category:** Logstash\
**Created:** [August 29, 2019, 4:58pm UTC](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383 "2019-08-29T16:58:49Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![nizztizz](https://avatars.discourse-cdn.com/v4/letter/n/439d5e/32.png) [@nizztizz](https://discuss.elastic.co/u/nizztizz)\
**Post date:** [August 29, 2019, 4:58pm UTC](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383/1 "2019-08-29T16:58:49Z")

</div>

I want logstash to automatically update and run as new logs are coming in from a web application and send them to kibana's graphs. As of right now I have to manually run logstash and once it finishes running it shows up in kibana. I want it to live parse the logs. How would I go about doing this?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 29, 2019, 5:15pm UTC](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383/2 "2019-08-29T17:15:25Z")

</div>

What input are you using?

---

<div class="post-metadata">

**Author:** ![nizztizz](https://avatars.discourse-cdn.com/v4/letter/n/439d5e/32.png) [@nizztizz](https://discuss.elastic.co/u/nizztizz)\
**Post date:** [August 29, 2019, 5:28pm UTC](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383/3 "2019-08-29T17:28:52Z")

</div>

Tomcat access logs to elastic search, logstash, kibana.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 29, 2019, 5:36pm UTC](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383/4 "2019-08-29T17:36:39Z")

</div>

In your logstash configuration, what does the input section look like?

---

<div class="post-metadata">

**Author:** ![nizztizz](https://avatars.discourse-cdn.com/v4/letter/n/439d5e/32.png) [@nizztizz](https://discuss.elastic.co/u/nizztizz)\
**Post date:** [August 29, 2019, 5:45pm UTC](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383/5 "2019-08-29T17:45:55Z")

</div>

input {  
file {  
path =\> ["/route/app/file/tomcat/logs/\*"]  
start\_position =\> "beginning"  
sincedb\_path =\> "NUL"

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 29, 2019, 5:53pm UTC](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383/6 "2019-08-29T17:53:21Z")

</div>

> [@nizztizz](#):
>
> path =\> ["/route/app/file/tomcat/logs/\*"]

If you leave logstash running, possibly as a service, it will keep reading lines from the logs as data is appended to files or new files are created.

---

<div class="post-metadata">

**Author:** ![nizztizz](https://avatars.discourse-cdn.com/v4/letter/n/439d5e/32.png) [@nizztizz](https://discuss.elastic.co/u/nizztizz)\
**Post date:** [August 29, 2019, 6:34pm UTC](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383/7 "2019-08-29T18:34:47Z")

</div>

So it automatically does it?

---

<div class="post-metadata">

**Author:** ![nizztizz](https://avatars.discourse-cdn.com/v4/letter/n/439d5e/32.png) [@nizztizz](https://discuss.elastic.co/u/nizztizz)\
**Post date:** [August 29, 2019, 6:38pm UTC](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383/8 "2019-08-29T18:38:23Z")

</div>

And once i parse through all the logs and set them in graphs how would I set it so that it only takes new logs created and send it to the graphs instead of starting the whole service again

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 29, 2019, 6:51pm UTC](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383/9 "2019-08-29T18:51:58Z")

</div>

That is what the sincedb is for. If you remove the sincedb\_path option then when it stops it will write out to disk (you do not need to worry where) how much of each file it has read. When it starts again it will continue reading each file from that point.

---

<div class="post-metadata">

**Author:** ![nizztizz](https://avatars.discourse-cdn.com/v4/letter/n/439d5e/32.png) [@nizztizz](https://discuss.elastic.co/u/nizztizz)\
**Post date:** [August 29, 2019, 7:05pm UTC](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383/10 "2019-08-29T19:05:36Z")

</div>

So when I removed sincedb by logs parse fine but when I had sincedb i get an error and my logs don't parse. It says failed to execute action.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 29, 2019, 7:10pm UTC](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383/11 "2019-08-29T19:10:28Z")

</div>

> [@nizztizz](#):
>
> when I had sincedb i get an error

What error, exactly?

---

<div class="post-metadata">

**Author:** ![nizztizz](https://avatars.discourse-cdn.com/v4/letter/n/439d5e/32.png) [@nizztizz](https://discuss.elastic.co/u/nizztizz)\
**Post date:** [August 29, 2019, 10:16pm UTC](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383/12 "2019-08-29T22:16:12Z")

</div>

I want logstash to parse the data where it left off. So if it parsed all of january's logs already, I want it to start from february since it already parsed the previous logs instead of starting over.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 29, 2019, 10:31pm UTC](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383/13 "2019-08-29T22:31:10Z")

</div>

That's what it would normally do with a file input with a sincedb.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 26, 2019, 10:31pm UTC](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383/14 "2019-09-26T22:31:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
