# Logstash creating 2 events for each line in json codec

**URL:** <https://discuss.elastic.co/t/logstash-creating-2-events-for-each-line-in-json-codec/68104>\
**Category:** Logstash\
**Created:** [December 5, 2016, 8:36pm UTC](https://discuss.elastic.co/t/logstash-creating-2-events-for-each-line-in-json-codec/68104 "2016-12-05T20:36:59Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![krishna\_chaitanya](https://avatars.discourse-cdn.com/v4/letter/k/b5a626/32.png) [@krishna\_chaitanya](https://discuss.elastic.co/u/krishna_chaitanya)\
**Post date:** [December 5, 2016, 8:36pm UTC](https://discuss.elastic.co/t/logstash-creating-2-events-for-each-line-in-json-codec/68104/1 "2016-12-05T20:36:59Z")

</div>

I want to stream each json line being written to a log file into Logstash. I am using json codec in file input plugin.

If I want to load entire file which is new at once, I can do with `start_position => beginning`  
But I want to stream each line as soon as it is written and not entire file at once.

I tried lot of combinations.  
`start_position => end` and `sincedb_path => dev/null`  
Only `start_position => end`  
Only `start_position => beginning`

In each case, whenever a linr is written to the file and I save the file, the previous line written is also getting written. So, this is creating duplicate events. How can I avoid this, and create one event per line and stream into Logstash?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 5, 2016, 9:03pm UTC](https://discuss.elastic.co/t/logstash-creating-2-events-for-each-line-in-json-codec/68104/2 "2016-12-05T21:03:55Z")

</div>

> In each case, whenever a linr is written to the file and I save the file, the previous line written is also getting written.

When you save a file with a text editor it typically writes a new file and renames it over the old file. This creates a new file that Logstash will parse from the beginning if you set `start_position => beginning`. When testing make sure you append to the file with e.g. `echo something >> filename.log`.

---

<div class="post-metadata">

**Author:** ![krishna\_chaitanya](https://avatars.discourse-cdn.com/v4/letter/k/b5a626/32.png) [@krishna\_chaitanya](https://discuss.elastic.co/u/krishna_chaitanya)\
**Post date:** [December 5, 2016, 9:55pm UTC](https://discuss.elastic.co/t/logstash-creating-2-events-for-each-line-in-json-codec/68104/3 "2016-12-05T21:55:45Z")

</div>

Thanks Magnus. That worked for me. How do we append to the file input in production? Can we try with `echo`?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 6, 2016, 6:34am UTC](https://discuss.elastic.co/t/logstash-creating-2-events-for-each-line-in-json-codec/68104/4 "2016-12-06T06:34:33Z")

</div>

> How do we append to the file input in production?

In production I assume you have a program that appends to the log file for you? Or what's the scenario?

> Can we try with echo?

You can pick any method that appends to the file instead of creating a new one.

---

<div class="post-metadata">

**Author:** ![krishna\_chaitanya](https://avatars.discourse-cdn.com/v4/letter/k/b5a626/32.png) [@krishna\_chaitanya](https://discuss.elastic.co/u/krishna_chaitanya)\
**Post date:** [December 6, 2016, 5:21pm UTC](https://discuss.elastic.co/t/logstash-creating-2-events-for-each-line-in-json-codec/68104/5 "2016-12-06T17:21:14Z")

</div>

Thanks magnus

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 3, 2017, 5:21pm UTC](https://discuss.elastic.co/t/logstash-creating-2-events-for-each-line-in-json-codec/68104/6 "2017-01-03T17:21:14Z")

</div>

Closed on January 3, 2017, 5:21pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
