# How to insert only newly added rows in csv in logstash in old index

**URL:** <https://discuss.elastic.co/t/how-to-insert-only-newly-added-rows-in-csv-in-logstash-in-old-index/278866>\
**Category:** Logstash\
**Created:** [July 16, 2021, 6:55am UTC](https://discuss.elastic.co/t/how-to-insert-only-newly-added-rows-in-csv-in-logstash-in-old-index/278866 "2021-07-16T06:55:12Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sarthak\_Mishra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sarthak_mishra/32/82772_2.png) [@Sarthak\_Mishra](https://discuss.elastic.co/u/Sarthak_Mishra)\
**Post date:** [July 16, 2021, 6:55am UTC](https://discuss.elastic.co/t/how-to-insert-only-newly-added-rows-in-csv-in-logstash-in-old-index/278866/1 "2021-07-16T06:55:12Z")

</div>

Hi community,

If I have a csv with 5 rows and I have shipped it in logstash. In future, the csv gets updated with 5(old) + 2(new) =7 rows. I want to ship the updated csv in the same index. Can I ship only the updated 2(new) rows of the csv file with the 5(old) rows already preexisting in the index? If yes, please help.

I want to visualise the old and new rows in kibana with the help of the timestamp

---

<div class="post-metadata">

**Author:** ![Krunal](https://avatars.discourse-cdn.com/v4/letter/k/ce7236/32.png) [@Krunal](https://discuss.elastic.co/u/Krunal)\
**Post date:** [July 16, 2021, 7:14am UTC](https://discuss.elastic.co/t/how-to-insert-only-newly-added-rows-in-csv-in-logstash-in-old-index/278866/2 "2021-07-16T07:14:13Z")

</div>

Hello,  
you can add 7 columns from the beginning of the index or you can also add field after creation of index also by adding field in index pattern.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/2/a2565fb80d4a68cfe892d1d214c85b70c86caea4.png)

---

<div class="post-metadata">

**Author:** ![Sarthak\_Mishra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sarthak_mishra/32/82772_2.png) [@Sarthak\_Mishra](https://discuss.elastic.co/u/Sarthak_Mishra)\
**Post date:** [July 16, 2021, 7:32am UTC](https://discuss.elastic.co/t/how-to-insert-only-newly-added-rows-in-csv-in-logstash-in-old-index/278866/3 "2021-07-16T07:32:07Z")

</div>

Cant I simply ingest the updated csv and it only adds the new rows in the index.  
Also can you please elaborate on your solution.  
Thanks a lot

---

<div class="post-metadata">

**Author:** ![Krunal](https://avatars.discourse-cdn.com/v4/letter/k/ce7236/32.png) [@Krunal](https://discuss.elastic.co/u/Krunal)\
**Post date:** [July 16, 2021, 12:23pm UTC](https://discuss.elastic.co/t/how-to-insert-only-newly-added-rows-in-csv-in-logstash-in-old-index/278866/4 "2021-07-16T12:23:44Z")

</div>

it will create duplicate rows... so you need to delete your index each time and again you have to create index.... normally you can create your updated csv for every day and ingest it.  
No. of rows update automatically but you must maintain the field = columns of the table.

---

<div class="post-metadata">

**Author:** ![Sarthak\_Mishra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sarthak_mishra/32/82772_2.png) [@Sarthak\_Mishra](https://discuss.elastic.co/u/Sarthak_Mishra)\
**Post date:** [July 16, 2021, 1:23pm UTC](https://discuss.elastic.co/t/how-to-insert-only-newly-added-rows-in-csv-in-logstash-in-old-index/278866/5 "2021-07-16T13:23:04Z")

</div>

OK noted. Can you please confirm that my desired test case be done or not with the solution that I have proposed.

Thank you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 13, 2021, 1:23pm UTC](https://discuss.elastic.co/t/how-to-insert-only-newly-added-rows-in-csv-in-logstash-in-old-index/278866/6 "2021-08-13T13:23:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
