# Import only delta (differences) from csv into mysql

**URL:** <https://discuss.elastic.co/t/import-only-delta-differences-from-csv-into-mysql/289283>\
**Category:** Logstash\
**Created:** [November 16, 2021, 8:54am UTC](https://discuss.elastic.co/t/import-only-delta-differences-from-csv-into-mysql/289283 "2021-11-16T08:54:20Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mihai\_Stuparu](https://avatars.discourse-cdn.com/v4/letter/m/bc8723/32.png) [@Mihai\_Stuparu](https://discuss.elastic.co/u/Mihai_Stuparu)\
**Post date:** [November 16, 2021, 8:54am UTC](https://discuss.elastic.co/t/import-only-delta-differences-from-csv-into-mysql/289283/1 "2021-11-16T08:54:20Z")

</div>

Hello,  
I am using logstash to call an API (input), save it as .csv (filter) and upload it into a mysql table (output). All is good.  
But I need to do it each day (or each night rather): call that API, get the .csv and upload it in mysql.  
For uploading I am using a simple:  
statement =\> INSERT INTO ... VALUES.  
This will duplicate the existing entries.

The question is: which is your advise, the best practice, to only upload the delta, the differences: so what is new in .csv file, but does not already exists in mysql.  
I don't have primary keys on mysql, probably I need to create one from 3 columns, there is no id, no unique identifier, rows values can appear multiple time. Example of the data from csv\mysql:

ColumnA ColumnB ColumnC ColumnD ColumnE  
ValueA ValueB ValueC ValueD ValueE  
ValueA ValueB ValueC1 ValueD1 ValueE  
ValueA ValueB ValueC ValueD1 ValueE1  
ValueA ValueB1 ValueC2 ValueD2 ValueE2

---

<div class="post-metadata">

**Author:** ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)\
**Post date:** [November 17, 2021, 1:19pm UTC](https://discuss.elastic.co/t/import-only-delta-differences-from-csv-into-mysql/289283/2 "2021-11-17T13:19:45Z")

</div>

Yes, create a primary key and then put a constraint on that column in the SQL database so that it will not insert a duplicate. It will throw an error when you do try to do that but that error can be caught and discarded.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 15, 2021, 1:20pm UTC](https://discuss.elastic.co/t/import-only-delta-differences-from-csv-into-mysql/289283/3 "2021-12-15T13:20:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
