# Records missing while stashing data from Logstash

**URL:** <https://discuss.elastic.co/t/records-missing-while-stashing-data-from-logstash/115704>\
**Category:** Logstash\
**Created:** [January 16, 2018, 12:11pm UTC](https://discuss.elastic.co/t/records-missing-while-stashing-data-from-logstash/115704 "2018-01-16T12:11:53Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![vinaygarg30](https://avatars.discourse-cdn.com/v4/letter/v/6bbea6/32.png) [@vinaygarg30](https://discuss.elastic.co/u/vinaygarg30)\
**Post date:** [January 16, 2018, 12:11pm UTC](https://discuss.elastic.co/t/records-missing-while-stashing-data-from-logstash/115704/1 "2018-01-16T12:11:53Z")

</div>

I am using logstash 6.1.0 to make a data pipeline to stash data from DEV server(where mysql data is present using jdbc input plugin) to TEST server( where elastic search in installed).  
The config file for logstash is as follows:  
input {  
jdbc {  
jdbc\_driver\_library =\> "/home/vinay/Applications/mysql-connector-java-5.1.45/mysql-connector-java-5.1.45-bin.jar"  
jdbc\_driver\_class =\> "com.mysql.jdbc.Driver"  
jdbc\_connection\_string =\> "jdbc:mysql://x.y.w.z:3306/dummyTable" //Connection String  
jdbc\_user =\> "**"  
jdbc\_password =\> "**"  
statement\_filepath =\> "query.sql"  
lowercase\_column\_names =\> false  
}  
}  
output {  
stdout { codec =\> json\_lines }  
elasticsearch {  
index =\> "index1"  
document\_id =\> "%{account\_id}"   
hosts =\> ["x.x.x.x:9200"] //TEST Server ES IP  
}  
}

The query statement is :  
SELECT  
opp.opportunity\_id AS opportunity\_id,  
opp.opportunity\_name AS opportunity\_name,  
acc.account\_id AS account\_id,  
acc.account\_name AS account\_name,  
acc.region AS region,  
prd.base\_uom AS product\_uom,  
prd.isGrowth AS is\_growth,  
prd.isActive AS is\_active,  
opp.est\_revenue AS est\_revenue,  
opp.opportunity\_score AS opportunity\_score,  
opp.opp\_state AS opp\_state,  
opp.est\_close\_date AS est\_close\_date  
FROM  
opportunity\_new AS opp  
LEFT JOIN  
account1 acc ON opp.account\_id = acc.account\_id  
LEFT JOIN  
product\_new prd ON opp.product\_id = prd.product\_id  
ORDER BY opportunity\_score;

The total number of records using this query is 6658 records in mysql.  
But, when i index my data from DEV(mysql) to TEST(elastic search), some records are missing. The count is little less than the original count.

When i try to do the same from DEV(mysql database) to my local ES, the number of records are perfect(same as original).

Please suggest a solution.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 13, 2018, 12:12pm UTC](https://discuss.elastic.co/t/records-missing-while-stashing-data-from-logstash/115704/2 "2018-02-13T12:12:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.